Configure physical separation of indexing and ingestion with SOK

Configure a new SOK-managed topology that physically separates ingestion and indexing by using the shared SmartBus queue, object storage, and separately managed indexer cluster.

Important:

Physical separation is supported only for new physical-separation topologies. Support for configuring physical separation on an existing SOK-managed deployment has not been established. Existing configuration and data are not automatically migrated or made available through the new configuration.

Before configuring physical separation, verify that your deployment meets the following requirements:

  • Splunk Enterprise version 10.6 or higher.

  • A Splunk Enterprise deployment that uses the Bring Your Own License (BYOL) model and runs on a customer managed platform (CMP) for Kubernetes.

  • A compatible version of Splunk Operator for Kubernetes (SOK).

  • A supported Kubernetes environment.

  • A separate indexer cluster and a separate, SOK-managed IngestorCluster.

  • The following customer-provisioned AWS resources:

    • An Amazon Simple Queue Service (SQS) queue, including a dead-letter queue.

    • An Amazon Simple Storage Service (S3) bucket for ingestion data that exceeds the queue message-size limit.

    • An AWS Key Management System (KMS) key if encryption is required for the ingestion object store.

    For multisite deployments, configure the queue and ingestion object-storage location according to the requirements for each site.

  • Service-account permissions for the queue, dead-letter queue, and object store.

  • Network connectivity between the SOK-managed IngestorCluster, the separate indexer cluster, and the AWS services.

Follow these steps to configure a new physical separation topology:

  1. Deploy the SOK-managed Queue, ObjectStorage, and IngestorCluster resources.

    Use one of the following methods:

    Note:

    You cannot change the configuration values in the Queue and ObjectStorage resources after creation. However, you can change the queueRef and objectStorageRef references in the IngestorCluster. When a reference changes, SOK regenerates the configuration resources used by the ingestion tier and updates the Kubernetes StatefulSet that manages the IngestorCluster pods.

    Configure the resource references so that traffic is routed correctly. Ensure that the IngestorCluster includes references to the shared Queue and ObjectStorage resources. SOK uses these references to configure SmartBus for the ingestion tier.

  2. Integrate the separate indexer cluster.

    Configure the separate indexer cluster to use the same SmartBus queue and object store as the IngestorCluster.

    Configure the indexing tier to:

    • Retrieve processed data asynchronously.

    • Configure handling of the dead-letter queue on the indexer side.

    • Index the retrieved data and make it available for search.

    • Maintain the required cluster and replication settings.

    For information about configuring the separately managed indexer cluster, see Indexer cluster configuration overview.

  3. Configure credential references for the SOK-managed resources.

    SOK delivers structural configuration separately from credentials. If you configure secretKeyRef, store the referenced Secrets in the same namespace as the resource that uses them.

    If you omit secretKeyRef, configure the pods to use IAM Roles for Service Accounts (IRSA) or another supported workload-identity mechanism. SOK manages the underlying configuration files, resources, and credential-delivery paths.

  4. Configure access to the SQS queue and S3 bucket.

    Configure each tier with the credentials required to access the queue, dead-letter queue, and object store. and, if applicable, encryption key. Grant only the permissions required for the configured resources. Use one of the following methods:

    • Workload identity - Configure a Kubernetes service account linked to a cloud-provider identity. This allows the pods to obtain temporary credentials without storing static credentials in Kubernetes Secrets. Ensure that the linked cloud identity has the required permissions.

      Example: This example creates the ingestor-sa service account by using the eksctl utility and attaches a customer-managed policy named ExamplePolicy.
      CODE 
      eksctl create iamserviceaccount \                                                                                                                                          
  --name ingestor-sa \
  --cluster ind-ing-sep-demo \
  --region us-west-2 \
  --attach-policy-arn arn:aws:iam::<account-id>:policy/ExamplePolicy \
  --approve \
  --override-existing-serviceaccounts

    • Static AWS credentials - Where required, configure credentials using the secretKeyRef parameter of the Queue resource ( Queue.spec.sqs.secretKeyRef).

    Configure credentials for the separately managed indexer cluster using its supported credential mechanism.

Physical separation is configured.

After you apply the configuration, SOK automatically performs the following actions:

  • Uses the resource references and configures SmartBus for the ingestion tier.

  • Regenerates the configuration when Queue or ObjectStorage references change.

  • Updates the corresponding StatefulSet .

  • Monitors the referenced credential Secrets.

  • Creates a new credential configuration and performs a rolling restart when credentials change.