Activate knowledge object discovery for federated searches of DDSS datasets

Arrange for federated searches of DDSS datasets to perform search-time event processing that discovers knowledge objects in your data like extracted fields, aliases, event types, and tags.

When you activate Knowledge object discovery at search time for your Dynamic Data Self Storage (DDSS) dataset, federated searches of that dataset run processes at search time that extract fields and discover useful knowledge objects such as field aliases, calculated fields, event types, and tags. These processes rely on source types that Splunk software infers from your DDSS dataset when you create it.

Because DDSS data does not pass through the standard Splunk platform indexing pipeline, it does not automatically get the index-time field extraction and event processing that you typically see with Splunk-indexed data. Knowledge object discovery therefore gives you a federated search experience that is comparable to what you might expect from searches over data that you have indexed into your Splunk Cloud Platform deployment.

Without this activation, this event processing and knowledge object discovery does not take place at search time. You can still run useful federated searches, but the results of those searches won't be enriched by the search-time field extractions and event metadata that knowledge object discovery can provide.

Note:

Knowledge object discovery for federated search of DDSS datasets currently does not support transform-based field extraction, automatic key-value field extraction, or lookups.

Knowledge object discovery for federated search of DDSS datasets also does not support knowledge object configurations for field extractions, field aliases, or calculated fields that are scoped to specific host or source values. This feature supports only source-type-scoped configurations.

Activate search-time knowledge object discovery

You can turn on search-time knowledge object discovery for a DDSS dataset at any time. You can activate knowledge object discovery when you create it, on the Configure dataset step. You can also activate knowledge object discovery for a DDSS dataset after it is created, through the Edit page for the dataset.

Note: When Knowledge object discovery at search time is activated for a DDSS dataset, federated searches of that dataset might take longer to complete.

To learn more about activating Knowledge object discovery at search time during DDSS dataset creation, see Define a DDSS dataset.

If you need to activate Knowledge object discovery at search time for a DDSS dataset that currently exists, see Manage a DDSS dataset.

Source type inference and knowledge object discovery

When you create your dataset, Splunk software runs a crawler in the background that infers the schema of your DDSS dataset. This crawler also infers the source types in your DDSS dataset. Source types provide the context that Splunk software requires for knowledge object discovery, starting with search-time field extraction.

When you run a federated search over the dataset, Splunk software uses the inferred source types to apply matching extraction rules to the raw events at search time. These extraction rules produce searchable fields without requiring you to restore or reindex your remotely-stored DDSS data.

Extracting the expected fields first allows Splunk software to apply the relevant knowledge objects to your federated data. As a result, you can run familiar searches over your federated data that include normalized field names, calculated fields, and event type categorization, much as you do with data locally indexed in your Splunk platform deployment.

See The sequence of search time operations in the Knowledge Management Manual.

Obtain source types for your DDSS searches

Ideally, federated searches of DDSS datasets should identify the sourcetype values that are associated with the extracted fields and knowledge objects referenced by those searches, as they might complete faster and scan less data than they would without sourcetype identification.

After your DDSS dataset is created, you can find the inferred sourcetype values that you need for your federated searches. On the Datasets listing page, select the DDSS dataset that you want to search, and then look in the right-hand information sidebar for the Sourcetype section. This section lists the sourcetype values that the crawler service inferred for the DDSS dataset.

When you write federated searches of the DDSS dataset, use WHERE clauses to reference sourcetype values.

Write federated searches for search-time knowledge object discovery

When you have Knowledge object discovery at search time activated for a DDSS dataset, you can run federated searches of that dataset that utilize extracted fields, field aliases, calculated fields, event types, and tags.

To take advantage of search-time knowledge object discovery, the federated searches you write must reference the extracted fields, field aliases, and calculated fields that you want to see in the results.

Note: Search-time knowledge object discovery for DDSS federated search can only extract fields that are referenced in the search, with one exception. If you run SELECT * while in Verbose mode, Splunk software extracts all possible fields in the search results.
Here is an example of a federated search that selects 5 extracted fields.
CODE 
SELECT _raw, EventCode, ComputerName, LogName, TaskCategory, Action
FROM my_ddss_index1 
WHERE sourcetype = "WinEventLog"
LIMIT 300

The following federated search includes 3 field alias values. The vendor_product field is an alias of product, the signature_id field is an alias of code, the vendor_severity field is an alias of severity.

CODE 
SELECT _raw, product, code, severity, vendor_product, signature_id, vendor_severity
FROM my_ddss_index1
WHERE sourcetype = "buttercup_games"
LIMIT 300
This is an example of a search you might run to verify that configured knowledge objects are being discovered as expected.

For more information about federated searches, see Write and run federated searches over federated datasets with SPL2.

Configuring knowledge objects for federated searches

Before they can appear in your search results, search-time field extractions, field aliases, calculated fields, event types and tags must be configured in your Splunk Cloud Platform deployment. The following table lists the knowledge objects currently supported for federated search of remote DDSS datasets and links you to guidance for setting up additional configurations.

Knowledge object type Description Splunk Web setup guidance
Inline field extraction A field extraction configuration that extracts a specific field or set of fields.

Navigate to Settings > Fields > Field extractions to review existing search-time field extraction configurations with a Type of Inline.

On the Field extractions listing page, select Add new to define an additional field extraction for a specific source type. See Use the Field extractions page in the Knowledge Management Manual.

Note: When you add a new field extraction with the Field extractions page, note that the field extraction Type must be set to Inline. The Knowledge object discovery feature for federated DDSS searches does not support transform-based field extractions.
Field aliases Alternate field names that map to existing fields to support field name normalization.

Navigate to Settings > Fields > Field aliases to review existing field alias configurations.

On the Field aliases listing page, select New field alias to define an additional field alias for a specific source type. See Create field aliases in Splunk web in the Knowledge Management Manual.
Calculated fields

Configurations that create one or more fields through the calculation of eval expressions and add those fields to events.

The eval expression can use values of fields that are already present in the event due to index-time or search-time field extraction processes.

Navigate to Settings > Fields > Calculated fields to review existing calculated field configurations.

On the Calculated fields listing page, select New Calculated Field to create an additional calculated field for a specific source type. See Create calculated fields with Splunk Web in the Knowledge Management Manual.
Event types Named searches that classify events into reusable categories.

Go to Settings > Event types to review existing event type configurations.

On the Event types listing page, select New Event Type to define an event type for a specific source type. See Define event types in Splunk Web in the Knowledge Management Manual.
Tags Labels that you apply to field and value pairs to facilitate efficient searches and data normalization.

Navigate to Settings > Tags > List by field value pair List by tag name, or All tag objects to review existing tag configurations.

On a tags listing page, select New Tag to create new tags for your searches. See Define and manage tags in Settings in the Knowledge Management Manual.

See also

Related information

The sequence of search time operations in the Knowledge Manager Manual.