loquisgon commented on a change in pull request #11912:
URL: https://github.com/apache/druid/pull/11912#discussion_r747861427
##########
File path: docs/development/extensions-core/kafka-ingestion.md
##########
@@ -41,19 +46,22 @@ Additionally, you can set `isolation.level` to
`read_uncommitted` in `consumerPr
If your Kafka cluster enables consumer-group based ACLs, you can set
`group.id` in `consumerProperties` to override the default auto generated group
id.
-## Submitting a Supervisor Spec
-
-To use the Kafka indexing service, load the `druid-kafka-indexing-service`
extension on both the Overlord and the MiddleManagers. Druid starts a
supervisor for a dataSource when you submit a supervisor spec. You can use the
following endpoint:
+## Load the Kafka indexing service
-`http://<OVERLORD_IP>:<OVERLORD_PORT>/druid/indexer/v1/supervisor`
+To use the Kafka indexing service, load the `druid-kafka-indexing-service`
extension on both the Overlord and the MiddleManagers. See [Loading
extensions](../extensions.md#loading-extensions) for instructions on how to
configure extensions.
-For example:
+## Define a supervisor spec
+Similar to the ingestion spec for batch ingestion, the supervisor spec
configures the data ingestion for Kafka streaming ingestion. A supervisor spec
has the following sections:
+- `dataSchema` to specify the Druid datasource name, primary timestamp,
dimensions, metrics, transforms, and any necessary filters.
+- `ioConfig` to configure Druid to connect to Kafka how to parse the data.
Kafka-specific connection details go in the `consumerProperties`. For more
information, see the [Kafka supervisor
reference](./kafka-supervisor-reference.md).
+- `tuningConfig` to control various tuning parameters specific to each
ingestion method.
+For a full description of all the fields and parameters in a Kafka supervisor
spec, see the [Kafka supervisor reference](./kafka-supervisor-reference.md).
+For information on how to configure the input format, see [Data
formats](../../ingestion/data-formats.md).
Review comment:
This brings the "input format" concept up front with no explanation...
maybe it is better to introduce it saying it is part of the `tuningConfig` and
maybe succinctly summarize what it is for? (i.e. specify the data format of
input data)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
