petermarshallio opened a new issue #11980: URL: https://github.com/apache/druid/issues/11980
**In documentation, ioConfig configuration options are split into different ingestion task types, introducing potential hops between pages, and potentially giving conflicting information when a reader searches for a config option that is common across ingestion types and described differently.** ### Affected Version 0.22.0 ### Description - `ioConfig` documentation is split across five pages: https://druid.apache.org/docs/latest/ingestion/native-batch.html#ioconfig https://druid.apache.org/docs/latest/development/extensions-core/kafka-ingestion.html#kafkasupervisorioconfig https://druid.apache.org/docs/latest/development/extensions-core/kinesis-ingestion.html#kinesissupervisorioconfig https://druid.apache.org/docs/latest/ingestion/hadoop.html#ioconfig https://druid.apache.org/docs/latest/ingestion/data-management.html#compaction-ioconfig - There are 8 options common to both kinesis and kafka ingestion that are duplicated - Some addition options are unique to kafka (6) and some unique to kinesis (8) but they're difficult to find - inputFormat is common to Kinesis, Kafka, and Batch but contains deprecated Parser information - Some options are only available in index_parallel, compact, and hadoop-type jobs ### Proposal Based on the presumption that learners are interested in tuning options specific to their own ingestion source systems: - New "Ingestion connections" section after "Ingestion spec" - New intro page begins with definition of ioConfig and has links out to hadoop and compaction ioConfig sections - New "Stream connections" page begins with the 8 common options, with sections for the Kafka and Kinesis-specific options - New "Batch connections" page focuses on index_parallel options only - Existing [Data Formats](https://druid.apache.org/docs/latest/ingestion/data-formats.html) page revisited to focus on `inputFormat` only, e.g. parser section could be separated and links added to the Stream connections page as "deprecated" - Refine the config options according to their functional effect with common headings across the batch and streaming pages. - Refine definitions for all config options, explaining differences by exception. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
