clintropolis commented on a change in pull request #11536:
URL: https://github.com/apache/druid/pull/11536#discussion_r704050643
##########
File path: docs/configuration/index.md
##########
@@ -1343,7 +1343,7 @@ Additional peon configs include:
|`druid.peon.mode`|Choices are "local" and "remote". Setting this to local
means you intend to run the peon as a standalone process (Not
recommended).|remote|
|`druid.indexer.task.baseDir`|Base temporary working
directory.|`System.getProperty("java.io.tmpdir")`|
|`druid.indexer.task.baseTaskDir`|Base temporary working directory for
tasks.|`${druid.indexer.task.baseDir}/persistent/task`|
-|`druid.indexer.task.useLegacyBatchProcessing`|If false, native batch
ingestion will use a new, recommended, code path with memory optimized code for
the segment creation phase. If true it will use the previous code path for the
create segments phase of batch ingestion. This does not apply to streaming
ingestion, just to batch. This setting should only be used when a bug is
suspected or found in the new optimized batch ingestion code. If a bug is
suspected or found, you can set this flag to `true` to fall back to previous,
working but more memory intensive, code path.|`false`|
+|`druid.indexer.task.batchProcessingMode`| Batch ingestion tasks have three
operating modes that control how intermediary segments are constructed and
tracked: `OPEN_SEGMENTS`, `CLOSED_SEGMENTS`, and `CLOSED_SEGMENT_SINKS`.
`OPEN_SEGMENTS` will use code based on the original batch ingestion path and
performs a `mmap` on intermediary segments to build a timeline so that these
segments can be queryable by realtime queries. This is not needed at all for
batch, so the default mode, `CLOSED_SEGMENTS`, eliminates `mmap` of
intermediary segments, but still tracks the entire set of segments in heap. The
`CLOSED_SEGMENTS_SINKS` mode is the most aggressive and should have the
smallest memory footprint, and works by eliminating in memory tracking and mmap
of intermediary segments produced during segment creation. This mode isn't as
well tested as other modes so is currently considered experimental.
`OPEN_SEGMENTS` mode can be selected if any problems occur with the 2 newer
modes. |`CLOSED_SEGM
ENTS`|
Review comment:
```suggestion
|`druid.indexer.task.batchProcessingMode`| Batch ingestion tasks have three
operating modes that control how intermediary segments are constructed and
tracked: `OPEN_SEGMENTS`, `CLOSED_SEGMENTS`, and `CLOSED_SEGMENT_SINKS`.
`OPEN_SEGMENTS` will use code based on the original batch ingestion path and
performs a `mmap` on intermediary segments to build a timeline so that these
segments can be queryable by realtime queries. This is not needed at all for
batch, so the default mode, `CLOSED_SEGMENTS`, eliminates `mmap` of
intermediary segments, but still tracks the entire set of segments in heap. The
`CLOSED_SEGMENTS_SINKS` mode is the most aggressive and should have the
smallest memory footprint, and works by eliminating in memory tracking and
`mmap` of intermediary segments produced during segment creation. This mode
isn't as well tested as other modes so is currently considered experimental.
`OPEN_SEGMENTS` mode can be selected if any problems occur with the 2 newer
modes. |`CLOSED_
SEGMENTS`|
```
should fix spellcheck CI failure i think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
