cryptoe commented on code in PR #13707: URL: https://github.com/apache/druid/pull/13707#discussion_r1091477874
########## docs/multi-stage-query/reference.md: ########## @@ -602,6 +602,7 @@ The following table lists the context parameters for the MSQ task engine: | `faultTolerance` | SELECT, INSERT, REPLACE<br /><br /> Whether to turn on fault tolerance mode or not. Failed workers are retried based on [Limits](#limits). Cannot be used when `durableShuffleStorage` is explicitly set to false. | `false` | | `composedIntermediateSuperSorterStorageEnabled` | SELECT, INSERT, REPLACE<br /><br /> Whether to enable automatic fallback to durable storage from local storage for sorting's intermediate data. Requires to setup `intermediateSuperSorterStorageMaxLocalBytes` limit for local storage and durable shuffle storage feature as well.| `false` | | `intermediateSuperSorterStorageMaxLocalBytes` | SELECT, INSERT, REPLACE<br /><br /> Whether to enable a byte limit on local storage for sorting's intermediate data. If that limit is crossed, the task fails with `ResourceLimitExceededException`.| `9223372036854775807` | +| `maxInputBytesPerWorker` | When dividing the input to a stage into slices to be assigned to workers, this parameter determines the maximum size in bytes for an input slice. This parameter is only used as a guideline during input slicing, and does not guarantee that a slice is not larger, for example when the number of workers and max number of slices per worker limits in [Limits](#limits) conflict with this. This value is used for all stages in a query. | `10 GB` | Review Comment: ```suggestion | `maxInputBytesPerWorker` | Should be used in conjunction with taskAssignment `auto` mode. When dividing the input of a stage among the workers, this parameter determines the maximum size in bytes that are given to a single worker before the next worker is chosen. This parameter is only used as a guideline during input slicing, and does not guarantee that a the input cannot be larger. For example, we have 3 files. 3 ,7 ,12 GB each. then we would end up using 2 worker W1-> 3,7 and W2-> 12. This value is used for all stages in a query. | `10737418240` | ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
