kfaraz commented on PR #14916: URL: https://github.com/apache/druid/pull/14916#issuecomment-1700312691
> I wasn't aware of this assumption elsewhere, but if thats the case then I think those places should probably change - so I'd go with a slight modification of your option 1 - all partitioning modes should respect the value of forceGuaranteedRollup if it they can (i.e. its still an error to ask for guaranteed rollup and dynamic partitioning, but otherwise we can respect the user's preference) I think there is some confusion here because of the two flags: - The flag [`rollup`](https://druid.apache.org/docs/latest/ingestion/ingestion-spec#granularityspec) in `granularitySpec` (default is `true`) is used to control whether we want to rollup data or not. - The flag [`forceGuaranteedRollup`](https://druid.apache.org/docs/latest/ingestion/native-batch-simple-task#tuningconfig) in `tuningConfig` (default is `false`) controls the type of rollup i.e. perfect or best-effort. - `hashed` and `range` partitioning _always_ perform perfect rollup (if `rollup` is enabled), same way as `dynamic` never does. - So while using `range` or `hashed`, `forceGuaranteedRollup` has to be `true`, because they always give perfect rollup. Thus the options are to either (a) ignore the value of `forceGuaranteedRollup` and use perfect rollup _only_ if the user has asked for rollup, or (b) throw an error. - Users can still choose to have data that is not rolled up by simply setting `rollup` to `false`. <img width="959" alt="Screenshot 2023-08-31 at 8 41 14 AM" src="https://github.com/apache/druid/assets/18635897/c5c58814-572c-4a37-8451-ee661edc86d4";> I think we should update the docs to clarify that for Native batch, the _kind_ of rollup we get depends on the partitioning type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
