rohangarg commented on code in PR #13368:
URL: https://github.com/apache/druid/pull/13368#discussion_r1081147729
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:
##########
@@ -562,13 +563,30 @@ private QueryDefinition initializeQueryDefAndState(final
Closer closer)
.orElse(MSQWarnings.DEFAULT_MAX_PARSE_EXCEPTIONS_ALLOWED);
}
-
+ ImmutableMap.Builder<String, Object> taskContextOverridesBuilder =
ImmutableMap.builder();
+ taskContextOverridesBuilder
+ .put(
+ MultiStageQueryContext.CTX_ENABLE_DURABLE_SHUFFLE_STORAGE,
+ isDurableStageStorageEnabled
+ ).put(
+
MultiStageQueryContext.CTX_COMPOSED_INTERMEDIATE_SUPER_SORTER_STORAGE,
+
MultiStageQueryContext.isComposedIntermediateSuperSorterStorageEnabled(
+ task.getQuerySpec().getQuery().context()
+ )
+ ).put(
Review Comment:
Yes, I agree with the documentation part but setting
`CTX_INTERMEDIATE_SUPER_SORTER_STORAGE_MAX_LOCAL_BYTES` is also not enough for
composition. The user would also have to set the durable storage setting and
all the configurations along with it. And in future, if any memory storage is
allowed in composition, then that is needed too.
So, I thought the documentation for the composition parameter would clearly
state what all is needed to configure it properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
