LakshSingla commented on code in PR #15007:
URL: https://github.com/apache/druid/pull/15007#discussion_r1332728217
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerMemoryParameters.java:
##########
@@ -130,11 +131,11 @@ public class WorkerMemoryParameters
private static final long SMALL_WORKER_CAPACITY_THRESHOLD_BYTES =
256_000_000;
/**
- * Fraction of free memory per bundle that can be used by {@link
org.apache.druid.msq.querykit.BroadcastJoinHelper}
- * to store broadcast data on-heap. This is used to limit the total size of
input frames, which we expect to
- * expand on-heap. Expansion can potentially be somewhat over 2x: for
example, strings are UTF-8 in frames, but are
- * UTF-16 on-heap, which is a 2x expansion, and object and index overhead
must be considered on top of that. So
- * we use a value somewhat lower than 0.5.
+ * Fraction of free memory per bundle that can be used by {@link
BroadcastJoinSegmentMapFnProcessor} to store broadcast
+ * data on-heap. This is used to limit the total size of input frames, which
we expect to expand on-heap. Expansion
+ * can potentially be somewhat over 2x: for example, strings are UTF-8 in
frames, but are UTF-16 on-heap, which is
+ * a 2x expansion, and object and index overhead must be considered on top
of that. So we use a value somewhat
+ * lower than 0.5.
*/
static final double BROADCAST_JOIN_MEMORY_FRACTION = 0.3;
Review Comment:
Passing comment: Since the broadcast table is computed once, perhaps we can
rework the space allocated for the broadcast to be a fraction of the whole than
per bundle?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
