zhuzhurk commented on a change in pull request #10452:
[FLINK-15035][table-planner-blink] Introduce unknown memory setting to table in
blink planner
URL: https://github.com/apache/flink/pull/10452#discussion_r354660349
##########
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/exec/BatchExecNode.scala
##########
@@ -32,4 +33,13 @@ trait BatchExecNode[T] extends ExecNode[BatchPlanner, T]
with Logging {
*/
def getDamBehavior: DamBehavior
+ def setManagedMemoryWeight[X](
+ transformation: Transformation[X], memoryBytes: Long): Transformation[X]
= {
+ // Using Bytes can easily overflow
+ // Using MebiBytes to cast to int
+ // Careful about zero
+ val memoryMB = Math.max(1, (memoryBytes >> 20).toInt)
Review comment:
An alternative is to make the min weight to 2 if `memoryBytes` > 0.
In this way we do not need to set the weight for operators that do not use
managed memory, by overriding value 1 weights to 0 in
`ExecutorUtils#setBatchProperties`, like what we do for ResourceSpec.
It's a bit hacky though. But cannot be easily replaced once job wide default
ResourceSpec is supported.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
