andygrove commented on code in PR #2462:
URL: https://github.com/apache/datafusion-comet/pull/2462#discussion_r2383448563
##########
spark/src/main/scala/org/apache/comet/CometExecIterator.scala:
##########
@@ -93,6 +93,14 @@ class CometExecIterator(
}
val protobufSparkConfigs = builder.build().toByteArray
+ val memoryLimitPerTask = if (offHeapMode) {
+ // this per-task limit is not used in native code when using unified
memory
+ // so we can skip calculating it and avoid logging irrelevant information
+ 0
Review Comment:
Yes, this code will get called regardless of on-heap vs off-heap. We compute
the value in JVM and pass into the native code, but the native code only uses
this value if the mode is on-heap. The API here is not very pretty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
