zhztheplayer commented on code in PR #11032:
URL:
https://github.com/apache/incubator-gluten/pull/11032#discussion_r2499363950
##########
gluten-arrow/src/main/scala/org/apache/gluten/runtime/Runtimes.scala:
##########
@@ -18,15 +18,25 @@ package org.apache.gluten.runtime
import org.apache.spark.task.TaskResources
+import java.util
+
object Runtimes {
/** Get or create the runtime which bound with Spark TaskContext. */
- def contextInstance(backendName: String, name: String): Runtime = {
+ def contextInstance(
Review Comment:
It turns out that we might have to add a new API, something like the
following and use it for creating runtime instances with extra configuration,
because these configurations we pass down to native might be different for
different invocations.
The original `contextInstance` finds existing instance in the cache so is
fast. But if the operation is relatively heavy, like when we create plan
evaluator instances, probably it's tolerable to create a new instance every
single time.
```scala
def contextInstance(backendName: String, name: String, extraConf:
Map[String, String]): Runtime = {
if (!TaskResources.inSparkTask()) {
throw new IllegalStateException("This method must be called in a Spark
task.")
}
TaskResources.addResource(s"$backendName:$name:$extraConf",
Runtime(backendName, name, extraConf))
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
