Tartarus0zm commented on code in PR #1532:
URL: https://github.com/apache/auron/pull/1532#discussion_r2471581642
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/SparkUDTFWrapperContext.scala:
##########
@@ -58,8 +62,11 @@ case class SparkUDTFWrapperContext(serialized: ByteBuffer)
extends Logging {
}
private val tc = TaskContext.get()
- private val batchSize = AuronConf.BATCH_SIZE.intConf()
- private val maxBatchMemorySize = AuronConf.SUGGESTED_BATCH_MEM_SIZE.intConf()
+ private val sparkAuronConfig: AuronConfiguration =
Review Comment:
> Should this be a method on AuronAdaptor? All the invocations of the
adaptor to get the configuration requests a `getInstance` call; we can abstract
this away by adding `getSparkAuronConfig()` as a public method in the Adaptor,
WDYT?
@ShreyeshArangath thanks for your comment. This is an excellent suggestion.
I suggest we open a separate issue to discuss this. We introduced the
AuronAdaptor to abstract away specific engines like Spark, Flink, and Trino, so
the API does not carry the engine name. Let's discuss this issue in #1535.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
