shrirangmhalgi commented on code in PR #57259:
URL: https://github.com/apache/spark/pull/57259#discussion_r3600940103
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/AsyncProgressTrackingMicroBatchExecution.scala:
##########
@@ -61,8 +62,15 @@ class AsyncProgressTrackingMicroBatchExecution(
// writes to execute in order in a serialized fashion
protected val asyncWritesExecutorService
= ThreadUtils.newDaemonSingleThreadExecutorWithRejectedExecutionHandler(
- "async-log-write",
- 2, // one for offset commit and one for completion commit
+ AsyncProgressTrackingMicroBatchExecution.ASYNC_LOG_WRITE_THREAD_NAME,
+ if (trigger.isInstanceOf[RealTimeTrigger]) {
+ // Two tasks can be buffered, one in the active task and one in the
queue.
Review Comment:
nit: comment says "Two tasks can be buffered" but the parameter value is `1`
(the queue capacity). The comment is describing system capacity (1 active + 1
queued = 2 in-flight), not the parameter. Slightly confusing - consider: `1 //
queue capacity; with the active task, at most 2 writes in flight`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
