mbutrovich opened a new pull request, #5155: URL: https://github.com/apache/datafusion-comet/pull/5155
## Which issue does this PR close? Closes #. ## Rationale for this change Ahead of the 1.0 release, Comet's logging should be consistent with Spark's own conventions: informational messages that fire once per query or plan node are fine at INFO, but messages that fire once per task should not be, since they scale with partition count and can produce an unreasonable number of log lines for large jobs. Spark's `FileFormatWriter.executeTask` has no equivalent per-task success log, only a `logError` on abort, so Comet's per-task INFO logs are out of step with that convention. ## What changes are included in this PR? - `CometExecIterator.getMemoryConfig` logged the computed memory pool configuration at INFO on every task (it runs once per `CometExecIterator` construction, i.e. once per native-exec task per stage). Changed to `logDebug`. - `CometNativeWriteExec` logged `"Task ... committed successfully"` at INFO for every successful write task. Changed to `logDebug` to match Spark's `FileFormatWriter`, which stays silent on task-commit success. ## How are these changes tested? No behavior change, only log levels. Existing test suite covers the surrounding code paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
