LuciferYang commented on PR #58343: URL: https://github.com/apache/spark/pull/58343#issuecomment-5675709126
Thanks, both settled. 1. Went with the follow-up. The `nonEmpty` guards, the comment and the three empty-value test rows are gone, so the PR no longer takes a position on empty values and the write path is untouched. That also lines the two steps up: the driver step does not filter empty values either, and now neither does this one. Filed SPARK-59518 for the write path and the doc mismatch. 2. Lowered to `logDebug`. I preferred that over once-per-application because the flag would have to be static on a step that is rebuilt per pod, which then leaks into test ordering. The WARN stays per pod. One wrinkle if you read the test: `LogAppender` drops anything below INFO on its own threshold, so the DEBUG case needs `appender.setThreshold(Level.DEBUG)` on top of the logger level. 3. `configurePod` is a single `match` on `podServiceAccount(pod)` now, so the identity copy is gone and the read happens once. The suite has one `messagesAt(appender, level)`. The restating sentence is out of both table rows. 4. Added the migration guide entry, under "Upgrading from Core 4.2 to 4.3" with "Since Spark 4.3". The wording turns out not to be awkward, because the backport stops at `branch-4.3`: SPARK-58872 only reached `master`, `branch-4.x` and `branch-4.3`, so on `branch-4.2` and older this would need the driver step hunk dropped and `LogKeys.SERVICE_ACCOUNT_NAME` added, which is more than this bug is worth. 4.3.0 is not released yet, so 4.3 is the first release carrying the change either way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
