SEZ9 commented on PR #11307:
URL: https://github.com/apache/seatunnel/pull/11307#issuecomment-5738838520
Thanks for re-checking the shell launcher on head d3d34f292. Your
description of ee21221df89c ("Preserve Spark 3.5 launcher arguments and exit
codes") — the unquoted `args=$@` removed, arguments passed through quoted, the
Spark args read back NUL-delimited into an array, and `${CLASS_PATH}` quoted at
its point of use — matches what F6 asked for. Before I close it, could you link
the relevant hunk or paste the `java` invocation and the array read-back as
they appear on head d3d34f292? I'd like to confirm against the diff rather than
the description.
You mentioned the same commit also covers F1, F2 and F5, but those changes
aren't described in this thread yet:
- F2: is the `eval` of the command string gone from the `.sh`? If the
array-based invocation replaced it, a pointer to that is enough.
- F1: the original problem was that `setlocal disabledelayedexpansion` kept
`!errorlevel!`/`!CMD!` from expanding, so `call` targeted the literal text
`!CMD!`. Could you show how the `.cmd` now expands and invokes the command? The
single-string argument assembly into `spark-submit.cmd` you mention is a
separate concern I'd also like addressed — either preserve individual argument
boundaries or document the limitation.
- F5: how does the Windows launcher now propagate a non-zero exit from the
starter java process when it prints nothing on stdout?
Still open, as you noted:
- F4: please confirm whether both `log4j-slf4j-impl` and `log4j-slf4j2-impl`
remain in the starter logging include list; if so, please keep one per starter
to avoid a dual-binding conflict.
- F7: please add at least one micro-batch streaming test against Spark
3.5.8, or adjust the quick-start docs until that path is verified.
Not yet covered:
- F3: is there a plan to compile the translation layer against 3.5.x, or add
a test exercising the Catalyst surface it touches, so a binary incompatibility
fails in CI rather than as a runtime NoSuchMethodError?
- F8: what should Spark 3.4.x users use, and can the engine docs say so?
Happy to take another pass once these are in.
<!-- streview-comment:1166 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
