SEZ9 commented on issue #12457: URL: https://github.com/apache/seatunnel/issues/12457#issuecomment-5975680829
@nzw921rx thanks for recording the shared/pre-staged distribution question raised by @LeonYoah as a non-goal; that boundary matches what was discussed above. Two small follow-ups on that section so it is self-explanatory for later readers: 1. Briefly state the motivation (slow upload of a large archive) and list what a follow-up STIP would need to address: immutable content/version identity, integrity verification, HDFS permission and Kerberos handling, cache lifecycle/eviction, and connector/classloader upgrade semantics without mixing jars from two distributions. That way the deferral does not read as a rejection of the idea. 2. In the `yarn.distribution` option description, state explicitly that it is a local archive path, that one self-contained archive is localized read-only per submission for the ApplicationMaster and workers, and that remote/shared paths are out of scope for v1. This only covers the staging point; the broader items in my previous comment (YARN-only first implementation, section pointers for the frozen lifecycle contract, and the single ownership boundary between ApplicationResourceManager and ResourceManagerDriver) remain open. <!-- streview-comment:1495 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
