Hi All,

I am opening this thread to discuss FLIP-563: Support for provided lib
archives in YARN Application Mode
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-563+Support+provided+lib+archives+for+YARN+application+mode>
.

Today *yarn.provided.lib.dirs* assumes pre-uploaded directories of jars. In
practice, many deployments package shared libs as *.tgz/.tar.gz* and/or hit
classpath “*argument too long*” issues when many jars are enumerated.

*Proposal:* introduce *yarn.provided.lib.archives* to point to pre-uploaded
HDFS archives. Flink would localize them as public archive resources and
add a single wildcard classpath entry per archive (e.g., <archiveDir>/*).
Backward compatible and no change unless configured.

Please feel free to reply to this email thread and share your opinions.

Thanks,
Archit Goyal

Reply via email to