[
https://issues.apache.org/jira/browse/HADOOP-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122254#comment-15122254
]
Sangjin Lee commented on HADOOP-12747:
--------------------------------------
Thanks for that [~cnauroth]! There is also
{{ApplicationClassLoader#constructUrlsFromClasspath}} that does a similar
thing. Perhaps we can refactor this logic into a common place.
One interesting thing to consider is whether we want to limit this wildcard
expansion to local paths or remote filesystems as well. I think both
{{FileUtil}} and {{ApplicationClassLoader}} consider only local paths. Although
supporting expanding for remote paths might make this enhancement more
consistent, it might increase the scope somewhat. Thoughts?
> support wildcard in libjars argument
> ------------------------------------
>
> Key: HADOOP-12747
> URL: https://issues.apache.org/jira/browse/HADOOP-12747
> Project: Hadoop Common
> Issue Type: New Feature
> Components: util
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
>
> There is a problem when a user job adds too many dependency jars in their
> command line. The HADOOP_CLASSPATH part can be addressed, including using
> wildcards (\*). But the same cannot be done with the -libjars argument. Today
> it takes only fully specified file paths.
> We may want to consider supporting wildcards as a way to help users in this
> situation. The idea is to handle it the same way the JVM does it: \* expands
> to the list of jars in that directory. It does not traverse into any child
> directory.
> Also, it probably would be a good idea to do it only for libjars (i.e. don't
> do it for -files and -archives).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
