Sanil Jain created SAMZA-1817:
---------------------------------
Summary: Long classpath support
Key: SAMZA-1817
URL: https://issues.apache.org/jira/browse/SAMZA-1817
Project: Samza
Issue Type: Bug
Reporter: Sanil Jain
Assignee: Sanil Jain
Samza team could also support a problem for long classpath by implementing
something that would reduce the classpath length.
Some potential ideas to fix:
- Change the generated classpath (done in Samza's bash scripts) to be a list of
`<directory>/*jar` entries instead of a list of every individual jar. Wildcards
in the classpath are supported in jdk >= 6.
This should be a relatively simple change and can be implemented by getting the
parent directory of all jar files into a list, sorting and making the list
unique, then append a wildcard to the end to pick up all jars under each
directory in the list (ask Jon Bringhurst if you have any questions).
See
https://github.com/apache/samza/blob/92ae4c628abb3d113520ec47ca82f08c480123ad/samza-shell/src/main/bash/run-class.sh#L63
for where the code change for this should probably be.
- Use the manifest in a special "uber-jar" to list all dependencies in the
classpath. Then just include that one "uber-jar" in the classpath on the cli.
This would probably be tough to do, and seems like it would be annoying to
manage, but apparently IntelliJ uses this strategy.
Note that this is not an issue with the number of arguments (argc limit, which
is based on a multiple of the ulimit -s stack size), it is a problem with the
length of a single argument (strlen(*argv) limit based on MAX_ARG_STRLEN, which
is a multiple of the PAGE_SIZE). So, the only way to get around this limit is
to recompile the kernel with a different page size (which obviously isn't
practical).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
