[
https://issues.apache.org/jira/browse/SAMZA-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597959#comment-16597959
]
ASF GitHub Bot commented on SAMZA-1817:
---------------------------------------
GitHub user Sanil15 opened a pull request:
https://github.com/apache/samza/pull/619
Samza 1817: Long classpath support for non-split deployments
https://jira.apache.org/jira/browse/SAMZA-1817
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Sanil15/samza SAMZA-1817
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/619.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #619
----
commit 576b9bc1074036a7299a4a773f8c639015be1cdd
Author: Sanil Jain <snjain@...>
Date: 2018-08-27T21:28:34Z
Using wildcarding in classpath to add support for long classpath
commit 6b7f6a2cc4bd1ade258e820fd812883c7f259444
Author: Sanil Jain <snjain@...>
Date: 2018-08-27T21:28:34Z
Using wildcarding in classpath to add support for long classpath in general
deployements only
Note that this does not include support for long classpath for split
deployements
commit b0b058544d9df8b141ea2ae25776c7f378c20496
Author: Sanil Jain <snjain@...>
Date: 2018-08-30T21:45:04Z
Merge branch 'SAMZA-1817' of https://github.com/Sanil15/samza into
SAMZA-1817
commit 23c3fa6a8d35c93e559619520ed9fe603f87e1d8
Author: Sanil Jain <snjain@...>
Date: 2018-08-30T21:46:50Z
Reverting changes for Split deployement
----
> Long classpath support
> ----------------------
>
> Key: SAMZA-1817
> URL: https://issues.apache.org/jira/browse/SAMZA-1817
> Project: Samza
> Issue Type: Bug
> Reporter: Sanil Jain
> Assignee: Sanil Jain
> Priority: Major
>
> Samza team could also support a problem for long classpath by implementing
> something that would reduce the classpath length.
> Some potential ideas to fix:
> - Change the generated classpath (done in Samza's bash scripts) to be a list
> of `<directory>/*jar` entries instead of a list of every individual jar.
> Wildcards in the classpath are supported in jdk >= 6.
> This should be a relatively simple change and can be implemented by getting
> the parent directory of all jar files into a list, sorting and making the
> list unique, then append a wildcard to the end to pick up all jars under each
> directory in the list (ask Jon Bringhurst if you have any questions).
> See
> https://github.com/apache/samza/blob/92ae4c628abb3d113520ec47ca82f08c480123ad/samza-shell/src/main/bash/run-class.sh#L63
> for where the code change for this should probably be.
> - Use the manifest in a special "uber-jar" to list all dependencies in the
> classpath. Then just include that one "uber-jar" in the classpath on the cli.
> This would probably be tough to do, and seems like it would be annoying to
> manage, but apparently IntelliJ uses this strategy.
> Note that this is not an issue with the number of arguments (argc limit,
> which is based on a multiple of the ulimit -s stack size), it is a problem
> with the length of a single argument (strlen(*argv) limit based on
> MAX_ARG_STRLEN, which is a multiple of the PAGE_SIZE). So, the only way to
> get around this limit is to recompile the kernel with a different page size
> (which obviously isn't practical).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
