[
https://issues.apache.org/jira/browse/GROOVY-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898998#comment-15898998
]
Ion Alberdi edited comment on GROOVY-8097 at 3/7/17 12:17 PM:
--------------------------------------------------------------
To give more context on our use case: we use grab to download up-to-date
versions of libraries that implement part of the computation of thousands of
concurrent jenkins jobs. The rate of updates reached for now a peak of 19 new
versions a single day.
[~paulk] Switching to something else than Ivy may indeed be more adapted to use
cases similar to ours. For now, during the updates done in a month, at least
two concurrent jobs write to the same resolutionCacheDir simultaneously. In
this scenario, the systemGroovyScripts relying on these libraries are broken as
the grab executed on the master's JVM uses a corrupted resolutionCacheDir. In
this scenario, we have no choice but rebooting the master.
[~jexler] The concurrency issue is not between resolutionCacheDir and
repositoryCache dir. The repositoryCache dir is thread-safe (as Ivy permits to
put a lock on repositoryCache dirs).
Let's call cJob1, cJob2 two concurrent jobs. The issue is between the
resolutionCacheDir of cJob1 and cJob2 that are executed in parallel. We'd like
to provide an API to make
- cJob1 use repositoryCacheDir, resolutionCacheDir1
- cJob2 use repositoryCacheDir, resolutionCacheDir2
Providing we create and upload thousands of different grapeConfig0.xml, ...,
grapeConfig999.xml and using the grape.config system property you suggested, we
will be able to make for ex, cJob2 use resolutionCacheDir2. I however see two
drawbacks:
- quite a lot of duplicated grabeConfig.xml, they will indeed all be the same
(only the resolutionCacheDir will be different),
- put a dependency between the code that deploys grapeConfig.xml-s and the code
that deploys new jobs: if 1000 new jobs should be added, 1000 new files should
be generated/deployed.
Regarding processes/Java VMs, issues within a VM, we have both use cases
- grab done in groovyCommand, are in the "between process/Java VMs" use case,
- grab done in systemGroovyCommands, are in the "same VM" use case.
More precisely https://issues.apache.org/jira/browse/GROOVY-7407 seems impacted
by the same limitation of Ivy. When reading,
"If Groovy scripts that import the same libraries via Grape are compiled in
separate threads, compilation may fail due to race conditions."
my guess is that if threads use the same locked repositoryCacheDir and
different resolutionCacheDir-s, the bug will not occur.
was (Author: yetanotherion):
To give more context on our use case: we use grab to download up-to-date
versions of libraries that implement part of the computation of thousands of
concurrent jenkins jobs. The rate of updates reached for now a peak of 19 new
versions a single day.
[~paulk] Switching to something else than Ivy may indeed be more adapted to use
cases similar to ours. For now, during the updates done in a month, at least
two concurrent jobs write to the same resolutionCacheDir simultaneously. In
this scenario, the systemGroovyScripts relying on these libraries are broken as
the grab executed on the master's JVM uses a corrupted resolutionCacheDir. In
this scenario, we have no choice but rebooting the master.
[~jexler] The concurrency issue is not between resolutionCacheDir and
repositoryCache dir. The repositoryCache dir is thread-safe (as Ivy permits to
put a lock on repositoryCache dirs).
Let's call cJob1, cJob2 two concurrent jobs. The issue is between the
resolutionCacheDir of cJob1 and cJob2 that are executed in parallel. We'd like
to provide an API to make
- cJob1 use repositoryCacheDir, resolutionCacheDir1
- cJob2 use repositoryCacheDir, resolutionCacheDir2
Providing we create and upload thousands of different grapeConfig1.xml, ...,
grapeConfig999.xml and using the grape.config system property you suggested, we
will be able to make for ex, cJob2 use resolutionCacheDir2. I however see two
drawbacks:
- quite a lot of duplicated grabeConfig.xml, they will indeed all be the same
(only the resolutionCacheDir will be different),
- put a dependency between the code that deploys grapeConfig.xml-s and the code
that deploys new jobs: if 1000 new jobs should be added, 1000 new files should
be generated/deployed.
Regarding processes/Java VMs, issues within a VM, we have both use cases
- grab done in groovyCommand, are in the "between process/Java VMs" use case,
- grab done in systemGroovyCommands, are in the "same VM" use case.
More precisely https://issues.apache.org/jira/browse/GROOVY-7407 seems impacted
by the same limitation of Ivy. When reading,
"If Groovy scripts that import the same libraries via Grape are compiled in
separate threads, compilation may fail due to race conditions."
my guess is that if threads use the same locked repositoryCacheDir and
different resolutionCacheDir-s, the bug will not occur.
> Add an argument to set the resolution cache path in @Grab
> ---------------------------------------------------------
>
> Key: GROOVY-8097
> URL: https://issues.apache.org/jira/browse/GROOVY-8097
> Project: Groovy
> Issue Type: New Feature
> Components: Grape
> Affects Versions: 2.4.8
> Reporter: Ion Alberdi
> Priority: Minor
>
> Ivy does not support concurrent access to its resolution cache
> https://issues.apache.org/jira/browse/IVY-654
> Grape relies on Ivy. For this reason, Grape cannot support concurrent access
> to its resolution cache neither.
> When using the @Grab annotation in jenkins groovyCommand or
> systemGroovyCommand, the related code is vulnerable to race conditions. When
> the race condition appears in a systemGroovyCommand, we have no choice but to
> reboot jenkins as all consecutive calls to @Grab fail.
> Among the two solutions we tried:
> - Protect the calls to grab with a lock similar to ivy's "artifact-lock-nio"
> strategy. Works but slow.
> - Set Ivy's lock on the repository cache and setup Grab to use a different
> cache resolution cache for each concurrent jobs. The following code permits
> to fix a test we did to reproduce the race condition.
> {code}
> static IvySettings createIvySettings(String resolutionPath, boolean
> dumpSettings) {
> // Copy/Paste/Purged from GrapeIvy.groovy
> IvySettings settings = new IvySettings()
> settings.load(new File(GROOVY_HOME, "grapeConfig.xml"))
> // set up the cache dirs
> settings.defaultCache = new File(GRAPES_HOME)
> settings.setVariable("ivy.default.configuration.m2compatible", "true")
> settings.setDefaultResolutionCacheBasedir(resolutionPath)
> return settings
> }
> static GrapeIvy ivyWithCustomResolutionPath(String resolutionPath) {
> Class<?> grapeIvyClass = Class.forName("groovy.grape.GrapeIvy");
> Object instance = grapeIvyClass.newInstance()
> Field field = grapeIvyClass.getDeclaredField("ivyInstance");
> field.setAccessible(true);
> field.set(instance,
> Ivy.newInstance(createIvySettings(resolutionPath)));
> return ((GrapeIvy)instance)
> }
> {code}
> We'd like to propose to add an additional argument to Grab to setup Ivy's
> resolution cache directory.
> Note that this solution seems to have been adopted by these users too
> https://rbcommons.com/s/twitter/r/3436/
> Would you agree on such a feature ? We'd be glad to propose a PR.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
