Very interesting. After reading this, I configured one cache per repository and I do believe it fixed one of odd behaviors that I saw earlier.
The odd behavior was an artifact was selected as a winner from the local repository, but a subsequent last second check for that artifact in the cache resulted in a cached artifact from a remote repository being delivered. This no longer seems to be an issue but I am also testing a few other changes so can't be sure if this is fixed and by this change. But I think it is. Now I can't imagine why I would ever want a shared cache for multiple repositories. I think I like one cache per repo much better. Thanks for the input. It made a tangible positive difference in my testing. I am currently testing a potential bug fix in the ChainResolver. It might not be a bug, in which case it would make a great enhancement. Either way, I like it very much. The fix is to get the chain resolver to return the latest snapshot from any of all configured repositories. Some code looked odd in that the intent looked to be to return the most recent artifact given two artifacts, but no date comparison was ever performed. Rather, the method always returned the same Boolean regardless of the age of the artifacts. Performing an actual date comparison at that point unlocked a bunch of great behaviors. It was a 4 line fix in ChainResolver. I will share asap. L.K. -----Original Message----- From: Sven Zethelius [mailto:sv...@expedia.com] Sent: Monday, March 23, 2015 4:07 PM To: ivy-user@ant.apache.org Subject: Re: cache busting and integration question There are uses where a FileSystemRepository need to be cached for performance. On Windows, UNC path files will be resolved by the FileSystemRepository. On Linux, smb or NFS file mounts will benefit from local caching as well. Have you tried configuring caching differently for each repository in your ivy.settings.xml. Here¹s a snippet from mine. We make use of the local cache as a repository when the build is given special instructions to be completely offline. <caches default="remote-cache" defaultCacheDir="${ivycache.base}" ivyPattern="[organisation]/[module]/ivy-[revision].xml" artifactPattern="[organisation]/[module]/[artifact]-[revision].[ext]"> <cache name="local-cache" useOrigin="true" /> <cache name="remote-cache" useOrigin="false" defaultTTL="${ivycache.timeout}" /> <cache name="offline-cache" basedir="${ivycache.base}/../offline" useOrigin="true" /> </caches> Then in each resolver, I specify which of Œlocal-cache¹, Œremote-cache¹, Œoffline-cache¹ it should use using the cache attribute. On 3/23/15, 3:09 PM, "Loren Kratzke" <lkrat...@blueorigin.com> wrote: >Ivy works perfectly with a single repository or even a chain of >repositories provided that a dependency does not try to come from two >places. > >Ivy records where an artifact comes from but does not take into account >that it may come from a different place next time. When Ivy finds a >candidate artifact in a repository somewhere, it checks the .ivy cache >for that artifact, finds an artifact in the cache (that happens to >originate from somewhere else), doesn't check or realize that it is >from somewhere else, checks the module descriptor against the origin >and determines that the stale artifact is up to date with its stale >origin, and then assumes that the stale artifact is the same as the >found artifact. It does not compare the locations and invalidate the >cached artifact. Rather, it delivers the stale artifact that originated >from a different location than the found artifact. > >THAT is a bug. It has to be a bug. There is no way that this is not a >bug. Got that out of my system. Whew! > >That said, using a local repo and "force" tends to overcome this >behavior (in theory, and in my testing, and only sometimes). Results of >whether you get a fresh artifact range from "never", to "the second >time and after that", to "always but only ever from one place" >depending upon the configuration. > >I think this is fixable or at least patchable though. > >For example, when I find an artifact in a file system repository, WHY >would I want to check my cache for that artifact? The artifact is >already right-freaking-there on my file system. It is "here". It can't >be any closer without already being delivered to my project. Only >trouble can come from querying the cache when you have the desired >artifact in your hand already. And to then proceed to find and return >an older artifact from the cache that came from a different repo is just not >cool at all. > >**Should not check the cache for an artifact that was found in a file >system repository. >**Should not assume that cached artifact came from same repo as found >artifact. >**Should at least check the date to know that the cached artifact about >to be returned is older than the found artifact. > >Perhaps using a different cache directory for each repository would >help on that last point, but there is nothing that makes me think that >Ivy would not search all of its caches and produce a similar outcome of >providing a stale artifact fetched at an earlier time and from a >different repository. I may be a bit jaded at this point. But I have >many hours in on this so can vent a little. > >Part of the problem is that Ivy developers somewhere along the line >separated the module descriptor from the module that it describes, and >both of those from the repository which provided each, recording the >origin location of the artifact in the cached module descriptor instead >of using independent cache directories per repo and using metadata >files, and then not leveraging that origin information to know that it >has totally not found a cached version of a found artifact. That's 4 >strikes that all add up to a very broken and at times quite silly cache >mechanism. > >I am banking on some of this being a bug and some of it being by >design, but mostly bug. When I go back into Ivy source code, I am going >to see if I can disable the cache checking when an artifact is found >using a file system resolver. That would be a major step forward and >would make much of this odd behavior go away. > >L.K. > >-----Original Message----- >From: Marc De Boeck [mailto:mdeb...@gmail.com] >Sent: Monday, March 23, 2015 1:29 PM >To: ivy-user@ant.apache.org >Subject: Re: cache busting and integration question > >It's an interesting problem to investigate, and I understand your >frustration. But since we don't have exactly the same setup as you, >it's not evident to tell exactly what the problem is and how you can solve it. >We also have an environment with a local and a central repo, but we >don't use the checkmodified attribute. If it would work, it is indeed >a good idea to setup such a resolver chain with SNAPSHOT builds for the >developers. >In our case (without checkmodified=true) the detection of changes in >both local and central repo works correctly for us. > >Now, I can only search with you in the docs hoping to find a missing >setting (if it's not an actual bug as you suggested). > >I suppose that the publication date of the ivy.xml that you published >in your local repo is more recent than the one in your cache ? Did you >also try setting the "changing" attribute directly in the dependency >itself (allthough I expect the behaviour will be similar). >Could you try a setup where you only publish and resolve from the local >repo (so not chained with your Nexus) ? > >You also wrote that latest.integration is a non-starter for you ? So >could you describe how you specify your dependencies ? If you specify >an exact revision in your dependency (like rev="1.0.0-SNAPSHOT), then >maybe you could also try to set the attribute >"alwaysCheckExactRevision" in the settings of your resolver. > > >Regards, >Marc > > >2015-03-23 17:33 GMT+01:00 Loren Kratzke <lkrat...@blueorigin.com>: > >> I am not getting the results that you describe... >
