Hi everybody, We have some projects, like Zookeeper, that use Ant/Ivy. In some situations such a project needs to be built multiple times, for different OSes. To save time, we do it on the same machine and in parallel (we use docker containers to build for different OSes). The main problem in this scenario is when we want all these builds to share the same Ivy cache, i.e. use it for concurrent reads and writes.
Ivy 2.4 and artifact-lock-nio locking strategy are supposed to address this. However, when I attempted to switch to this combination and enable a single cache, my builds failed. I've verified that NIO-based file locking works across docker containers. I then found and enabled debug logging in Ivy's FileBasedLockStrategy (one has to invoke ant with '-Divy.log.locking=true'). It looks like locking for artifacts that are downloaded from the web works as expected. However, in the end there is a failure that says "failed to parse report: /home/cauldron/.ivy2/cache/ org.apache.zookeeper-zookeeper-default.xml". See the end of the log file pasted below. The file in question contains essentially the list of Zookeeper's external dependencies, and looks like Ivy itself has just generated it. So I am confused - why is Ivy trying to read it immediately after writing? And, generally, is there any chance for us to fix or work around this problem and use the single, shared Ivy cache? Regards, Misha [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area (locking) [ivy:retrieve] Thread[main,5,main] 1477355454558 current status for /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck is 1 held locks: Thread[main,5,main] [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock acquired on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck in 0ms - hold locks = 2 [ivy:retrieve] Thread[main,5,main] 1477355454558 releasing lock on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/ metadata-3.10.5.Final.ivy.lck [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area (unlocking) [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock released on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck - hold locks = 1 [ivy:retrieve] Thread[main,5,main] 1477355454559 releasing lock on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/ metadata-3.10.5.Final.ivy.lck [ivy:retrieve] Thread[main,5,main] 1477355454559 entered synchronized area (unlocking) [ivy:retrieve] Thread[main,5,main] 1477355454559 lock released on /home/cauldron/.ivy2/cache/io.netty/netty/metadatas/ metadata-3.10.5.Final.ivy.lck [ivy:retrieve] :: resolution report :: resolve 468ms :: artifacts dl 131ms --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 5 | 0 | 0 | 0 || 5 | 0 | --------------------------------------------------------------------- [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper [ivy:retrieve] confs: [default] BUILD FAILED /container.debian7/output/cdh/zookeeper/zookeeper-3.4.5+cdh6.x+0/build.xml:362: impossible to ivy retrieve: java.lang.RuntimeException: problem during retrieve of org.apache.zookeeper#zookeeper: java.text.ParseException: failed to parse report: /home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml: Premature end of file. at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve( RetrieveEngine.java:249) at org.apache.ivy.Ivy.retrieve(Ivy.java:561) at org.apache.ivy.ant.IvyRetrieve.doExecute(IvyRetrieve.java:98) at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:271) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.tools.ant.dispatch.DispatchUtils.execute( DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.Project.executeTarget(Project.java:1368) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets( DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.Main.runBuild(Main.java:809) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109) Caused by: java.text.ParseException: failed to parse report: /home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml: Premature end of file. at org.apache.ivy.plugins.report.XmlReportParser.parse( XmlReportParser.java:293) at org.apache.ivy.core.retrieve.RetrieveEngine. determineArtifactsToCopy(RetrieveEngine.java:329) at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve( RetrieveEngine.java:118) ... 19 more Caused by: org.xml.sax.SAXParseException; systemId: file:/home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml; lineNumber: 1; columnNumber: 1; Premature end of file. at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper. createSAXParseException(ErrorHandlerWrapper.java:203) at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper. fatalError(ErrorHandlerWrapper.java:177) at com.sun.org.apache.xerces.internal.impl. XMLErrorReporter.reportError(XMLErrorReporter.java:400) at com.sun.org.apache.xerces.internal.impl. XMLErrorReporter.reportError(XMLErrorReporter.java:327) at com.sun.org.apache.xerces.internal.impl.XMLScanner. reportFatalError(XMLScanner.java:1472) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$ PrologDriver.next(XMLDocumentScannerImpl.java:1014) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next( XMLDocumentScannerImpl.java:602) at com.sun.org.apache.xerces.internal.impl. XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl .java:505) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse( XML11Configuration.java:841) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse( XML11Configuration.java:770) at com.sun.org.apache.xerces.internal.parsers.XMLParser. parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse( AbstractSAXParser.java:1213) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$ JAXPSAXParser.parse(SAXParserImpl.java:643) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl. parse(SAXParserImpl.java:327) at javax.xml.parsers.SAXParser.parse(SAXParser.java:328) at org.apache.ivy.plugins.report.XmlReportParser$ SaxXmlReportParser.parse(XmlReportParser.java:249) at org.apache.ivy.plugins.report.XmlReportParser.parse( XmlReportParser.java:291) ... 21 more