I did some tests with "type". What I've found out is that I need to specify both "conf" and "type" to the same value in my publications, "cortex-m4_gcc_withoutLoc" for example.
conf is needed for resolving the dependency (type can't be used with the <dependency> element). type is needed for allowing different paths during unpacking using cache patterns (unpacking doesn't use conf or extra attributes in cache patterns). Once I do this, I'm able to retrieve the proper artifact. I can even use the same path and filename for the same dependency name within my project as long as my ant task cleans-up the "/lib" folder before calling <ivy:retrieve />. If I don't do this, of course I get into the same problem as I had in the cache, having the same folder so Ivy sees no needs to copy-over from cache and keeps the dirty version. I did test the "sync" attributes on the <ivy:retrieve /> task, but it seems that it too, isn't compatible with unpacking. When set to true, my /lib folder stays empty. I find it a bit redundant having to specify the same value twice within two attributes (type and conf). I guess I could specify a ivy property if I really wanted to avoid it. But even with this little annoyance, I prefer it to unpacking manually as it looks tidier and reduce the amount of scripting and duplication of configuration variables/settings. Best regards, Jérémie -----Original Message----- From: Jeremie Faucher-Goulet [mailto:jeremie.faucher-gou...@trilliantinc.com] Sent: July 5, 2016 12:52 PM To: ivy-user@ant.apache.org Subject: RE: Packaging attribute, unpack location Hello Mark, Thank you for taking the time in confirming what I observed. I had thought about using the "name" instead, but it doesn't play well in my mind. Using a Ivy property to define an artifact name would feel a bit weird to me. I need to use a property because the conf is only resolved at invocation time, either by passing arguments to ant or by environment variable. If I can keep the triplet "org, name and rev" static, that feels much better. I have no doubt it would work, but I don't really like obfuscating my ivy.xml file that much. "Type" sounds like an interesting idea though. I'll have a look on whether I prefer that to using "conf" and doing the unpacking myself. I kinda like the flexibility/control gained by doing the unpacking myself. All I need, in its simplest form, is the following snippet in build.xml: <target name="unzip" description="Unzip artifacts"> <mkdir dir="${basedir}/lib/slf4ec" /> <unzip dest="${basedir}/lib/slf4ec"> <fileset dir="${basedir}/lib"> <include name="**/slf4ec*.zip" /> </fileset> </unzip> </target> Fundamentally, I believe Ivy should do the unpacking in the same location within it's cache as the file downloaded. In essence, follow the cache pattern too. I might open a ticket on this subject. Best regards, Jérémie -----Original Message----- From: Marc De Boeck [mailto:mdeb...@gmail.com] Sent: July 5, 2016 3:13 AM To: ivy-user@ant.apache.org Subject: Re: Packaging attribute, unpack location Hello Jérémie, Just to let you know: I did some tests with the packaging attribute, and experienced the same behaviour as you. I found however the following workaround: You can use the type attribute to download the artifacts to different directories in your cache-directory. In your environment this would mean: <publications> <artifact name="slf4ec" type="cortex-m4_gcc_withLoc" e:arch="x86" e:compiler="gcc" e:locInfo="withLoc" ext="zip" type="native" conf="x86_gcc_withLoc" packaging="zip" /> <artifact name="slf4ec" type="cortex-m4_gcc_withoutLoc" e:arch="x86" e:compiler="gcc" e:locInfo="withoutLoc" ext="zip" type="native" conf="x86_gcc_withoutLoc" packaging="zip" /> .... </publications> And then change the artifactPattern of your cache-settings as follows: <caches ivyPattern="[organisation]/[module]/[revision]/[artifact](-[arch])(-[compiler])(-[locInfo])-[revision].xml" artifactPattern="[organisation]/[module]/[type]/[revision]/[artifact](-[arch])(-[compiler])(-[locInfo])-[revision](-[classifier]).[ext]" /> You can also consider to use the "name" attribute of your artifacts to distinguish your artifacts. In its simpliest case, your ivy-publications section would become: <publications> <artifact name="slf4ec-cortex-m4_gcc_withLoc" packaging="zip" conf="cortex-m4_gcc_withLoc" /> <artifact name="slf4ec-cortex-m4_gcc_withoutLoc" packaging="zip" conf="cortex-m4_gcc_withoutLoc" /> .... </publications> And your cache settings can become something like this: <caches ivyPattern="[organisation]/[module]/[revision]/ivy-[revision].xml" artifactPattern="[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" /> Probably you also have to adapt the layout of your repositories. The latter solution is much simplier and is for me the best way to use Ivy: inside an ivy-module, you have one or more artifacts each with their own unique name (possibly for different configurations). The extra-attributes provide some extra flexibility, but make everything more complex. They can be useful if you want to parse the published ivy.xml by other tools, or if you have to mimic maven behaviour (e.g. by using the e:classifier attribute). Regards, Marc 2016-07-04 21:37 GMT+02:00 Jeremie Faucher-Goulet < jeremie.faucher-gou...@trilliantinc.com>: > Thanks for your feedback Hugh, > > Interesting concept of having a unpack cache to reuse across projects, > I'll keep that in mind as our needs evolve. I'll make this an > iterative process so this could very much prove useful in the mid to long > term... > For now I'll simply use the ant <unzip> task in my ant file to unpack > from the zip located in the ivy cache to a local folder of the project > using the dependency. But this will probably grow inefficient soon > enough ;) > > Thank you, > > Jérémie > > -----Original Message----- > From: Greene, Hugh [mailto:hgre...@tmvse.com] > Sent: July 1, 2016 4:58 AM > To: ivy-user@ant.apache.org > Subject: RE: Packaging attribute, unpack location > > Hi Jérémie, > > I have some experience which might be useful to you in your experiments. > > We have a set of Gradle plugins for doing something similar for Ivy > modules with ZIP artifacts. We do the unzipping separately because we > had to (pre-Ivy-2.4) and because we chose to unzip modules to a > separate folder whose location we control. That means that you can > have one such "unpack cache" per volume and symlinks/hardlink the > files into your project workspace, thereby saving disk space if you > have multiple workspaces with an overlapping set of dependencies. > > One subtle mistake we made, and have yet to undo, is that we unzipped > all ZIPs from a module into one folder, even though different > configurations of that module should really only expose the contents > of some subset of those ZIPs. That means you can just symlink the > whole folder, rather than having to somehow "merge" the separate > folders for each configuration. But it means that a folder symlinks > in your workspace may contain more than it should according to your > project's specified dependencies, if you previously unzipped files on > the same machine for a different project with a larger set of configurations > of that module. > > Hope that's useful (and makes sense!), > > Hugh Greene, Senior Software Developer Toshiba Medical Visualization > Systems Europe, Ltd Bonnington Bond, 2 Anderson Place, Edinburgh EH6 > 5NP, UK Tel + 44 (0)131 472 4792 / Fax + 44 > (0) 131 472 4799 http://www.tmvse.com / mailto:hgre...@tmvse.com > > DISCLAIMER > Unless indicated otherwise, the information contained in this message > is privileged and confidential, and is intended only for the use of > the > addressee(s) named above and others who have been specifically > authorized to receive it. If you are not the intended recipient, you > are hereby notified that any dissemination, distribution or copying of > this message and/or attachments is strictly prohibited. The company > accepts no liability for any damage caused by any virus transmitted by > this email. Furthermore, the company does not warrant a proper and > complete transmission of this information, nor does it accept > liability for any delays. If you have received this message in error, > please contact the sender and delete the message. > > > -----Original Message----- > From: Jeremie Faucher-Goulet [mailto: > jeremie.faucher-gou...@trilliantinc.com] > Sent: 30 June 2016 18:59 > To: ivy-user@ant.apache.org > Subject: RE: Packaging attribute, unpack location > > Thank you Marc for your sanity check, > > As I'm brand new to Ivy and experimenting with it, there's always this > nagging doubt I made a stupid mistake somewhere ;-) We're working on > putting together a new CI and CD process for embedded software > projects, and Ivy could prove to be a major part of it. > > I only just noticed that the "packaging" attribute is something new, > only introduced in Ivy 2.4, so I'm probably experimenting with > features that don't have a widespread usage currently. > > Thanks for your help, > > Jérémie > > -----Original Message----- > From: Marc De Boeck [mailto:mdeb...@gmail.com] > Sent: June 30, 2016 9:29 AM > To: ivy-user@ant.apache.org > Subject: Re: Packaging attribute, unpack location > > Jérémie, > > Your settings look correct to me. We also work with the > extra-attributes to distinguish artifacts, but we don't the package > attribute. If I need to unpack something, I do it outside of Ivy in an > ant-task (like you suggest as alternative). > I also agree that the extra-attributes in the <conf> section are not > needed. We don't have them either, but we do use them inside the > publications sections. > If I have some time tomorrow, I will try to make a test with the > package-attribute in our environment and see if I can reproduce your > problem. > > Regards, > Marc > > 2016-06-29 22:50 GMT+02:00 Jeremie Faucher-Goulet < > jeremie.faucher-gou...@trilliantinc.com>: > > > Other useful tidbits.... > > > > Here is my retrieve task in build.xml: > > > > <target name="retrieve" description="Resolve dependencies"> > > <ivy:retrieve > > > pattern="lib/[artifact]/[artifact](_[arch])(_[compiler])(_[locInfo])-[revision](-[classifier]).[ext]" > > /> > > </target> > > > > So in my cache I get the following directory structure: > > > > +.ivy2\cache\com.trilliantinc\slf4ec\1.0.0+15.db80236\ > > |----+slf4ec-1.0.0+15.db80236 (unpack folder) > > |----+slf4ec-x86-gcc-withLoc-1.0.0+15.db80236.zip (artifact > > |----+downloaded > > according to cache pattern) > > > > > > And in my project I get: > > > > +lib\ > > |----+slf4ec > > |----+slf4ec_x86_gcc_withLoc-1.0.0+15.db80236 (Unpacked > > artifact copied from cache, content of the "unpack folder" but > > following the retrieve pattern) > > > > Regards, > > > > Jérémie > > > > -----Original Message----- > > From: Jeremie Faucher-Goulet [mailto: > > jeremie.faucher-gou...@trilliantinc.com] > > Sent: June 29, 2016 4:35 PM > > To: ivy-user@ant.apache.org > > Subject: RE: Packaging attribute, unpack location > > > > Hi Marc, > > > > When not specifying the packaging attribute, everything seems to > > work as expected. I suppose I could always write a custom Ant action > > to unzip the files. I was trying to use the Ivy feature. > > > > Here is my current cache configuration in ivysettings.xml : > > <caches > > > ivyPattern="[organisation]/[module]/[revision]/[artifact](-[arch])(-[compiler])(-[locInfo])-[revision].xml" > > > artifactPattern="[organisation]/[module]/[revision]/[artifact](-[arch])(-[compiler])(-[locInfo])-[revision](-[classifier]).[ext]" > > /> > > > > Of course, [arch], [compiler] and [locInfo] are extra-attributes > > that are defined in the library I'm resolving from Artifactory. Here > > is the ivy.xml file of that library: > > > > <ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra";> > > <info organisation="com.trilliantinc" module="slf4ec" /> > > <configurations> > > <conf name="cortex-m4_gcc_withLoc" e:arch="cortex-m4" > > e:compiler="gcc" > > e:locInfo="withLoc" > > description="Binary for ARM Cortex-M4 > > compiled with GCC with location information" /> > > <conf name="cortex-m4_gcc_withoutLoc" e:arch="cortex-m4" > > e:compiler="gcc" e:locInfo="withoutLoc" > > description="Binary for ARM Cortex-M4 > > compiled with GCC without location information" /> > > <conf name="cortex-m4_iar_withLoc" e:arch="cortex-m4" > > e:compiler="iar" > > e:locInfo="withLoc" > > description="Binary for ARM Cortex-M4 > > compiled with IAR with location information" /> > > <conf name="cortex-m4_iar_withoutLoc" e:arch="cortex-m4" > > e:compiler="iar" e:locInfo="withoutLoc" > > description="Binary for ARM Cortex-M4 > > compiled with IAR without location information" /> > > <conf name="x86_gcc_withLoc" e:arch="x86" > e:compiler="gcc" > > e:locInfo="withLoc" > > description="Binary for x86 compiled with > > GCC with location information" /> > > <conf name="x86_gcc_withoutLoc" e:arch="x86" > > e:compiler="gcc" > > e:locInfo="withoutLoc" > > description="Binary for x86 compiled with > > GCC without location information" /> > > <conf name="artifacts" visibility="private" > > description="Build's artifacts" /> > > <conf name="all" extends="*" description="All artifacts" > /> > > </configurations> > > <publications> > > <artifact name="slf4ec" e:arch="x86" e:compiler="gcc" > > e:locInfo="withLoc" ext="zip" type="native" > > conf="x86_gcc_withLoc" > > packaging="zip" /> > > <artifact name="slf4ec" e:arch="x86" e:compiler="gcc" > > e:locInfo="withoutLoc" ext="zip" type="native" > > conf="x86_gcc_withoutLoc" > > packaging="zip" /> > > <artifact name="slf4ec" e:arch="cortex-m4" > e:compiler="gcc" > > e:locInfo="withLoc" ext="zip" type="native" > > conf="cortex-m4_gcc_withLoc" > > packaging="zip" /> > > <artifact name="slf4ec" e:arch="cortex-m4" > e:compiler="gcc" > > e:locInfo="withoutLoc" ext="zip" type="native" > > conf="cortex-m4_gcc_withoutLoc" > > packaging="zip" /> > > <artifact name="slf4ec" e:arch="cortex-m4" > e:compiler="iar" > > e:locInfo="withLoc" ext="zip" type="native" > > conf="cortex-m4_iar_withLoc" > > packaging="zip" /> > > <artifact name="slf4ec" e:arch="cortex-m4" > e:compiler="iar" > > e:locInfo="withoutLoc" ext="zip" type="native" > > conf="cortex-m4_iar_withoutLoc" > > packaging="zip" /> > > <artifact name="doc" ext="zip" type="html" > conf="artifacts" > > packaging="zip" /> > > <artifact name="tests" ext="zip" type="xUnit" > > conf="artifacts" > > packaging="zip" /> > > </publications> > > </ivy-module> > > > > I'm not exactly sure what use the extra-attributes could have inside > > the <conf /> section. Currently they don’t seem to impact anything > > and I'm probably going to remove them. > > Ideally I would have proffered to use these extra-attributes instead > > of configurations (and a long configuration name string) when > > resolving dependencies, but the resolver looks for these > > extra-attributes in the module, not the artifacts so cannot resolve > > when using extra-attributes. At least that's what I found out with > > my > experimentations. > > > > Best regards, > > > > Jérémie > > > > -----Original Message----- > > From: Marc De Boeck [mailto:mdeb...@gmail.com] > > Sent: June 29, 2016 4:16 PM > > To: ivy-user@ant.apache.org > > Subject: Re: Packaging attribute, unpack location > > > > Can you tell us the cache pattern that you have configured ? > > Maybe you can also try to retrieve the same artifacts but with a > > module descriptor (ivy-file) where the packaging attribute is not set. > > This way, you could find out if it is related to the packaging > > attribute, or related to something else. > > > > Regards, > > Marc > > > > 2016-06-29 21:22 GMT+02:00 Jeremie Faucher-Goulet < > > jeremie.faucher-gou...@trilliantinc.com>: > > > > > Hello, > > > > > > > > > > > > I’m encountering an issue where, when using a different > > > configuration for the same artifact during a retrieve operation, > > > the artifact is downloaded but the wrong artifact is copied in my project. > > > > > > Is it a limitation of the automatic unpacking (zip file in my > > > case) not keeping the extra attributes? I’m new to Ivy so there > > > might be a configuration option I haven’t found yet. > > > > > > > > > > > > For example, I’m using a custom pattern for the Ivy cache so I can > > > get the different configuration downloaded and a similar custom > > > pattern for the retrieve itself so I can get these different > > > artifacts in my > > project. > > > However, calling retrieve with a different configuration does > > > create a new folder in my project (retrieve pattern), but it’s > > > content is the same as with the previous configuration. > > > > > > Looking in the Ivy cache, I see that the download did create a new > > > (proper) artifact in my cache according to my custom cache > > > pattern, but the same folder was used (and not overridden) for unpacking. > > > > > > > > > > > > It seems whatever pattern I set, the unpacking location will > > > happen here in the cache: > > > [organisation]/[module]/[revision]/[artifact]-[revision]/* > > > > > > > > > > > > My assumption currently, is that Ivy will find the same unpacking > > > location so will skip the unpacking step. Retrieve will then copy > > > over a dirty/invalid version of what was unpacked. Are my > > > assumptions > correct? > > > > > > > > > > > > Is there a way to configure the unpacking location, if my > > > understanding of the issue is correct? > > > > > > > > > > > > P.S. I’m using configuration to differentiate different native > > > C/C++ builds (x86, arm, etc…) Perhaps I’m not using the proper approach? > > > > > > > > > > > > Thank you, > > > > > > > > > > > > [image: Description: cid:image001.jpg@01C9B6D4.5B951A30] > > > > > > Jeremie Faucher-Goulet, Jr. Eng. > > > Firmware Developer > > > Trilliant Inc > > > Tel: 450.375.0556 ext. 368 > > > jeremie.faucher-gou...@trilliantinc.com > > > > > > www.trilliantinc.com > > > > > > > > > > > > > > > > > > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > For more information please visit http://www.symanteccloud.com > ______________________________________________________________________ >
