ETA for 2.4.0

[Geißler , Daniel]

Hello,

is there a planned date for the next ivy 2.4.x release/candidate?
Some very annoying issues (looking at JIRA) have been solved and I'd love to 
see them released.

kind regards
Daniel

_
www.salt-solutions.dehttp://www.salt-solutions.de

Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081

ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1

[Geißler , Daniel]

Hi,

today i was investigating the build times of different versions of our 
projects. What catched my eye is that the older version of a pretty large multi 
module project has significantly faster build times than the newer one. The 
older builds in 20 minutes while the newer takes about 1 hour, this is not 
explainable by new code introduced.

What I found is that the ivy:report task takes extraordinary amounts of time. 
It was never very fast but on some projects the report build time changed like 
this:

Ivy version

2.2.0

2.3.0-rc1

Diff

Module 1







ivy:report time

00:00:21

00:06:18

+1800%

html report file size

42.19 MB

867.92 MB

+2057%

graphml report

513.55 KB

573.38 KB

+11%

dot report

83.60 KB

94.11 KB

+12%

Module 2







ivy:report time

00:01:29

00:20:43

+1396%

html report file size

129.58 MB

2.18 GB

+1682%

graphml report

370.96 KB

457.42 KB

+23%

dot report

61.17 KB

75.73 KB

+23%


The call looks like this:

ivy:report conf=${fetch.configurations},build
xml=true
dot=true
todir=${project.dir.build.ivy-reports} /

We kept the reports building just in case something strange happens with the 
dependencies and with Jenkins it never used to be a problem to wait some extra 
minutes, but with these build times it s totaly over the top. Not to mention 
that I don't know any browser that can display a html page of 2 GB in a 
reasonable amount of time.

Are there some new features I'd miss in the release notes or should I open a 
JIRA ticket for this?

kind regards
Daniel

_
www.salt-solutions.dehttp://www.salt-solutions.de

Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081

AW: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1

[Geißler , Daniel]

Hi,

Ii found an answer myself. As the ivy-report.xsl didn't change (at least not 
when looking at the git history) I guess my dependency tree grew a way too much 
to get a useful html report out of it.

So the thing I changed was cutting of the dependency trees in the html report 
(see the recursion on the end of template called).
This was cutting down my builds to 10-20% of their previous duration.

It seems that there is some kind of exponential growth with the xsl describing 
the html report. I'd suggest to review it, as it is a very useful tool but 
costs too many resources.
One idea is to cut off all depdendency trees like I did, or maybe show the one 
overall tree but change the module detail dependencies to show only direct 
dependencies.

Anyway thanks for reading
Daniel
_

www.salt-solutions.de

Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081
-Ursprüngliche Nachricht-
Von: Geißler, Daniel 
Gesendet: Mi, 5. November 2014 12:27
An: ivy-user@ant.apache.org
Betreff: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1

Hi,

today i was investigating the build times of different versions of our 
projects. What catched my eye is that the older version of a pretty large multi 
module project has significantly faster build times than the newer one. The 
older builds in 20 minutes while the newer takes about 1 hour, this is not 
explainable by new code introduced.

What I found is that the ivy:report task takes extraordinary amounts of time. 
It was never very fast but on some projects the report build time changed like 
this:

Ivy version

2.2.0

2.3.0-rc1

Diff

Module 1







ivy:report time

00:00:21

00:06:18

+1800%

html report file size

42.19 MB

867.92 MB

+2057%

graphml report

513.55 KB

573.38 KB

+11%

dot report

83.60 KB

94.11 KB

+12%

Module 2







ivy:report time

00:01:29

00:20:43

+1396%

html report file size

129.58 MB

2.18 GB

+1682%

graphml report

370.96 KB

457.42 KB

+23%

dot report

61.17 KB

75.73 KB

+23%


The call looks like this:

ivy:report conf=${fetch.configurations},build
xml=true
dot=true
todir=${project.dir.build.ivy-reports} /

We kept the reports building just in case something strange happens with the 
dependencies and with Jenkins it never used to be a problem to wait some extra 
minutes, but with these build times it s totaly over the top. Not to mention 
that I don't know any browser that can display a html page of 2 GB in a 
reasonable amount of time.

Are there some new features I'd miss in the release notes or should I open a 
JIRA ticket for this?

kind regards
Daniel

_
www.salt-solutions.dehttp://www.salt-solutions.de

Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081

AW: IVY publish problem

[Geißler , Daniel]

You may just use the following:




This will require some tweaking of your scripts as the tasks name will change 
as you should put "xmlns:ivy="antlib:org.apache.ivy.ant"" in the projects 
Namespace declaration and so the tasks will appear as:

http://www.salt-solutions.de>

Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081

Von: Plamondon, Pierrick [mailto:pierrick.plamon...@frq.gouv.qc.ca]
Gesendet: Mo, 16. November 2015 16:44
An: ivy-user@ant.apache.org
Betreff: IVY publish problem

Hi,

I have a problem generating a jar file using ivy. It worked in the past but I 
don't know what is missing. Here is the problem :

Buildfile: C:\dev\projet\pilotageDev\ant\build.xml
clean:
   [delete] Deleting directory C:\dev\projet\pilotageDev\build
init-ivy:
 [echo] Download Ivy
[mkdir] Created dir: C:\dev\projet\pilotageDev\build\lib
  [get] Getting: 
http://192.168.55.11/ivy/repo/apache/ivy/2.0/jars/ivy-2.0.0-beta2.jar
  [get] To: C:\dev\projet\pilotageDev\build\lib\ivy.jar
 [echo] Configuration de taches
No public execute() in class org.apache.ivy.ant.IvyAntSettings

BUILD FAILED
C:\dev\projet\pilotageDev\ant\build-macro.xml:395: The following error occurred 
while executing this line:
C:\dev\projet\pilotageDev\ant\build-macro.xml:18: No public execute() in class 
org.apache.ivy.ant.IvyAntSettings

Total time: 967 milliseconds
--

Here is the line 395 :

  

  Download Ivy
  
  
  Configuration de taches
   LINE 395
  
  
  
  
  

  


Can you help me ?

Thanks,

Pierrick Plamondon
Technicien en informatique

Bureau du scientifique en chef du Québec
Fonds de recherche du Québec
 Fonds de recherche du Québec
140, Grande Allée Est, bureau 450
Québec (Québec) G1R 5M8
Téléphone : 418 643-8560, poste 3286
Télécopieur : 418 643-1451
pierrick.plamon...@frq.gouv.qc.ca
www.frq.gouv.qc.ca
[FRQ-3F(NT-S-SC)_pr_v2]   [banniere-signature] 


P Devez-vous vraiment imprimer ce courriel? Pensons à l'environnement!

Avis sur la confidentialité et avertissement relatif à la Loi sur l'accès aux 
documents des organismes publics et sur la protection des renseignements 
personnels (L.R.Q., c.A-2.1)
L'information transmise par ce courriel est de nature privilégiée et 
confidentielle. Elle est destinée à l'usage exclusif du destinataire ci-dessus. 
Si vous n'êtes pas le destinataire visé, vous êtes par la présente avisé qu'il 
est strictement interdit d'utiliser cette information, de la copier, de la 
distribuer ou la diffuser. Si cette communication vous a été transmise par 
erreur, veuillez la détruire et nous en aviser immédiatement par courriel.



Pierrick Plamondon
Technicien en informatique

Bureau du scientifique en chef du Québec
Fonds de recherche du Québec
 Fonds de recherche du Québec
140, Grande Allée Est, bureau 450
Québec (Québec) G1R 5M8
Téléphone : 418 643-8560, poste 3286
Télécopieur : 418 643-1451
pierrick.plamon...@frq.gouv.qc.ca
www.frq.gouv.qc.ca
[FRQ-3F(NT-S-SC)_pr_v2]   [banniere-signature] 


P Devez-vous vraiment imprimer ce courriel? Pensons à l'environnement!

Avis sur la confidentialité et avertissement relatif à la Loi sur l'accès aux 
documents des organismes publics et sur la protection des renseignements 
personnels (L.R.Q., c.A-2.1)
L'information transmise par ce courriel est de nature privilégiée et 
confidentielle. Elle est destinée à l'usage exclusif du destinataire ci-dessus. 
Si vous n'êtes pas le destinataire visé, vous êtes par la présente avisé qu'il 
est strictement interdit d'utiliser cette information, de la copier, de la 
distribuer ou la diffuser. Si cette communication vous a été transmise par 
erreur, veuillez la détruire et nous en aviser immédiatement par courriel.

AW: Issue Publishing with UrlResolver and later Resolving with IBiblioResolver

[Geißler , Daniel]

The ibiblio resolver is for dependency resolution in maven repositories. If you 
don't publish a pom.xml then resolving might be somewhat strange.

So you'll need to create a pom first: 
http://ant.apache.org/ivy/history/2.4.0/use/makepom.html where you can map your 
ivy configurations to maven scopes.
Add the pom to your publish section in the ivy.xml and try it again.

When resolving with the ibiblio resolver you are however bound to automatically 
created configurations that meet the maven scopes (master, compile, test ...). 
As far as I remember default is a combination of master (the artifact itself) 
and runtime (it's runtime dependencies).
 
When publishing with ivy.xml the url resolver should be sufficient and even 
faster than ibiblio.

Hope that helps.

Kind regards
Daniel

_

www.salt-solutions.de

Geschäftsführer: Dr. Bernhard Blüthner, Norbert H. Fiedler, Dieter Heyde, 
Markus Honold
Sitz: München, AG München, HRB 146081 

-Ursprüngliche Nachricht-
Von: Vincent Case [mailto:vincent.c...@gmail.com] 
Gesendet: Mi, 2. November 2016 18:36
An: ivy-user@ant.apache.org
Betreff: Issue Publishing with UrlResolver and later Resolving with 
IBiblioResolver

Problem Overview: My goal is to publish an artifact using the UrlResolver and 
then retrieve it using the IBiblioResolver.  However, resolve fails if
the dependency conf mapping is to something other than "->default".   What
I observe is:
1- The ivy:resollve task succeeds if the UrlResolver is used for resolving as 
well as publishing.
2- The ivy:resolve task succeeds with the IBiblioResolver only if the conf 
mapping is to the "->default" for the published artifact.
3- When the dependency conf mapping is to something other than ->default, 
verbose logging indicates that "no ivy file file found for XXX: using default 
data"
4- When the "m2compitiblity" settings are the same for publish and resolve, the 
failure is always the same, regardless of its setting, and regardless of the 
"usepoms" setting.

I have created a trivial example (below) that exhibits the behavior described 
above.

Thanks for your help.

=== settings.properties ===
pattern.artifacts.pub=[organization]/[module]/[revision]/[type]s/[artifact].[ext]
pattern.ivy.pub=[organization]/[module]/[revision]/[type]s/[artifact].[ext]

=== settings.xml ===

 


 







=== ivy.xml for Publish ===




  






=== ivy.xml for Resolving the published artifact ===  



  


  

=== Build File ===





 

 
 


 
 file:pub.jar 





 



 







=== Verbose Logging Output 

[ivy:resolve]   tried http://centos-7:8882/
artifactory/rps-integration/com/vcase/rps/pub/[revision]/jars/pub.jar
[ivy:resolve]   resolver-res: no ivy file found for
com.vcase.rps#pub;latest.integration: using default data
[ivy:resolve]   [1.0] com.vcase.rps#pub
:
:
[ivy:resolve]   :: com.vcase.rps#pub;1.0: configuration not found
in com.vcase.rps#pub;1.0: 'runtime'. It was required from 
com.vcase.rps#;working@vcase-PC-new compile


EOM

AW: Problem when building the same project concurrently for several OSes

[Geißler , Daniel]

Hi Misha,

as far as i remember the problem might not be the artifact cache, but the 
resolution cache.
The behavior I observed was, that ivy keeps some kind of 
dependency-resolution-cache of recently resolved artifacts. This cache seems to 
have no locking and multiple builds may rewrite files that are currently read 
by another build.

http://ant.apache.org/ivy/history/2.0.0/settings/caches.html
see the properties for repositoryCacheDir vs. resolutionCacheDir

I haven’t had time to evaluate the behavior, but any feedback is appreciated.

Regards,
Daniel

_

SALT Solutions GmbH

Geschäftsführer: Dr. Bernhard Blüthner, Norbert H. Fiedler, Dieter Heyde, 
Markus Honold
Sitz: München, AG München, HRB 146081 

Hi everybody,

We have some projects, like Zookeeper, that use Ant/Ivy. In some situations 
such a project needs to be built multiple times, for different OSes. To save 
time, we do it on the same machine and in parallel (we use docker containers to 
build for different OSes). The main problem in this scenario is when we want 
all these builds to share the same Ivy cache, i.e. use it for concurrent reads 
and writes.

Ivy 2.4 and artifact-lock-nio locking strategy are supposed to address this. 
However, when I attempted to switch to this combination and enable a single 
cache, my builds failed. I've verified that NIO-based file locking works across 
docker containers. I then found and enabled debug logging in Ivy's 
FileBasedLockStrategy (one has to invoke ant with '-Divy.log.locking=true'). It 
looks like locking for artifacts that are downloaded from the web works as 
expected. However, in the end there is a failure that says "failed to parse 
report: /home/cauldron/.ivy2/cache/ 
org.apache.zookeeper-zookeeper-default.xml". See the end of the log file pasted 
below. The file in question contains essentially the list of Zookeeper's 
external dependencies, and looks like Ivy itself has just generated it.

So I am confused - why is Ivy trying to read it immediately after writing?
And, generally, is there any chance for us to fix or work around this problem 
and use the single, shared Ivy cache?

Regards,

Misha


 [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area
(locking)
 [ivy:retrieve] Thread[main,5,main] 1477355454558 current status for 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
is 1 held locks: Thread[main,5,main]
 [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock acquired on 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
in 0ms - hold locks = 2
 [ivy:retrieve] Thread[main,5,main] 1477355454558 releasing lock on 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] Thread[main,5,main] 1477355454558 entered synchronized area
(unlocking)
 [ivy:retrieve] Thread[main,5,main] 1477355454558 reentrant lock released on 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/metadata-3.10.5.Final.ivy.lck
- hold locks = 1
 [ivy:retrieve] Thread[main,5,main] 1477355454559 releasing lock on 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] Thread[main,5,main] 1477355454559 entered synchronized area
(unlocking)
 [ivy:retrieve] Thread[main,5,main] 1477355454559 lock released on 
/home/cauldron/.ivy2/cache/io.netty/netty/metadatas/
metadata-3.10.5.Final.ivy.lck
 [ivy:retrieve] :: resolution report :: resolve 468ms :: artifacts dl 131ms
 -
 |  |modules||   artifacts   |
 |   conf   | number| search|dwnlded|evicted|| number|dwnlded|
 -
 |  default |   5   |   0   |   0   |   0   ||   5   |   0   |
 -
 [ivy:retrieve] :: retrieving :: org.apache.zookeeper#zookeeper
 [ivy:retrieve] confs: [default]

 BUILD FAILED
 /container.debian7/output/cdh/zookeeper/zookeeper-3.4.5+cdh6.x+0/build.xml:362:
impossible to ivy retrieve: java.lang.RuntimeException: problem during retrieve 
of org.apache.zookeeper#zookeeper: java.text.ParseException:
failed to parse report:
/home/cauldron/.ivy2/cache/org.apache.zookeeper-zookeeper-default.xml:
Premature end of file.
 at org.apache.ivy.core.retrieve.RetrieveEngine.retrieve(
RetrieveEngine.java:249)
 at org.apache.ivy.Ivy.retrieve(Ivy.java:561)
 at org.apache.ivy.ant.IvyRetrieve.doExecute(IvyRetrieve.java:98)
 at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:271)
 at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
 at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
 at 

ivy cache, changing patterns and time to live

[Geißler , Daniel]

Hi,

we are investigating some serious pain with our build times and identified 
ivy:resolve as the main time consumer.
While having similar dependency trees in most of the components in the multi 
module build, all artifacts seem to be searched in the remote repository all 
the time.

We know that dynamic and SNAPSHOT versions are one reason for that, but still 
there must be a way to keep the results at least for one build fresh.
This is where we found the defaultTTL or ttl aspects of caches 
(http://ant.apache.org/ivy/history/latest-milestone/settings/caches/ttl.html).

With some rough testing we figured out that enabling the checkmodified flag on 
a resolver forces every artifact to be resolved (be it a SNAPSHOT dependency or 
not, all artifacts are listed as searched by the resolve task).
But when we set checkmodified="false" and a short defaultTTL on the cache, 
repeated calls will not trigger a search until the time expires.

Here's the resolver configuration for the repository containing SNAPSHOT and 
released artifacts:




and this is how our cache is configured:



Anyone can give brief explanation when to use what and which sideeffects to 
expect? The documentation is not exactly clear about how to handle changing 
artifacts in a effective way.

any help would be appreciated
Daniel Geißler
_

SALT Solutions AG

www.salt-solutions.de

Vorstand: Dieter Heyde (Vors.), Dr. Bernhard Blüthner, Dr. Hans Christoph 
Dönges, Claudia Lang, Frank Reinecke
Vorsitzender des Aufsichtsrats: Dr. Michael Fuchs
Sitz: München, AG München, HRB 228545