[Bug 43144] tar task is very slow when a is used as the source resource
https://bz.apache.org/bugzilla/show_bug.cgi?id=43144
--- Comment #10 from Vincent Privat ---
Hi,
Thanks for replies! Yes I was building an uber-jar using jar > restrict >
archives > zips > fileset, resulting in a 85 Mb jar file with ~2000 files. The
build time took more than 30 minutes.
I changed it using unzip + jar and the build time is now about 1 minute, see
this commit for details:
https://trac.openstreetmap.org/changeset/34703/subversion/applications/editors/josm/plugins/javafx/build.xml
I debugged Ant quickly with VisualVM and found out the processing of all
different files resulted in hundreds of thousands of calls to
ZipResource.getInputStream(). The symptom looks the same as Stefan first
comment ("We open and close the zip archive for each entry").
I will try your PR (thanks for creating it so fast!)
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 43144] tar task is very slow when a is used as the source resource
https://bz.apache.org/bugzilla/show_bug.cgi?id=43144 --- Comment #9 from Stefan Bodewig --- Jaikiran, see Noel's example here which probably tries to build an ueberjar. Likely background: https://stackoverflow.com/q/35577351/4524982 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 43144] tar task is very slow when a is used as the source resource
https://bz.apache.org/bugzilla/show_bug.cgi?id=43144 --- Comment #8 from Jaikiran Pai --- > With other build systems (Maven, Gradle...) making an uber-jar that contains > all dependencies is very fast. Vincent, you mention uber-jar whereas this issue started off with the tar task and the proposed patch is currently only applied in the tar task. Are you using the jar task with a zipfileset and that one too is showing a slow performance? I haven't checked the code for that yet but I won't be surprised if that task too is impacted. Do you have a sample build file showing your usage? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 43144] tar task is very slow when a is used as the source resource
https://bz.apache.org/bugzilla/show_bug.cgi?id=43144 --- Comment #7 from Jaikiran Pai --- I have opened a pull request[1] with a potential way to solve this. [1] https://github.com/apache/ant/pull/76 -- You are receiving this mail because: You are the assignee for the bug.
[jira] [Commented] (IVY-1586) Retrieves test-library instead of binary-library
[
https://issues.apache.org/jira/browse/IVY-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669915#comment-16669915
]
Sebastian Nagel commented on IVY-1586:
--
A similar or even the same issue has been observed in the Nutch nightly builds,
see NUTCH-2672.
I can confirm that this issue is easily reproducible using the ivy.xml provided
by [~a.schulze] running ivy twice with the same cache:
{noformat}
> # first run
> java -jar ./ivy-2.5.0-rc1.jar -ivy ivy.xml -cache ./cache
:: loading settings :: url =
jar:file:/home/ubuntu/ivy-1586/ivy-2.5.0-rc1.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: resolving dependencies :: MyOrganisation#IvyResolveBug;working@valsa
confs: [default]
found ch.qos.logback#logback-classic;1.2.3 in public
found ch.qos.logback#logback-core;1.2.3 in public
found org.slf4j#slf4j-api;1.7.25 in public
downloading
https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar
...
. (283kB)
.. (0kB)
[SUCCESSFUL ] ch.qos.logback#logback-classic;1.2.3!logback-classic.jar
(259ms)
downloading
https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar
...
(460kB)
.. (0kB)
[SUCCESSFUL ] ch.qos.logback#logback-core;1.2.3!logback-core.jar (316ms)
downloading
https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar
...
.. (40kB)
.. (0kB)
[SUCCESSFUL ] org.slf4j#slf4j-api;1.7.25!slf4j-api.jar (139ms)
:: resolution report :: resolve 2504ms :: artifacts dl 734ms
-
| |modules|| artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
-
| default | 3 | 3 | 3 | 0 || 3 | 3 |
-
> tree ./cache/ch.qos.logback/logback-core/
./cache/ch.qos.logback/logback-core/
├── ivy-1.2.3.xml
├── ivy-1.2.3.xml.original
├── ivydata-1.2.3.properties
└── jars
└── logback-core-1.2.3.jar
1 directory, 4 files
> cat ./cache/ch.qos.logback/logback-core/ivydata-1.2.3.properties
#ivy cached data file for ch.qos.logback#logback-core;1.2.3
#Wed Oct 31 11:09:18 CET 2018
artifact\:ivy\#ivy\#xml\#768556845.is-local=false
artifact\:logback-core\#jar\#jar\#1486237905.is-local=false
resolver=public
artifact\:logback-core\#pom.original\#pom\#-1156166729.location=https\://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.pom
artifact\:ivy\#ivy\#xml\#768556845.original=artifact\:logback-core\#pom.original\#pom\#-1156166729
artifact\:logback-core\#jar\#jar\#1486237905.original=artifact\:logback-core\#jar\#jar\#1486237905
artifact\:logback-core\#pom.original\#pom\#-1156166729.exists=true
artifact\:logback-core\#pom.original\#pom\#-1156166729.is-local=false
artifact\:ivy\#ivy\#xml\#768556845.exists=true
artifact\:logback-core\#jar\#jar\#1486237905.exists=true
artifact\:logback-core\#pom.original\#pom\#-1156166729.original=artifact\:logback-core\#pom.original\#pom\#-1156166729
artifact.resolver=main
artifact\:ivy\#ivy\#xml\#768556845.location=https\://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.pom
artifact\:logback-core\#jar\#jar\#1486237905.location=https\://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar
> # second run
> java -jar ./ivy-2.5.0-rc1.jar -ivy ivy.xml -cache ./cache
:: loading settings :: url =
jar:file:/home/ubuntu/ivy-1586/ivy-2.5.0-rc1.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: resolving dependencies :: MyOrganisation#IvyResolveBug;working@valsa
confs: [default]
found ch.qos.logback#logback-classic;1.2.3 in public
found ch.qos.logback#logback-core;1.2.3 in public
found org.slf4j#slf4j-api;1.7.25 in public
downloading
https://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3-tests.jar
...
... (370kB)
.. (0kB)
[SUCCESSFUL ]
ch.qos.logback#logback-core;1.2.3!logback-core.jar(test-jar) (978ms)
downloading
https://repo1.maven.org/maven2/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25-tests.jar
...
.. (31kB)
.. (0kB)
[SUCCESSFUL ] org.slf4j#slf4j-api;1.7.25!slf4j-api.jar(test-jar) (188ms)
:: resolution report :: resolve 133ms :: artifacts dl 1189ms
-
| |modules|| artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
