[
https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591613#comment-13591613
]
Martin Todorov commented on IVY-1197:
-------------------------------------
Maarten,
I did try it with the httpclient included in the latest Ivy. I added it to the
classpath. Yes, indeed -- the OOME is no longer there, but uploading the file
takes a ridiculously long time. I tried with a file of 1.7 GB by deploying and
artifact to Nexus with (first) Ivy and (then) Maven. Ivy took ~ 2 hr 30 mins,
whereas under Maven 3.0.4 (without any extra configuration) it took ~ 5 min 25
sec. This is a huge difference, even with the httpclient on the classpath.
I can't invest more time into analyzing this myself, but I can tell you what is
most-likely happening. Check these two links for further details:
- [http://maven.apache.org/docs/3.0.4/release-notes.html Maven 3.0.4's release
notes]
- [http://maven.apache.org/guides/mini/guide-http-settings.html Guide to HTTP
settings]
Check the second link in regards to the http.protocol.expect-continue=true
part. This explains what is happening. Apart from that, the PosterOutputStream
is actually extending the ByteArrayOutputStream. If you have a look at it, it
clones the byte array. This leads to using twice as much memory, as the file
(at least).
I think this should be properly addressed and that it is quite easily
reproducible.
> OutOfMemoryError duriong ivy:publish
> ------------------------------------
>
> Key: IVY-1197
> URL: https://issues.apache.org/jira/browse/IVY-1197
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0
> Reporter: Michael Rumpf
> Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] published ppg to
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
> The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:2786)
> at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
> at
> org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
> at
> org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
> at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
> at
> org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
> at
> org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
> at
> org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
> at
> org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
> at
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
> at
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
> at
> org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
> at org.apache.ivy.Ivy.publish(Ivy.java:600)
> at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
> at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
> at
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
> at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
> at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
> at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around
> 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are
> published to an Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
