[
https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325308#comment-14325308
]
Sebb commented on IO-468:
-------------------------
I assume the Error column is the margin of error?
This increases dramatically (as a percentage) for the largest copy size.
Perhaps this indicates garbage collection starting to become signficant?
I'm suprised that copyThreadLocal should appear to be slighlty faster than
copyFixedArray for the larger copy sizes.
That seems very counter-intuitive; I suspect is is an artefact of using lots
more memory for the copied data.
I wonder if it makes sense to use memory arrays for the test, especially the
larger sizes which will swamp the buffer size.
It's unlikely that such copies will ever be a common use-case - much more
likely is copying files.
For the smaller data buffer sizes, this shows that a fixed array is the fastest
- as was to be expected.
It might make sense to pick a fixed smaller data buffer and vary the copy
buffer size.
Once the copy buffer has been made available to the copyLarge method, it is
fixed for the duration of the copy method, so does it really matter how many
bytes travel through it?
> Avoid allocating memory for method internal buffers, use threadlocal memory
> instead
> -----------------------------------------------------------------------------------
>
> Key: IO-468
> URL: https://issues.apache.org/jira/browse/IO-468
> Project: Commons IO
> Issue Type: Improvement
> Components: Utilities
> Affects Versions: 2.4
> Environment: all environments
> Reporter: Bernd Hopp
> Priority: Minor
> Labels: newbie, performance
> Fix For: 2.5
>
> Attachments: PerfTest.java, micro-benchmark.zip,
> monitoring_with_threadlocals.png, monitoring_without_threadlocals.png,
> performancetest.ods, performancetest_weakreference.ods
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This
> is a performance drawback since many of these allocations could be avoided if
> we would use threadlocal buffers that can be reused. For example, consider
> the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new
> byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used
> outside of the method and could easily and safely reused, as long as is is
> thread-local. So instead of allocating new memory, a new utility-class could
> provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to
> be significant, especially when the streams itself are not the performance
> bottleneck.
> Git PR is at https://github.com/apache/commons-io/pull/6/files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
