[
https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311556#comment-14311556
]
Bernd Hopp commented on IO-468:
-------------------------------
[~tn]
please look again, there is a jit-phase at the start of main. It's where the
comment says 'jit everything'.
Now for your comments before: It seems like your concerns boil down to memory
leakage in environments with massive amounts of threads. As I said before, I
don't share these concerns and I am still waiting for you to show me a test
case with reasonable parameters where memory usage is at least on MB higher
than without TLs.
The link you posted is about threadlocals for classloaders, which I agree is a
bad idea, but a totally different thing since classloaders may hold references
to other resources that can subsequently not get gc'ed. The link to
stackoverflow seems to contain only general informations on TLs, so let me
assure you: I do know how they work.
But the discussion is stuck here, so I just pushed a commit that makes the
threadlocals use WeakReferences, so that possible memory leaks are prevented.
The performance is not as good as without WeakReferences, but still 5 times
better than the current implementation, see attached file
performancetest_weakreferences.ods. Does that convince you? If not, what would?
Regards
Bernd
> Avoid allocating memory for method internal buffers, use threadlocal memory
> instead
> -----------------------------------------------------------------------------------
>
> Key: IO-468
> URL: https://issues.apache.org/jira/browse/IO-468
> Project: Commons IO
> Issue Type: Improvement
> Components: Utilities
> Affects Versions: 2.4
> Environment: all environments
> Reporter: Bernd Hopp
> Priority: Minor
> Labels: newbie, performance
> Fix For: 2.5
>
> Attachments: PerfTest.java, monitoring_with_threadlocals.png,
> monitoring_without_threadlocals.png, performancetest.ods,
> performancetest_weakreference.ods
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This
> is a performance drawback since many of these allocations could be avoided if
> we would use threadlocal buffers that can be reused. For example, consider
> the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new
> byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used
> outside of the method and could easily and safely reused, as long as is is
> thread-local. So instead of allocating new memory, a new utility-class could
> provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to
> be significant, especially when the streams itself are not the performance
> bottleneck.
> Git PR is at https://github.com/apache/commons-io/pull/6/files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
