[
https://issues.apache.org/jira/browse/HUDI-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BALAJI VARADARAJAN reopened HUDI-104:
-------------------------------------
> Replace ThreadLocal with Resource pool when caching resource type classes
> -------------------------------------------------------------------------
>
> Key: HUDI-104
> URL: https://issues.apache.org/jira/browse/HUDI-104
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: Common Core, Performance
> Reporter: BALAJI VARADARAJAN
> Priority: Major
>
> This is another comment that stemmed from Timeline Server code-review.
>
> Notes from PR comments:
> '''RandomAccessFile is not threadsafe. Hence, we would have to create
> separate RandomAccessFileObjects for concurrent access. Agree, the current
> approach of opening a new file per get() call is slow. I have made changes to
> use ThreadLocal objects. This should reduce the open() calls greatly but for
> a long running timeline service, there could be open handles getting
> accumulated per thread. A better approach would be to use Resource Pool but
> needs to be config tuned and tested. There is another place (ThreadLocal
> where we might require similar treatment. Will open a ticket to address it.
> If current perf testing points to the direction of Resource pool, will
> address it as part of this PR. Otherwise, we can take this in subsequent PR'''
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
