ravwojdyla opened a new pull request, #1240: URL: https://github.com/apache/parquet-mr/pull/1240
The surface of the synchronized methods is unnecessarily large which in case of large number of concurrent writers may be suboptimal, threads may starve each other for access to the writerList (if writers are being dynamically closed and created). Please see the profile reports in the Jira issue. The fix in this PR is to use `ConcurrentHashMap` for `MemoryManager`'s `writerList` instead of sync methods. ### Jira - [x] My PR addresses the following https://issues.apache.org/jira/browse/PARQUET-2412 issues and references them in the PR title. ### Tests - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason: there are already tests for `MemoryManager`. ### Commits - [x] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" ### Style - [x] My contribution adheres to the code style guidelines and Spotless passes. ### Documentation - [x] In case of new functionality, my PR adds documentation that describes how to use it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
