shangxinli commented on PR #959:
URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1193390004
@theosib-amazon, I am not concerned if release/close isn't called and I
agree the caller must call release/close after finishing. My question is that
before release/close is called, there could be short-living threads that are
used to create the compressor/decompressor in the cache. Those short-living
threads exit and the cache is not aware of that, then that causes the cache
grows with a lot of dead compressor/decompressors. In the scenario where
short-living threads just come and go as a normal business, this could be a
problem. I know normally it is not a problem because in most of the cases we
use thread pool but I am just not sure there is a corner case like that.
Parquet is a low-level library and is used in so many cases.
I am sorry if I didn't make my previous comment more obvious.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
