Steve Loughran created HADOOP-16570:
---------------------------------------
Summary: S3A committers leak threads on job commit
Key: HADOOP-16570
URL: https://issues.apache.org/jira/browse/HADOOP-16570
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Affects Versions: 3.1.2, 3.2.0
Reporter: Steve Loughran
Assignee: Steve Loughran
The fixed size ThreadPool created in AbstractS3ACommitter doesn't get cleaned
up at EOL; as a result you leak the no. of threads set in
"fs.s3a.committer.threads"
Not visible in MR/distcp jobs, but ultimately causes OOM on Spark
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
