[ https://issues.apache.org/jira/browse/HADOOP-13826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799044#comment-15799044 ]
Sean Mackrory commented on HADOOP-13826: ---------------------------------------- I'll rev the patch on Steve's last feedback. It would be nice to get this into 2.8 if it doesn't delay things. > S3A Deadlock in multipart copy due to thread pool limits. > --------------------------------------------------------- > > Key: HADOOP-13826 > URL: https://issues.apache.org/jira/browse/HADOOP-13826 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 > Affects Versions: 2.7.3 > Reporter: Sean Mackrory > Assignee: Sean Mackrory > Priority: Critical > Attachments: HADOOP-13826.001.patch, HADOOP-13826.002.patch, > HADOOP-13826.003.patch > > > In testing HIVE-15093 we have encountered deadlocks in the s3a connector. The > TransferManager javadocs > (http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html) > explain how this is possible: > {quote}It is not recommended to use a single threaded executor or a thread > pool with a bounded work queue as control tasks may submit subtasks that > can't complete until all sub tasks complete. Using an incorrectly configured > thread pool may cause a deadlock (I.E. the work queue is filled with control > tasks that can't finish until subtasks complete but subtasks can't execute > because the queue is filled).{quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org