[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182187#comment-17182187 ] Dyno commented on HADOOP-17201: --- what i can confirm now is - the bucket is versioned. - HADOOP-17063 is

[jira] [Updated] (HADOOP-17201) Spark job stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17201: -- Description: usually our spark job took 1 hour or 2 to finish, occasionally it runs for more than 3 hour

[jira] [Updated] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17201: -- Summary: Spark job with s3acommitter stuck at the last stage (was: Spark job stuck at the last stage) >

[jira] [Created] (HADOOP-17201) Spark job stuck at the last stage

Dyno created HADOOP-17201: - Summary: Spark job stuck at the last stage Key: HADOOP-17201 URL: https://issues.apache.org/jira/browse/HADOOP-17201 Project: Hadoop Common Issue Type: Bug

[jira] [Updated] (HADOOP-17201) Spark job stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17201: -- Description: usually our spark job took 1 hour or 2 to finish, occasionally it runs for more than 3 hour

[jira] [Updated] (HADOOP-17201) Spark job stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17201: -- Description: usually our spark job took 1 hour or 2 to finish, occasionally it runs for more than 3 hour

[jira] [Updated] (HADOOP-17201) Spark job stuck at the last stage

[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17201: -- Description: usually our spark job took 1 hour or 2 to finish, occasionally it runs for more than 3 hour

[jira] [Updated] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyno updated HADOOP-17063: -- Attachment: jstack_exec-74.log jstack_exec-40.log jstack_exec-34.log >

[jira] [Commented] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129802#comment-17129802 ] Dyno commented on HADOOP-17063: --- it happens again i attached the jstack. thanks for looking into it. i

[jira] [Comment Edited] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129802#comment-17129802 ] Dyno edited comment on HADOOP-17063 at 6/9/20, 9:03 PM: it happens again i

[jira] [Commented] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17132746#comment-17132746 ] Dyno commented on HADOOP-17063: --- > important: you are using the classic FileOutputCommitter; this is slow

[jira] [Commented] (HADOOP-17063) S3A deleteObjects hanging/retrying forever

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17133778#comment-17133778 ] Dyno commented on HADOOP-17063: --- we do have s3guard turns on, and let me try if magic works or not on our

[jira] [Commented] (HADOOP-17063) S3A deleteObjects hanging/retrying forever

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134609#comment-17134609 ] Dyno commented on HADOOP-17063: --- switch to magic looks working. thanks for your help.

[jira] [Created] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

Dyno created HADOOP-17063: - Summary: S3ABlockOutputStream.putObject looks stuck and never timeout Key: HADOOP-17063 URL: https://issues.apache.org/jira/browse/HADOOP-17063 Project: Hadoop Common

[jira] [Commented] (HADOOP-17063) S3ABlockOutputStream.putObject looks stuck and never timeout

[ https://issues.apache.org/jira/browse/HADOOP-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125413#comment-17125413 ] Dyno commented on HADOOP-17063: --- log in the spark executor, Jun 3, 2020 @ 22:57:23.032 2020-06-03