[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873349#comment-15873349
]
Luke Miner commented on HADOOP-13811:
-
[~steve_l] As an update, we're starting to see another error
Luke Miner created HADOOP-14101:
---
Summary: MultiObjectDeleteException thrown when writing directly
to s3a
Key: HADOOP-14101
URL: https://issues.apache.org/jira/browse/HADOOP-14101
Project: Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876307#comment-15876307
]
Luke Miner commented on HADOOP-13811:
-
Steve,
I'll give the new JARs a shot. I change the setting
[
https://issues.apache.org/jira/browse/HADOOP-14101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Miner updated HADOOP-14101:
Component/s: fs/s3
> MultiObjectDeleteException thrown when writing directly to s3a
>
[
https://issues.apache.org/jira/browse/HADOOP-14101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878720#comment-15878720
]
Luke Miner commented on HADOOP-14101:
-
Thanks for explaining the problem with writing directly to
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657802#comment-15657802
]
Luke Miner commented on HADOOP-13811:
-
Per [SPARK-18402] I am getting this stack trace. Would be
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684576#comment-15684576
]
Luke Miner commented on HADOOP-13811:
-
Okay built off the PR
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684576#comment-15684576
]
Luke Miner edited comment on HADOOP-13811 at 11/21/16 8:07 PM:
---
Okay built
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703276#comment-15703276
]
Luke Miner edited comment on HADOOP-13811 at 11/28/16 10:07 PM:
That
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703276#comment-15703276
]
Luke Miner commented on HADOOP-13811:
-
That worked great. It is running!
However, there's a new
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668288#comment-15668288
]
Luke Miner commented on HADOOP-13811:
-
[~steve_l] anything you'd like me to do to try to diagnose
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15661864#comment-15661864
]
Luke Miner commented on HADOOP-13811:
-
Yeah, this happens every time I run this job.
> s3a:
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681899#comment-15681899
]
Luke Miner commented on HADOOP-13811:
-
I built a snapshot of spark 2.1.0 with a build of hadoop
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706266#comment-15706266
]
Luke Miner edited comment on HADOOP-13811 at 11/29/16 7:22 PM:
---
Turns out
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706266#comment-15706266
]
Luke Miner commented on HADOOP-13811:
-
Turns out I still had {{hadoop-aws:2.7.3}} in my spark conf
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706266#comment-15706266
]
Luke Miner edited comment on HADOOP-13811 at 11/29/16 7:23 PM:
---
Turns out
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732908#comment-15732908
]
Luke Miner commented on HADOOP-13811:
-
Sorry to be a pain about this. Would it be possible for you
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709517#comment-15709517
]
Luke Miner commented on HADOOP-13811:
-
I've been looking through the classpath but can't find any
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713133#comment-15713133
]
Luke Miner commented on HADOOP-13811:
-
So I printed out the classpath and it points to the snapshot
[
https://issues.apache.org/jira/browse/HADOOP-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716636#comment-15716636
]
Luke Miner commented on HADOOP-13811:
-
I double checked and {{SPARK_HOME}} is unset and there
Luke Miner created HADOOP-14161:
---
Summary: Failed to rename S3AFileStatus
Key: HADOOP-14161
URL: https://issues.apache.org/jira/browse/HADOOP-14161
Project: Hadoop Common
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HADOOP-14161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901815#comment-15901815
]
Luke Miner edited comment on HADOOP-14161 at 3/8/17 7:29 PM:
-
[
https://issues.apache.org/jira/browse/HADOOP-14161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Miner updated HADOOP-14161:
Description:
I'm getting non deterministic rename errors while writing to S3 using spark and
[
https://issues.apache.org/jira/browse/HADOOP-14161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901815#comment-15901815
]
Luke Miner commented on HADOOP-14161:
-
[~ste...@apache.org] FYI. Incidentally, we're trying to move
24 matches
Mail list logo