[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742453#comment-14742453
]
Steve Loughran commented on SPARK-4879:
---
What about splitting the issue into HDFS commits
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708331#comment-14708331
]
Igor Berman commented on SPARK-4879:
there is blog post
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631463#comment-14631463
]
Josh Rosen commented on SPARK-4879:
---
That's a good point about the post-save-action
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628380#comment-14628380
]
Daniel Darabos commented on SPARK-4879:
---
Thanks, Josh! I wonder when the
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628086#comment-14628086
]
Daniel Darabos commented on SPARK-4879:
---
I've managed to reproduce on Spark 1.3.1
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628300#comment-14628300
]
Josh Rosen commented on SPARK-4879:
---
I think that this is going to be a complex issue to
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14623956#comment-14623956
]
Josh Rosen commented on SPARK-4879:
---
[~darabos], do you think that this issue might have
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14624000#comment-14624000
]
Daniel Darabos commented on SPARK-4879:
---
Good idea! I'll try with 1.3.1 next week.
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1466#comment-1466
]
Daniel Darabos commented on SPARK-4879:
---
I wonder if this issue is serious enough to
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620570#comment-14620570
]
Daniel Darabos commented on SPARK-4879:
---
I have a fairly reliable reproduction with
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586266#comment-14586266
]
Igor Berman commented on SPARK-4879:
I'm experiencing this issue. Sometimes rdd with 4
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572429#comment-14572429
]
Peng Zhen commented on SPARK-4879:
--
Hi, is there any updates for this issue?
Missing
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348987#comment-14348987
]
Mridul Muralidharan commented on SPARK-4879:
[~joshrosen] The former - the
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348522#comment-14348522
]
Mridul Muralidharan commented on SPARK-4879:
With 1.3 RC, we are still seeing
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349188#comment-14349188
]
Matt Cheah commented on SPARK-4879:
---
Can you perhaps jstack or profile the driver and
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348527#comment-14348527
]
Mridul Muralidharan commented on SPARK-4879:
Just to add - we are seeing this
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14348590#comment-14348590
]
Josh Rosen commented on SPARK-4879:
---
[~mridulm80], when you say that the call itself
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325969#comment-14325969
]
Andrew Ash commented on SPARK-4879:
---
That sounds exactly like what I'd expect to happen
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324418#comment-14324418
]
Andrew Ash commented on SPARK-4879:
---
[~romi-totango] what filesystem are you writing to?
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324278#comment-14324278
]
Romi Kuntsman commented on SPARK-4879:
--
Could this happen very very rarely when not
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315087#comment-14315087
]
Andrew Ash commented on SPARK-4879:
---
This is really great work [~joshrosen]! I really
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315077#comment-14315077
]
Josh Rosen commented on SPARK-4879:
---
This issue is _really_ hard to reproduce, but I
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286771#comment-14286771
]
Apache Spark commented on SPARK-4879:
-
User 'mccheah' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279406#comment-14279406
]
Josh Rosen commented on SPARK-4879:
---
I'm not sure that SparkHadoopWriter's use of
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279711#comment-14279711
]
Apache Spark commented on SPARK-4879:
-
User 'JoshRosen' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274019#comment-14274019
]
Josh Rosen commented on SPARK-4879:
---
I think that part of the reproduction issues that I
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272291#comment-14272291
]
Zach Fry commented on SPARK-4879:
-
Hey Josh,
I was able to reproduce the missing file
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14272292#comment-14272292
]
Zach Fry commented on SPARK-4879:
-
For clarity, here is the scala code I used in the REPL:
[
https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271788#comment-14271788
]
Josh Rosen commented on SPARK-4879:
---
Hey Zach,
From my last round of attempts (maybe a
29 matches
Mail list logo
