[GitHub] incubator-spark pull request: [SPARK-1100] prevent Spark from over...

[CodingCat]

Github user CodingCat commented on the pull request:

    https://github.com/apache/incubator-spark/pull/626#issuecomment-35760516
  
    @mridulm I tested that and found that it is actually not handled in Spark, 
    
    
![abc](https://f.cloud.github.com/assets/678008/2233295/4a6efde6-9b28-11e3-9877-02a8829ac056.png)
    
    part-00000 and part-00001 are generated by the second run of the job (the 
partition is set to 2)
    
    part-00002 and part-00003 are generated by the first run of (partition 
number of is set to 4), 
    
    I also found this when I run Spark over S3, the old file is still left over 
in the output directory



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastruct...@apache.org or file a JIRA ticket with INFRA.
---