It's really Hadoop's support for S3. Hadoop FS semantics need directories, and S3 doesn't have a proper notion of directories.
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/s3native/NativeS3FileSystem.html You should leave them AFAIK. On Wed, Sep 24, 2014 at 6:41 PM, Shay Seng <[email protected]> wrote: > Hi, > Why does RDD.saveAsObjectFile() to S3 leave a bunch of *_$folder$ empty > files around? Is it possible for the saveas to clean up? > > tks --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
