[
https://issues.apache.org/jira/browse/HDFS-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648213#comment-16648213
]
Daryn Sharp commented on HDFS-13988:
------------------------------------
I'm -1 if it's in the NN. The NN's primary role is maintaining data durability.
Not managing the auto-delete, auto-rename, auto-copy, auto-storage-downgrade,
etc data policies that will surely follow. Over the years we've had the same
debates with the balancer, the mover, etc. If you want to create an optional
external service that doesn't alter apis, that's ok.
If you have problems with users scribbling outside their staging/output dirs,
made you should tell them to stop or provide a scribble directory that is
regularly purged.
Note this is an old idea. See: HDFS-6382 "HDFS File/Directory TTL". This jira
really should be dupped to that one.
> Allow for Creating Temporary Files
> ----------------------------------
>
> Key: HDFS-13988
> URL: https://issues.apache.org/jira/browse/HDFS-13988
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs, hdfs-client
> Affects Versions: 3.2.0
> Reporter: BELUGA BEHR
> Priority: Major
>
> Allow clients to create temporary files in HDFS.
> {code:java|title=FileSystem.java}
> public FSDataOutputStreamBuilder createFile(Path path);
> public FSDataOutputStreamBuilder appendFile(Path path);
> // New feature
> public FSDataOutputStreamBuilder createTempFile(Path path, TimeUnit unit,
> long ttl);
> {code}
> This would register the file with the NameNode to have a TTL. The NameNode
> would be responsible for deleting the files after some custom TTL. This
> would be helpful in the case that a MapReduce (or Spark) job fails during
> execution and does not properly clean up after itself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
