Alkis Evlogimenos created HADOOP-19251:
------------------------------------------
Summary: Add Options.Rename.THROW_NON_ATOMIC
Key: HADOOP-19251
URL: https://issues.apache.org/jira/browse/HADOOP-19251
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Affects Versions: 3.3.6
Reporter: Alkis Evlogimenos
I propose we add an option `Options.Rename.THROW_NON_ATOMIC` to change
`rename()` behavior to throw when the underlying filesystem's rename operation
is not atomic.
This would be useful for callers that expect to perform an atomic op but want
to fail if when an atomic rename fails.
At first this might seem something that can be done by querying capabilities of
the filesystem but that would only work on real filesystems. A motivating
example would be a virtual filesystem for which paths can resolve to any
concrete filesystem (s3, etc). If `rename()` is called with two virtual paths
that resolve to different filesystems (s3 and gcs for example) then obviously
the operation can't be atomic since bytes must be copied from one fs to another.
What do you think [~steve_l] ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
