gallushi opened a new pull request #1871:
URL: https://github.com/apache/iceberg/pull/1871
This PR introduces support for File System with Atomic Write guarantees in
Hadoop Tables.
until now, hadoop tables required Atomic Rename from the underlying FS, now
they can also be configured to use Atomic Write.
when using Atomic Write, the final metadata file will be (atomically)
written instead of using a tempfile and atomically renaming it (this is till
used when atomic rename, which remains the default, is used).
To enable the use of Atomic Write, (and assuming this guarantee is indeed
supplied by the FS), a new config param is added.
`iceberg.engine,hadoop.<SCHEME>.atomic.write` (defaults to `false`) where
`SCHEME` is the FS scheme (e.g., `{file, hdfs, cos}`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
