flyrain commented on issue #3997:
URL: https://github.com/apache/iceberg/issues/3997#issuecomment-1042349608
Thanks for sharing. The Hudi metadata server makes sense generally. However,
Iceberg doesn’t have the some of issues in Hudi, for example, file listing
issue in Hudi metadata.
I list some benefits of an Iceberg metadata server.
1. Multiple-table transaction
2. Performance improvement
1. Avoid send the full metadata json file from client to server
2. Queuing multiple commits, other than each client resolves
write-write conflict by themselves.
3. Caching resources like JDBC connections
3. Safer commits
1. Clients with different versions can safely commit, without
worrying about overwriting the newer properties
2. Server side holds the truth of table format, upgrade to a newer
version doesn’t need changes of all clients as long as API compatibility is
kept.
There could be more benefits though.
References
* Related PRs: https://github.com/apache/iceberg/pull/3770
* Table metadata Rest API design Doc:
https://docs.google.com/document/d/1D0R3G0slssEhggH5XnIzMwsUIP-c385Qp2sjv5E7e6E/edit#heading=h.eo4x0coo8esy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
