Hi Ryan, Thanks for the proposal. Here are some questions and comments.
1. Are we going to open for other ways to store the table metadata instead of the metadata.json files? For example, a relational database or a key-value database. This will be a big change if that’s so, since it changes the assumption that the table version is preserved as a file. 2. Assume that we still keep metadata.json files, does that mean the catalog server needs to write a new metadata.json for each commit, which needs the permission to access the file system where the table is located? It is tricky to do that for some users whose catalog doesn’t have the access of the file system. 3. What’s the major benefits of the REST catalog APIs? Considering that the current user has to make a lot of changes to adopt it, either from client side and server side. The client side should be fine since Iceberg may provide it, but the server side is a big task for existing users. 4. How does an existing catalog support the new APIs? For example, HMS may need to add an extra layer or a plugin to support the server side of the APIs. Best, Yufei `This is not a contribution` On Mon, Jan 3, 2022 at 10:42 AM Ryan Blue <b...@tabular.io> wrote: > Hi everyone, > > I wrote up a design doc that covers making changes to Iceberg tables in > the REST catalog API > <https://docs.google.com/document/d/1D0R3G0slssEhggH5XnIzMwsUIP-c385Qp2sjv5E7e6E/edit?usp=sharing> > . > > The doc outlines the current changes to metadata, identifies a more > orthogonal set of changes that we can use, and outlines conflict detection. > In the end, the proposed API is to send a list of requirements, like > "current snapshot must be <ID>", and a list of changes, like "add snapshot > ..." and "set current snapshot to <ID>". > > Please have a look and discuss! > > Ryan > > -- > Ryan Blue > Tabular >
