Thanks for the reply Jack! 1. That is great to have! However, we might also like to preserve order among the key columns. What's the reason that it is a set?
2. Sorry maybe I wasn't clear about the use-case. We are working to extend Iceberg for our users. Here I meant metadata that is of our users' interest but not related to Iceberg's functionality. For example, the table owner may want to note that "Table A is about my research on topic B, and it should be published to group C after 2022Q1". This is not a P0 requirement to us and I just want to learn in case it is available. It seems to me that *table.properties* aren't intended for it according to "This is used to control settings that affect reading and writing and is not intended to be used for arbitrary metadata. <https://iceberg.apache.org/spec/#table-metadata-fields>" 3. Naming snapshot is a feature that we want to support. For example, it is convenient for a user who often time-travel to a weekly checkpoint which is named by formatted-date-string instead of a random int. This feature is also related to compaction, which was discussed in a previous thread and meeting, that we hope to compact and replace a named Snapshot so that the time-travel reader gets the better performance, which means the SnapshotSummary could contain information about compaction. Let alone the compaction (I know the current proposal of compaction doesn't work in that way), does it sound reasonable to add an optional "name" field in the SnapshotSummary for this? On Fri, May 14, 2021 at 12:16 PM Jack Ye <[email protected]> wrote: > 1. The primary key concept is now added to Iceberg as the identifier > concept in schema. I am in the progress of adding the documentation. You > can read the javadoc for more details for now: > https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/Schema.java#L189-L210 > > 2. Yes, properties is the one for user metadata. > > 3. Why do you want to store additional configs and names for snapshot? The > snapshot.summary field is written by engine and has these defined fields: > https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/SnapshotSummary.java > > -Jack > > On Fri, May 14, 2021 at 8:49 AM QH Yan <[email protected]> wrote: > >> Hi there, >> We want to store 3 different kinds of metadata/config in Iceberg tables. >> >> 1. Additional settings/admin properties for a table (e.g. PrimaryKey info) >> I think it is* table.properties* according to here >> <https://iceberg.apache.org/spec/#table-metadata-fields> and would like >> to confirm. >> >> 2. User metadata at table level. >> Sometimes user wants to take notes about a table. Is there a field for >> this? (map of String is good enough and I don't want to abuse >> table.properties also as the doc points out) >> >> 3. Snapshot name and additional configs >> Seems that* snapshot.summary* is for both of these according to here >> <https://iceberg.apache.org/spec/#snapshots>, am I right? >> >> Thank you! >> >> -- >> *Qinhua* >> >> -- *Qinhua*
