electrum commented on a change in pull request #3556:
URL: https://github.com/apache/iceberg/pull/3556#discussion_r750712833



##########
File path: site/docs/spec.md
##########
@@ -212,6 +212,24 @@ Columns in Iceberg data files are selected by field id. 
The table schema's colum
 
 For example, a file may be written with schema `1: a int, 2: b string, 3: c 
double` and read using projection schema `3: measurement, 2: name, 4: a`. This 
must select file columns `c` (renamed to `measurement`), `b` (now called 
`name`), and a column of `null` values called `a`; in that order.
 
+Tables may also define a property `schema.name-mapping.default` with a JSON 
`name mapping` containing a list of `field mapping` objects. These mappings 
provide fallback field ids to be used when a data file does not contain field 
id information. Each object should contain
+
+* `names`: A required list of 0 or more names for a field. 
+* `field-id`: An optional Iceberg field ID used when a field's name is present 
in `names`
+* `fields`: An optional list of field mappings for child field of structs, 
maps, and lists.
+
+Field mapping fields are constrained by the following rules
+
+* A name may contain `.` but this refers to a literal name, not a nested 
field. For example, `a.b` refers to a field named `a.b`, not child field `b` of 
field `a`. 
+* Each child field should be defined with their own `field-mapping` under 
`fields`. Multiple values for `names` may be mapped to a single field ID to 
support cases where a field may have different names in different data files. 

Review comment:
       It seems like the second sentence should be part of the next item.
   
   * Each child field should be defined with their own `field-mapping` under 
`fields`.
   * Multiple values for `names` may be mapped to a single field ID to support 
cases where a field may have different names in different data files. For 
example, all Avro field aliases should be listed in `names`.
   

##########
File path: site/docs/spec.md
##########
@@ -212,6 +212,24 @@ Columns in Iceberg data files are selected by field id. 
The table schema's colum
 
 For example, a file may be written with schema `1: a int, 2: b string, 3: c 
double` and read using projection schema `3: measurement, 2: name, 4: a`. This 
must select file columns `c` (renamed to `measurement`), `b` (now called 
`name`), and a column of `null` values called `a`; in that order.
 
+Tables may also define a property `schema.name-mapping.default` with a JSON 
`name mapping` containing a list of `field mapping` objects. These mappings 
provide fallback field ids to be used when a data file does not contain field 
id information. Each object should contain
+
+* `names`: A required list of 0 or more names for a field. 
+* `field-id`: An optional Iceberg field ID used when a field's name is present 
in `names`
+* `fields`: An optional list of field mappings for child field of structs, 
maps, and lists.
+
+Field mapping fields are constrained by the following rules
+
+* A name may contain `.` but this refers to a literal name, not a nested 
field. For example, `a.b` refers to a field named `a.b`, not child field `b` of 
field `a`. 
+* Each child field should be defined with their own `field-mapping` under 
`fields`. Multiple values for `names` may be mapped to a single field ID to 
support cases where a field may have different names in different data files. 
+* For example, all Avro field aliases should be listed in `names`.
+* Fields which exist only in the Iceberg schema and not in imported data files 
may be included as `field-mapping`s with an empty `names` list.
+* Fields that exist in imported files but not in the Iceberg schema may omit 
`field-id`.
+* List types should contain a mapping in `fields` for `element`. 
+* Map types should contain mappings in `fields` for `key` and `value`. 
+* Struct types should contain mappings inf `fields` for their child fields.
+
+For details on serialization see [Appendix C](#name-mapping-serialization)

Review comment:
       Nit: comma before "see" and period at end of sentence

##########
File path: site/docs/spec.md
##########
@@ -989,6 +1007,26 @@ Table metadata is serialized as a JSON object according 
to the following table.
 |**`sort-orders`**|`JSON sort orders (list of sort field object)`|`See above`|
 |**`default-sort-order-id`**|`JSON int`|`0`|
 

Review comment:
       Nit: looks like the convention is to have two blank lines between 
sections, so add one here?

##########
File path: site/docs/spec.md
##########
@@ -212,6 +212,24 @@ Columns in Iceberg data files are selected by field id. 
The table schema's colum
 
 For example, a file may be written with schema `1: a int, 2: b string, 3: c 
double` and read using projection schema `3: measurement, 2: name, 4: a`. This 
must select file columns `c` (renamed to `measurement`), `b` (now called 
`name`), and a column of `null` values called `a`; in that order.
 
+Tables may also define a property `schema.name-mapping.default` with a JSON 
`name mapping` containing a list of `field mapping` objects. These mappings 
provide fallback field ids to be used when a data file does not contain field 
id information. Each object should contain
+
+* `names`: A required list of 0 or more names for a field. 
+* `field-id`: An optional Iceberg field ID used when a field's name is present 
in `names`
+* `fields`: An optional list of field mappings for child field of structs, 
maps, and lists.
+
+Field mapping fields are constrained by the following rules

Review comment:
       Nit: colon at end of sentence
   
   I like this new formatting as a list of rules -- makes it easy to read.

##########
File path: site/docs/spec.md
##########
@@ -212,6 +212,24 @@ Columns in Iceberg data files are selected by field id. 
The table schema's colum
 
 For example, a file may be written with schema `1: a int, 2: b string, 3: c 
double` and read using projection schema `3: measurement, 2: name, 4: a`. This 
must select file columns `c` (renamed to `measurement`), `b` (now called 
`name`), and a column of `null` values called `a`; in that order.
 
+Tables may also define a property `schema.name-mapping.default` with a JSON 
`name mapping` containing a list of `field mapping` objects. These mappings 
provide fallback field ids to be used when a data file does not contain field 
id information. Each object should contain
+
+* `names`: A required list of 0 or more names for a field. 
+* `field-id`: An optional Iceberg field ID used when a field's name is present 
in `names`
+* `fields`: An optional list of field mappings for child field of structs, 
maps, and lists.
+
+Field mapping fields are constrained by the following rules
+
+* A name may contain `.` but this refers to a literal name, not a nested 
field. For example, `a.b` refers to a field named `a.b`, not child field `b` of 
field `a`. 
+* Each child field should be defined with their own `field-mapping` under 
`fields`. Multiple values for `names` may be mapped to a single field ID to 
support cases where a field may have different names in different data files. 
+* For example, all Avro field aliases should be listed in `names`.
+* Fields which exist only in the Iceberg schema and not in imported data files 
may be included as `field-mapping`s with an empty `names` list.
+* Fields that exist in imported files but not in the Iceberg schema may omit 
`field-id`.
+* List types should contain a mapping in `fields` for `element`. 
+* Map types should contain mappings in `fields` for `key` and `value`. 
+* Struct types should contain mappings inf `fields` for their child fields.

Review comment:
       Typo "inf"

##########
File path: site/docs/spec.md
##########
@@ -212,6 +212,24 @@ Columns in Iceberg data files are selected by field id. 
The table schema's colum
 
 For example, a file may be written with schema `1: a int, 2: b string, 3: c 
double` and read using projection schema `3: measurement, 2: name, 4: a`. This 
must select file columns `c` (renamed to `measurement`), `b` (now called 
`name`), and a column of `null` values called `a`; in that order.
 
+Tables may also define a property `schema.name-mapping.default` with a JSON 
`name mapping` containing a list of `field mapping` objects. These mappings 
provide fallback field ids to be used when a data file does not contain field 
id information. Each object should contain
+
+* `names`: A required list of 0 or more names for a field. 
+* `field-id`: An optional Iceberg field ID used when a field's name is present 
in `names`
+* `fields`: An optional list of field mappings for child field of structs, 
maps, and lists.
+
+Field mapping fields are constrained by the following rules
+
+* A name may contain `.` but this refers to a literal name, not a nested 
field. For example, `a.b` refers to a field named `a.b`, not child field `b` of 
field `a`. 
+* Each child field should be defined with their own `field-mapping` under 
`fields`. Multiple values for `names` may be mapped to a single field ID to 
support cases where a field may have different names in different data files. 
+* For example, all Avro field aliases should be listed in `names`.
+* Fields which exist only in the Iceberg schema and not in imported data files 
may be included as `field-mapping`s with an empty `names` list.

Review comment:
       The pluralization of the JSON name looks a bit funny. Maybe change the 
sentence to avoid the need for it:
   
   * Each field which exists only in the Iceberg schema and not in imported 
data files may be included as a `field-mapping` with an empty `names` list.
   
   Or change it to match the below _Fields that exist_ rule:
   
   * Fields which exist only in the Iceberg schema and not in imported data 
files may use an empty `names` list.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to