rdblue commented on a change in pull request #2654:
URL: https://github.com/apache/iceberg/pull/2654#discussion_r652201213
##########
File path: site/docs/spec.md
##########
@@ -176,6 +178,15 @@ Columns in Iceberg data files are selected by field id.
The table schema's colum
For example, a file may be written with schema `1: a int, 2: b string, 3: c
double` and read using projection schema `3: measurement, 2: name, 4: a`. This
must select file columns `c` (renamed to `measurement`), `b` (now called
`name`), and a column of `null` values called `a`; in that order.
+#### Identifier Field IDs
+
+A schema can optionally track the set of primitive fields that identify rows
in a table, using the property `identifier-field-ids` (see JSON encoding in
Appendix C).
+
+Two rows are the "same"---that is, the rows represent the same entity---if the
identifier fields are equal. However, uniqueness of rows by this identifier is
not guaranteed or required by Iceberg because it is needlessly inefficient in
some cases. Reasonable steps to enforce uniqueness are the responsibility of
processing engines.
+
+Optional fields can be used as identifier fields. Identifier fields may be
nested in structs but may not be nested within maps or lists; a nested
identifier field is considered null if it or any parent struct is null. For
identifier comparison, null is considered equal to itself.
Review comment:
I'm going to update this so that float and double columns aren't
allowed. I think there are too many issues if we were to allow them and
implementations would probably have bugs handling signalling NaN values and -0.
We already don't allow floating points in some places where they make little
sense (like bucket partitioning) so I think being conservative here is the
right call.
Also, I doubt there are many valid use cases for having a floating point
value in a row identifier.
And while I'm updating this, I'm also going to disallow float and double
columns used for delete columns in equality delete files because of the same
issues with signalling NaN and -0 values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
