szehon-ho commented on code in PR #18057:
URL: https://github.com/apache/iceberg/pull/18057#discussion_r3993764905
##########
format/udf-spec.md:
##########
@@ -118,19 +122,22 @@ following fields required. Any other fields must be
ignored.
e.g., `{ "type": "struct", "fields": [ { "name": "id", "type": "int" }, {
"name": "name", "type": "string" } ] }`
#### Definition ID
-The `definition-id` is a canonical string derived from the parameter types,
formatted as a comma-separated list with no
-spaces. Each type uses the following string representation:
+The `definition-id` is a canonical string derived from the parameter types,
formatted as a comma-separated list. The
+separators that this format adds must not be followed by a space. Each type
uses the following string representation:
* Primitives and semi-structured: the type name (e.g., `int`, `variant`)
* List: `list<element-type>` (e.g., `list<int>`)
* Map: `map<key-type,value-type>` (e.g., `map<string,int>`)
* Struct: `struct<name1:type1,name2:type2,...>` with field names and types
(e.g., `struct<id:int,name:string>`)
+A type string is embedded unchanged, so a parameterized type may still
contribute spaces of its own (see [Types](#types)).
Review Comment:
Please derive `definition-id` from normalized type identity rather than
embedding the serialized string unchanged. Iceberg treats geometry/geography
CRS values case-insensitively, but serialization preserves the supplied casing,
so equal types such as `geometry(EPSG:4326)` and `geometry(epsg:4326)` can
otherwise produce different IDs. That conflicts with the requirement that only
one definition exist per signature. The spec should define the normalization
used for the ID, or explicitly define UDF type identity differently from
Iceberg type equality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
