rdblue commented on a change in pull request #4301:
URL: https://github.com/apache/iceberg/pull/4301#discussion_r840012797
##########
File path: format/spec.md
##########
@@ -965,10 +976,31 @@ Types are serialized according to this table:
|**`fixed(L)`**|`JSON string: "fixed[<L>]"`|`"fixed[16]"`|
|**`binary`**|`JSON string: "binary"`|`"binary"`|
|**`decimal(P, S)`**|`JSON string: "decimal(<P>,<S>)"`|`"decimal(9,2)"`,<br
/>`"decimal(9, 2)"`|
-|**`struct`**|`JSON object: {`<br /> `"type": "struct",`<br
/> `"fields": [ {`<br /> `"id": <field id
int>,`<br /> `"name": <name string>,`<br
/> `"required": <boolean>,`<br
/> `"type": <type JSON>,`<br
/> `"doc": <comment string>`<br
/> `}, ...`<br /> `] }`|`{`<br
/> `"type": "struct",`<br /> `"fields": [ {`<br
/> `"id": 1,`<br /> `"name":
"id",`<br /> `"required": true,`<br
/> `"type": "uuid"`<br /> `}, {`<br
/> `"id": 2,`<br /> `"name":
"data",`<br /> `"required": false,`<br
/> `"type": {`<br
/> `"type": "list",`<br
/> `...`<br />
`}`<br /> `} ]`<br />`}`|
+|**`struct`**|`JSON object: {`<br /> `"type": "struct",`<br
/> `"fields": [ {`<br /> `"id": <field id
int>,`<br /> `"name": <name string>,`<br
/> `"required": <boolean>,`<br
/> `"type": <type JSON>,`<br
/> `"doc": <comment string>,`<br
/> `"initial-default": <JSON encoding of default
value>,`<br /> `"write-default": <JSON encoding of
default value>`<br /> `}, ...`<br /> `]
}`|`{`<br /> `"type": "struct",`<br /> `"fields": [ {`<br
/> `"id": 1,`<br /> `"name":
"id",`<br /> `"required": true,`<br
/> `"type": "uuid",`<br
/> `"initial-default":
"0db3e2a8-9d1d-42b9-aa7b-74ebe558dceb",`<br
/> `"write-default": "ec5911be
-b0a7-458c-8438-c9a3e53cffae"`<br /> `}, {`<br
/> `"id": 2,`<br /> `"name":
"data",`<br /> `"required": false,`<br
/> `"type": {`<br
/> `"type": "list",`<br
/> `...`<br /> `}`<br
/> `} ]`<br />`}`|
|**`list`**|`JSON object: {`<br /> `"type": "list",`<br
/> `"element-id": <id int>,`<br /> `"element-required":
<bool>`<br /> `"element": <type JSON>`<br />`}`|`{`<br
/> `"type": "list",`<br /> `"element-id": 3,`<br
/> `"element-required": true,`<br /> `"element":
"string"`<br />`}`|
|**`map`**|`JSON object: {`<br /> `"type": "map",`<br
/> `"key-id": <key id int>,`<br /> `"key": <type
JSON>,`<br /> `"value-id": <val id int>,`<br
/> `"value-required": <bool>`<br /> `"value": <type
JSON>`<br />`}`|`{`<br /> `"type": "map",`<br
/> `"key-id": 4,`<br /> `"key": "string",`<br
/> `"value-id": 5,`<br /> `"value-required": false,`<br
/> `"value": "double"`<br />`}`|
+For default values, the serialization depends on the type of the corresponding
column or nested field. The mapping of types and their corresponding default
value JSON serialization is described in the following table:
+
+| Type | Json type | Example
| Note
|
+|--------------------|--------------------|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **`boolean`** | **`json boolean`** | `true`
|
|
+| **`int`** | **`json int`** | `1`
|
|
+| **`long`** | **`json long`** | `1`
|
|
+| **`float`** | **`json float`** | `1.1`
|
|
+| **`double`** | **`json double`** | `1.1`
|
|
+| **`decimal(P,S)`** | **`string`** | `"0x3162"`
| Stores the unscaled value, as the two's-complement big-endian binary
using the minimum number of bytes, converted to a hexadecimal string prefixed
by `0x`
|
Review comment:
I think @RussellSpitzer suggested that we store these values as natural
strings and I agree. This should be `"123.45"` rather than the unscaled value.
##########
File path: format/spec.md
##########
@@ -965,10 +976,31 @@ Types are serialized according to this table:
|**`fixed(L)`**|`JSON string: "fixed[<L>]"`|`"fixed[16]"`|
|**`binary`**|`JSON string: "binary"`|`"binary"`|
|**`decimal(P, S)`**|`JSON string: "decimal(<P>,<S>)"`|`"decimal(9,2)"`,<br
/>`"decimal(9, 2)"`|
-|**`struct`**|`JSON object: {`<br /> `"type": "struct",`<br
/> `"fields": [ {`<br /> `"id": <field id
int>,`<br /> `"name": <name string>,`<br
/> `"required": <boolean>,`<br
/> `"type": <type JSON>,`<br
/> `"doc": <comment string>`<br
/> `}, ...`<br /> `] }`|`{`<br
/> `"type": "struct",`<br /> `"fields": [ {`<br
/> `"id": 1,`<br /> `"name":
"id",`<br /> `"required": true,`<br
/> `"type": "uuid"`<br /> `}, {`<br
/> `"id": 2,`<br /> `"name":
"data",`<br /> `"required": false,`<br
/> `"type": {`<br
/> `"type": "list",`<br
/> `...`<br />
`}`<br /> `} ]`<br />`}`|
+|**`struct`**|`JSON object: {`<br /> `"type": "struct",`<br
/> `"fields": [ {`<br /> `"id": <field id
int>,`<br /> `"name": <name string>,`<br
/> `"required": <boolean>,`<br
/> `"type": <type JSON>,`<br
/> `"doc": <comment string>,`<br
/> `"initial-default": <JSON encoding of default
value>,`<br /> `"write-default": <JSON encoding of
default value>`<br /> `}, ...`<br /> `]
}`|`{`<br /> `"type": "struct",`<br /> `"fields": [ {`<br
/> `"id": 1,`<br /> `"name":
"id",`<br /> `"required": true,`<br
/> `"type": "uuid",`<br
/> `"initial-default":
"0db3e2a8-9d1d-42b9-aa7b-74ebe558dceb",`<br
/> `"write-default": "ec5911be
-b0a7-458c-8438-c9a3e53cffae"`<br /> `}, {`<br
/> `"id": 2,`<br /> `"name":
"data",`<br /> `"required": false,`<br
/> `"type": {`<br
/> `"type": "list",`<br
/> `...`<br /> `}`<br
/> `} ]`<br />`}`|
|**`list`**|`JSON object: {`<br /> `"type": "list",`<br
/> `"element-id": <id int>,`<br /> `"element-required":
<bool>`<br /> `"element": <type JSON>`<br />`}`|`{`<br
/> `"type": "list",`<br /> `"element-id": 3,`<br
/> `"element-required": true,`<br /> `"element":
"string"`<br />`}`|
|**`map`**|`JSON object: {`<br /> `"type": "map",`<br
/> `"key-id": <key id int>,`<br /> `"key": <type
JSON>,`<br /> `"value-id": <val id int>,`<br
/> `"value-required": <bool>`<br /> `"value": <type
JSON>`<br />`}`|`{`<br /> `"type": "map",`<br
/> `"key-id": 4,`<br /> `"key": "string",`<br
/> `"value-id": 5,`<br /> `"value-required": false,`<br
/> `"value": "double"`<br />`}`|
+For default values, the serialization depends on the type of the corresponding
column or nested field. The mapping of types and their corresponding default
value JSON serialization is described in the following table:
+
+| Type | Json type | Example
| Note
|
+|--------------------|--------------------|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **`boolean`** | **`json boolean`** | `true`
|
|
+| **`int`** | **`json int`** | `1`
|
|
+| **`long`** | **`json long`** | `1`
|
|
+| **`float`** | **`json float`** | `1.1`
|
|
+| **`double`** | **`json double`** | `1.1`
|
|
+| **`decimal(P,S)`** | **`string`** | `"0x3162"`
| Stores the unscaled value, as the two's-complement big-endian binary
using the minimum number of bytes, converted to a hexadecimal string prefixed
by `0x`
|
Review comment:
I think @RussellSpitzer suggested that we store these values as natural
strings and I agree. This should be `"123.45"` rather than the unscaled value
as binary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
