boroknagyz commented on pull request #2948:
URL: https://github.com/apache/hive/pull/2948#issuecomment-1016768717
> > HiveSchemaConverter assigned field ids starting from 0, while Iceberg
assigns field ids starting from 1. This caused test failures because the name
mapping had wrong field ids. So in the second commit I fixed
HiveSchemaConverter as well.
>
> If we provide a schema where the id is starting from 0, would that mean
that the schema is not correct? Or this is just to be more aligned with the way
Iceberg generates ids?
Seems like Iceberg doesn't respect the field ids during table creation, it
always reassigns them.
E.g. I was trying to create a table with this schema:
`iceberg.mr.table.schema={"type":"struct","schema-id":0,"fields":[{"id":0,"name":"a","required":false,"type":"int"}]}`
or
`iceberg.mr.table.schema={"type":"struct","schema-id":0,"fields":[{"id":5,"name":"a","required":false,"type":"int"}]}`
And Iceberg created the following schema:
`table {
1: a: optional int
}`
OTOH, I don't think the value 0 is invalid. The spec reserves only these ids:
https://iceberg.apache.org/#spec/#reserved-field-ids
So I edited the Iceberg snapshot file and rewrote the field ids to start
from 0, then inserted data files and queried the table. Iceberg didn't complain
about the 0 field id.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
