rdblue commented on a change in pull request #830: Support name mapping
resolution for parquet
URL: https://github.com/apache/incubator-iceberg/pull/830#discussion_r405634868
##########
File path:
parquet/src/main/java/org/apache/iceberg/parquet/ParquetSchemaUtil.java
##########
@@ -87,29 +82,21 @@ public static boolean hasIds(MessageType fileSchema) {
// Try to convert the type to Iceberg. If an ID assignment is needed,
return false.
ParquetTypeVisitor.visit(fileSchema, new MessageTypeToType(fileSchema) {
@Override
- protected int nextId() {
- throw new IllegalStateException("Needed to assign ID");
+ protected int getId(org.apache.parquet.schema.Type type) {
+ org.apache.parquet.schema.Type.ID id = type.getId();
+ if (id != null) {
+ throw new IllegalStateException("at least one ID exists");
Review comment:
Yes, we expect schemas to either have all IDs or none, but we don't need to
fail if that assumption is violated. If we detect any ID in the schema, then we
should trust the file's IDs. If there are no IDs, then we should infer them
using a name mapping or position-based fallback, whatever was configured.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
