[
https://issues.apache.org/jira/browse/DRILL-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015451#comment-17015451
]
ASF GitHub Bot commented on DRILL-7509:
---------------------------------------
paul-rogers commented on pull request #1954: DRILL-7509: Incorrect TupleSchema
is created for DICT column when querying Parquet files
URL: https://github.com/apache/drill/pull/1954#discussion_r366606167
##########
File path:
metastore/metastore-api/src/main/java/org/apache/drill/metastore/util/SchemaPathUtils.java
##########
@@ -81,15 +106,50 @@ public static void addColumnMetadata(TupleMetadata
schema, SchemaPath schemaPath
names.add(colPath.getPath());
colMetadata = schema.metadata(colPath.getPath());
TypeProtos.MajorType pathType =
types.get(SchemaPath.getCompoundPath(names.toArray(new String[0])));
+
+ boolean isDict = pathType != null && pathType.getMinorType() ==
TypeProtos.MinorType.DICT;
+ boolean isList = pathType != null && pathType.getMinorType() ==
TypeProtos.MinorType.LIST;
+ String name = colPath.getPath();
Review comment:
More bad code smell. See notes above.
BTW: If you have to do a short-term fix, then this kind of error-prone,
redundant code is fine. But, if we have time to fix the underlying problem,
let's do it. Else, we can fix the problem in another PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Incorrect TupleSchema is created for DICT column when querying Parquet files
> ----------------------------------------------------------------------------
>
> Key: DRILL-7509
> URL: https://issues.apache.org/jira/browse/DRILL-7509
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Bohdan Kazydub
> Assignee: Bohdan Kazydub
> Priority: Major
> Fix For: 1.18.0
>
>
> When {{DICT}} column is queried from Parquet file, its {{TupleSchema}}
> contains nested element, e.g. `map`, itself contains `key` and `value`
> fields, rather than containing the `key` and `value` fields in the {{DICT}}'s
> {{TupleSchema}} itself. The nested element, `map`, comes from the inner
> structure of Parquet's {{MAP}} (which corresponds to Drill's {{DICT}})
> representation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
