Huaisi Xu has posted comments on this change. Change subject: IMPALA-3687: Prefer Avro field name during schema reconciliation ......................................................................
Patch Set 5: (5 comments) > Also, can you plz add the test that follows the reproduction steps > from the JIRA? i.e. create table, alter column in schema file, > invalidate metadata and select. > Also, can you plz add the test that follows the reproduction steps > from the JIRA? i.e. create table, alter column in schema file, > invalidate metadata and select. that is a little "intrusive" to our code base as it needs to load a changed schema file. avro.schema.literal has the same code path as avro.schema.url. I can change the reproduction step in the Jira though... http://gerrit.cloudera.org:8080/#/c/3331/5/fe/src/main/java/com/cloudera/impala/util/AvroSchemaUtils.java File fe/src/main/java/com/cloudera/impala/util/AvroSchemaUtils.java: PS5, Line 114: > "from the column definition" good to say "avro schema". PS5, Line 112: Mismatched name/type -> Prefer Avro column, except: : * A CHAR/VARCHAR column definition maps to an Avro STRING. The reconciled column : * will preserve the CHAR/VARCHAR type but prefer the column name and comment from : * the Avro column. > Maybe it will be more clear if you have: Did in another way. PS5, Line 119: AnalysisException > You need to comment when this function throws an AnalysisException. not sure how to comment.. analyze() can throw but I think it should have a comment in the first place... Line 136: // as a CHAR/VARCHAR in the reconciled schema. > Comment that the name and comment come from the Avro schema. Done http://gerrit.cloudera.org:8080/#/c/3331/5/testdata/workloads/functional-query/queries/QueryTest/avro-schema-resolution.test File testdata/workloads/functional-query/queries/QueryTest/avro-schema-resolution.test: PS5, Line 80: INT > Can we add a type mismatch as well? e.g. make this float Done -- To view, visit http://gerrit.cloudera.org:8080/3331 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia3e43b2885853c2b4f207a45a873c9d7f31379cd Gerrit-PatchSet: 5 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Huaisi Xu <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-HasComments: Yes
