Alex Behm has posted comments on this change. Change subject: IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format ......................................................................
Patch Set 1: (6 comments) http://gerrit.cloudera.org:8080/#/c/3136/1/be/src/exec/hdfs-avro-scanner.cc File be/src/exec/hdfs-avro-scanner.cc: Line 63: return Status("Missing avro schema in scan node. This could be due to stale " Avro schema (capitalize 'Avro' since it is a proper noun) Line 741: return Status("Missing avro schema in scan node. This could be due to stale " Avro http://gerrit.cloudera.org:8080/#/c/3136/1/fe/src/main/java/com/cloudera/impala/catalog/HdfsTable.java File fe/src/main/java/com/cloudera/impala/catalog/HdfsTable.java: Line 1255: private boolean hasParitionOfFileFormat(HdfsFileFormat fileFormat) { instead of going over all partitions again, let's set a flag hasAvroData_ or similar when calling loadPartitionsFromMetastore() it might be even better to populate a Set<HdfsFileFormat> because then we can check for emptiness (which would indicate an error) think 100k partition tables... Line 1269: Preconditions.checkState(isSchemaLoaded_); If possible, let's remove the isSchemaLoaded_ flag and use a more direct precondition like !nonPartFieldSchemas_.isEmpty() http://gerrit.cloudera.org:8080/#/c/3136/1/testdata/workloads/functional-query/queries/QueryTest/avro-stale-schema.test File testdata/workloads/functional-query/queries/QueryTest/avro-stale-schema.test: Line 3: drop table if exists functional_avro_snap.alltypesagg_staleschema use unique_database fixture and a non-qualified test table name http://gerrit.cloudera.org:8080/#/c/3136/1/tests/query_test/test_avro_schema_resolution.py File tests/query_test/test_avro_schema_resolution.py: Line 45: def test_avro_stale_schema(self, vector): use unique_database fixture -- To view, visit http://gerrit.cloudera.org:8080/3136 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09262d3a7b85a2263c721f3beafd0cab2a1bdf4b Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.6.0_5.8.0 Gerrit-Owner: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-HasComments: Yes
