Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format ......................................................................
IMPALA-3314/IMPALA-3513: Fix querying tables/partitions altered to Avro format Bug: Impalads crash if we query an Avro table with stale metadata Cause: This happens because avroSchema_ is not set in HdfsTable, which is not propagated to the avro scanner and it doesn't have appropriate checks to make sure the schema is non-null. The patch fixes the following. 1. Avro scanner should gracefully handle the case where the avro schema is not set. Appropriate null checks and a meaning error message have been added. 2. This is a special case with multi-fileformat partitioned tables. avroSchema_ should be set in HdfsTable even if any subset of the partitions are backed by avro. Without this patch, we only set it if the base table file format is Avro. Change-Id: I09262d3a7b85a2263c721f3beafd0cab2a1bdf4b Reviewed-on: http://gerrit.cloudera.org:8080/3136 Reviewed-by: Bharath Vissapragada <[email protected]> Tested-by: Internal Jenkins --- M be/src/exec/hdfs-avro-scanner.cc M be/src/exec/hdfs-avro-scanner.h M fe/src/main/java/com/cloudera/impala/catalog/HdfsPartition.java M fe/src/main/java/com/cloudera/impala/catalog/HdfsTable.java A testdata/workloads/functional-query/queries/QueryTest/avro-stale-schema.test M tests/query_test/test_avro_schema_resolution.py 6 files changed, 107 insertions(+), 19 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/3136 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I09262d3a7b85a2263c721f3beafd0cab2a1bdf4b Gerrit-PatchSet: 6 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.6.0_5.8.0 Gerrit-Owner: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins
