Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23801 )
Change subject: IMPALA-11926: Only loading Avro schema in LocalFsTable if the format is Avro In local catalog mode LocalFsTable was always trying to load the Avro schema if the avro.schema.url table property was present, even when the table format was not Avro. This made ...................................................................... Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/23801/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/23801/5//COMMIT_MSG@7 PS5, Line 7: IMPALA-11926: Only loading Avro schema in LocalFsTable if the format is Avro Need a blank line after the title. http://gerrit.cloudera.org:8080/#/c/23801/5//COMMIT_MSG@21 PS5, Line 21: Change-Id: I202665f978401894a1c837293529c06fa4270985 To be consistent with other patches, let keep this Change-Id the last line of the commit message. http://gerrit.cloudera.org:8080/#/c/23801/5/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/23801/5/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@161 PS5, Line 161: // If the table format is not Avro, skip loading the Avro schema. : // If there are Avro partitions, the schema will be handled later when : // partitions are accessed in toTHdfsTable(). nit: please remove this comment since the above comment already explains this. http://gerrit.cloudera.org:8080/#/c/23801/5/tests/query_test/test_avro_schema_resolution.py File tests/query_test/test_avro_schema_resolution.py: http://gerrit.cloudera.org:8080/#/c/23801/5/tests/query_test/test_avro_schema_resolution.py@72 PS5, Line 72: "CREATE TABLE {0} (id INT, name STRING) STORED AS PARQUET".format(tbl)) Can we add some data using mixed parquet and avro formats, i.e. some partitions are in avro format? Like what we do for functional.alltypesmixedformat in testdata/bin/load-dependent-tables.sql. Just want to verify if Avro schemas for Avro partitions in mixed-format tables are still handled correctly. http://gerrit.cloudera.org:8080/#/c/23801/5/tests/query_test/test_avro_schema_resolution.py@73 PS5, Line 73: import impala.dbapi : hive_conn = impala.dbapi.connect( : host='localhost', port=11050, auth_mechanism='PLAIN') : hive_cursor = hive_conn.cursor() : hive_cursor.execute( : "ALTER TABLE {0} SET TBLPROPERTIES " : "('avro.schema.url'='hdfs://invalid-host/nonexistent.avsc')".format(tbl)) : hive_cursor.close() : hive_conn.close() We can simply use self.run_stmt_in_hive(). -- To view, visit http://gerrit.cloudera.org:8080/23801 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I202665f978401894a1c837293529c06fa4270985 Gerrit-Change-Number: 23801 Gerrit-PatchSet: 5 Gerrit-Owner: Raghav Jindal <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Raghav Jindal <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 06 May 2026 14:07:46 +0000 Gerrit-HasComments: Yes
