[
https://issues.apache.org/jira/browse/IMPALA-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-3722:
----------------------------------
Priority: Minor (was: Major)
> Avro codegen can be unnecessarily disabled
> ------------------------------------------
>
> Key: IMPALA-3722
> URL: https://issues.apache.org/jira/browse/IMPALA-3722
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Skye Wanderman-Milne
> Priority: Minor
> Labels: avro, codegen, ramp-up
>
> We use avro_schema_equal() from the Avro C library to determine if a file's
> schema matches the table schema, and if they don't match we disable codegen
> for that file
> (https://github.com/cloudera/Impala/blob/cdh5-trunk/be/src/exec/hdfs-avro-scanner.cc#L153).
> However, avro_schema_equal() is unnecessarily restrictive, because it
> compares the records' names and namespaces, which don't have to be the same
> to enable codegen. There are probably other checks we don't need as well,
> e.g. default values. We should write our own schema comparison function that
> is tailored to what must match for codegen specifically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
