I think the behavior introduced in DRILL-4264 needs to be fixed: - Improve the metadata to flag a column as missing (no back-tick business) - All readers should apply the same fix - This will give an opportunity to upstream operators to provide explicit handling (e.g., Filter)
@ilooner, ideally the RecordBatchSizer should be using SchemaPath as keys to handle nested column names; I guess the Planner is injecting a project step to disambiguate such use-cases. For now, I let you guys decide; we either: a) Accept this temporary workaround and create a new JIRA to implement the right fix b) Fix the Parquet reader (remove the back-tick) with the side-effect that the schema-on-change behavior for filter based queries is broken c) Implement the right fix for DRILL-4264 as discussed above [ Full content available at: https://github.com/apache/drill/pull/1445 ] This message was relayed via gitbox.apache.org for [email protected]
