[
https://issues.apache.org/jira/browse/DRILL-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382224#comment-16382224
]
ASF GitHub Bot commented on DRILL-4120:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1138#discussion_r171606330
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/avro/AvroRecordReader.java
---
@@ -154,6 +156,12 @@ public int next() {
writer.setValueCount(recordCount);
+ // adds fields which don't exist in the table but should be present
in the schema
+ if (recordCount > 0) {
+ JsonReaderUtils.ensureAtLeastOneField(writer, getColumns(), false,
--- End diff --
In general, this is a bad idea, though existing code does this. If we find
an empty file in one scanner, but a real file in another, we create an
unnecessary schema change by making up a column.
Jinfeng's changes last year are supposed to handle the "fast none" case of
a reader with no rows. There should be no reason to add a dummy column. Old
code that adds such a column should be fixed. IMHO, code that does not add
dummy columns should not begin to do so.
> dir0 does not work when the directory structure contains Avro files
> -------------------------------------------------------------------
>
> Key: DRILL-4120
> URL: https://issues.apache.org/jira/browse/DRILL-4120
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.3.0
> Reporter: Stefán Baxter
> Assignee: Volodymyr Vysotskyi
> Priority: Major
> Fix For: 1.13.0
>
> Attachments:
> 0001-DRILL-4120-Support-reading-directories-having-avro-f.patch
>
>
> Any select statment containing dirN fails if the target directory structure
> contains Avro files.
> Steps to test:
> 1. create a simple directory structure
> 2. copy an avro file into each directory
> 3. execute a query containing dir0
> outcome:
> Error: VALIDATION ERROR: From line 1, column 117 to line 1, column 120:
> Column 'dir0' not found in any table
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
