[
https://issues.apache.org/jira/browse/DRILL-7306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16879106#comment-16879106
]
ASF GitHub Bot commented on DRILL-7306:
---------------------------------------
arina-ielchiieva commented on issue #1813: DRILL-7306: Disable schema-only
batch for new scan framework
URL: https://github.com/apache/drill/pull/1813#issuecomment-508696422
@paul-rogers I have re-cheked again (previous time I made runs on master and
on your branch to ensure failures are caused by your changes), result is the
same.
```
on commit c2c4f765dd039cf9073196e5078eebb942882f66 (DRILL-7306: Disable
schema-only batch for new scan framework)
two empty CSV failures
on commit 6ca5902573d06239c366f7cd788e72697366f617 (Fixed empty result set
issue) could not build the project
[ERROR]
/root/drillAutomation/builds/drill/exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/ReaderState.java:[365,8]
error: cannot find symbol
symbol: variable batchCount
location: class ReaderState
on commit 32fb3a7f8f9861d967929bfb3487d935fc683ff3 (Additional debugging)
Parquet failures
```
Link to SF1 data -
https://s3-us-west-1.amazonaws.com/drill-public/tpch/sf1/tpch_sf1_parquet.tar.gz
Tests were ran on 4 node cluster with the following options:
```
>> Query: alter system set `planner.enable_decimal_data_type` = true;
ok summary
true planner.enable_decimal_data_type updated.
>> Query: alter system set `new_view_default_permissions` = '777';
ok summary
true new_view_default_permissions updated.
>> Query: alter system set `planner.enable_limit0_optimization` = true;
ok summary
true planner.enable_limit0_optimization updated.
>> Query: alter system set `exec.errors.verbose` = true;
ok summary
true exec.errors.verbose updated.
>> Query: alter system set `planner.memory.max_query_memory_per_node` =
10737418240;
ok summary
true planner.memory.max_query_memory_per_node updated.
>> Query: alter system set `drill.exec.hashagg.fallback.enabled` = true;
ok summary
true drill.exec.hashagg.fallback.enabled updated.
>> Query: alter system set `drill.exec.hashjoin.fallback.enabled` = true;
ok summary
true drill.exec.hashjoin.fallback.enabled updated.
```
> Taking a step back, I'm actually completely mystified at how my changes
could impact Parquet (only). This PR only changed source files are for the
"new" scan, which Parquet does not use. Oddly, none of the text file queries
fail; which is the one area I did change.
Well, in the PR you do change some common classes, so I guess it somehow
influences, I don't think it purely connected with parquet just with some
filtering so something like that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Disable "fast schema" batch for new scan framework
> --------------------------------------------------
>
> Key: DRILL-7306
> URL: https://issues.apache.org/jira/browse/DRILL-7306
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
> Fix For: 1.17.0
>
>
> The EVF framework is set up to return a "fast schema" empty batch with only
> schema as its first batch because, when the code was written, it seemed
> that's how we wanted operators to work. However, DRILL-7305 notes that many
> operators cannot handle empty batches.
> Since the empty-batch bugs show that Drill does not, in fact, provide a "fast
> schema" batch, this ticket asks to disable the feature in the new scan
> framework. The feature is disabled with a config option; it can be re-enabled
> if ever it is needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
