[
https://issues.apache.org/jira/browse/IMPALA-5474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703872#comment-16703872
]
Yongjun Zhang commented on IMPALA-5474:
---------------------------------------
Thanks [Lars
Volker|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=lv] for
reporting the issue and [Tim
Armstrong|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=tarmstrong]
for the comments.
I finally got to spend time on this issue. Had the following observation and
understanding:
Let's call "select id from functional_parquet.bad_column_metadata t" query1 and
"select id, cnt from functional_parquet.bad_column_metadata t, (select 1 cnt)
u;" query2.
# query1 always reports ERROR, however, query2 reports WARN most of the time
but it intermittently reports ERROR. That seems to indicate some race condition
internally.
# query1 reports ERROR because it finds out that the query is in FINISHED
state instead of EXCEPTION state at [line 1037 |#L1037], thus it can reach
[line 1058 |#L1058] and report ERROR there.
# query2 always detects EXCEPTION state at [line 1037 |#L1037] which reports
a WARN.
We can see that there are three issues:
# Instead of reporting WARN at 3 above, it should report an ERROR for the
EXCEPTION state.
# Why the two queries detect different states at [line 1037 |#L1037]?
# Why query1 has intermittent behavior?
We can fix this issue with this jira here. However, Issue #2 and #3 deserves
some more investigation (they may just have the same root cause), for which I
will create a new jira with more details.
Thanks [~kwho] and [~joemcdonnell] for some good discussion.
> Adding a trivial subquery turns error into warning
> --------------------------------------------------
>
> Key: IMPALA-5474
> URL: https://issues.apache.org/jira/browse/IMPALA-5474
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.9.0
> Reporter: Lars Volker
> Assignee: Yongjun Zhang
> Priority: Major
>
> This query without a subquery results in an error, which is expected.
> {noformat}
> [localhost:21000] > set abort_on_error=1;
> ABORT_ON_ERROR set to 1
> [localhost:21000] > select id from functional_parquet.bad_column_metadata t;
> Query: select id from functional_parquet.bad_column_metadata t
> Query submitted at: 2017-06-08 15:30:44 (Coordinator: http://lv-desktop:25000)
> Query progress can be monitored at:
> http://lv-desktop:25000/query_plan?query_id=44e98719ba4c073:dec9e4d00000000
> ERROR: Column metadata states there are 11 values, but read 10 values from
> column id.
> file=hdfs://localhost:20500/test-warehouse/bad_column_metadata_parquet/bad_column_metadata.parquet
> {noformat}
> Adding a trivial subquery turns the error into a warning:
> {noformat}
> [localhost:21000] > set abort_on_error=1;
> ABORT_ON_ERROR set to 1
> [localhost:21000] > select id, cnt from
> functional_parquet.bad_column_metadata t, (select 1 cnt) u;
> Query: select id, cnt from functional_parquet.bad_column_metadata t, (select
> 1 cnt) u
> Query submitted at: 2017-06-08 15:34:57 (Coordinator: http://lv-desktop:25000)
> Query progress can be monitored at:
> http://lv-desktop:25000/query_plan?query_id=7a4eb13bf147d0f1:9156b2700000000
> WARNINGS: Column metadata states there are 11 values, but read 10 values from
> column id.
> file=hdfs://localhost:20500/test-warehouse/bad_column_metadata_parquet/bad_column_metadata.parquet
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
