[
https://issues.apache.org/jira/browse/IMPALA-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312870#comment-17312870
]
ASF subversion and git services commented on IMPALA-10564:
----------------------------------------------------------
Commit 4ef67c21153157d552a6d5db09f4c1e15cbe8ac0 in impala's branch
refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4ef67c2 ]
IMPALA-10607: Fixed test_ctas_exprs failure for S3 build
New test case TestDecimalOverflowExprs::test_ctas_exprs was added
in the patch for IMPALA-10564. But it failed in S3 build with
Parquet format and complained the Parquet file had an invalid file
length when accessing a table. The table was created by CTAS which
finished with error "decimal expression overflowed". Verified this
issue does not happen if query option s3_skip_insert_staging is set
as false.
When s3_skip_insert_staging is set true by default, INSERT writing
to S3 goes directly to their final location rather than being
copied there by the coordinator. If CTAS finishs with error during
INSERT, the parquet partition file is left in un-finalized without
file footer. This causes subsequent query failed with error like
"have an invalid file length on S3" when the query attemps to
access the same table.
This patch fixed the issue by deleting the un-finalized file in
its final location when AppendRows() return error and staging has
been skipped.
Testing:
- Reproduced the test failure in local box with defaultFS as s3.
Verified the fixing by running test_ctas_exprs with defaultFS
as s3.
- Passed core tests.
Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd
Reviewed-on: http://gerrit.cloudera.org:8080/17234
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> No error returned when inserting an overflowed value into a decimal column
> --------------------------------------------------------------------------
>
> Key: IMPALA-10564
> URL: https://issues.apache.org/jira/browse/IMPALA-10564
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
> Affects Versions: Impala 4.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.0
>
>
> When using CTAS statements or INSERT-SELECT statements to insert rows to
> table with decimal columns, Impala insert NULL for overflowed decimal values,
> instead of returning error. This issue happens when the data expression for
> the decimal column in SELECT sub-query consists at least one alias. This
> issue is similar as IMPALA-6340, but IMPALA-6340 only fixed the issue for the
> cases with the data expression for the decimal columns as constants so that
> the overflowed decimal values could be detected by frontend during expression
> analysis. If there is alias (variables) in the data expression for the
> decimal column, Frontend could not evaluate data expression in expression
> analysis phase. Only backend could evaluate the data expression when backend
> execute fragment instances for SELECT sub-queries. The log messages showed
> that the executor detected the decimal overflow error, but somehow it did not
> propagate the error to the coordinator, hence the error was not returned to
> the client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
