[Impala-ASF-CR] WIP IMPALA-10564: Return error when inserting an overflowed decimal value

2021-03-10 Thread Wenzhe Zhou (Code Review) 
Wenzhe Zhou has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/17168 )

Change subject: WIP IMPALA-10564: Return error when inserting an overflowed 
decimal value
..


Patch Set 1:

(2 comments)

Fixed the issue for HDFS and Kudu table. HBase don't have this issue since it 
does not insert NULL to the table.

http://gerrit.cloudera.org:8080/#/c/17168/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/17168/1//COMMIT_MSG@7
PS1, Line 7: overflowed
> See related comment below.. it can occur for other error conditions besides
Done


http://gerrit.cloudera.org:8080/#/c/17168/1/be/src/exec/hdfs-text-table-writer.cc
File be/src/exec/hdfs-text-table-writer.cc:

http://gerrit.cloudera.org:8080/#/c/17168/1/be/src/exec/hdfs-text-table-writer.cc@107
PS1, Line 107: overflowed
> It may not always be overflow.  For instance I get the same NULL value inse
Done



--
To view, visit http://gerrit.cloudera.org:8080/17168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64ce4ed194af81ef06401ffc1124e12f05b8da98
Gerrit-Change-Number: 17168
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Wenzhe Zhou 
Gerrit-Comment-Date: Wed, 10 Mar 2021 21:48:09 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10564: Return error when inserting an overflowed decimal value

2021-03-09 Thread Aman Sinha (Code Review) 
Aman Sinha has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/17168 )

Change subject: WIP IMPALA-10564: Return error when inserting an overflowed 
decimal value
..


Patch Set 1: Code-Review+1

(2 comments)

Code change LGTM. Have a couple of comments. I can upgrade to +2 once you add a 
unit test and address comments.
One other question is about non-HDFS tables. Is insert into Kudu tables covered 
through these ? You don't have to test it for this fix but something to check 
in the future.

http://gerrit.cloudera.org:8080/#/c/17168/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/17168/1//COMMIT_MSG@7
PS1, Line 7: overflowed
See related comment below.. it can occur for other error conditions besides 
overflow so you can change the message to be something more generic..like 
'inserting an invalid decimal value'


http://gerrit.cloudera.org:8080/#/c/17168/1/be/src/exec/hdfs-text-table-writer.cc
File be/src/exec/hdfs-text-table-writer.cc:

http://gerrit.cloudera.org:8080/#/c/17168/1/be/src/exec/hdfs-text-table-writer.cc@107
PS1, Line 107: overflowed
It may not always be overflow.  For instance I get the same NULL value inserted 
in the table for a divide-by-zero error:

insert into t1 select cast((a+b+c)/0 as decimal (28,10)) as x from (select 
cast(654964569154.9565 as decimal (28,7)) as a, cast(44658554984 as decimal 
(28,7)) as b, cast(2.111 as decimal (28,7)) as c) q;

I see these are the error conditions for decimal :

grep SetError exprs/decimal-operators-ir.cc

  ctx->SetError("Decimal expression overflowed"); \
  ctx->SetError("String to Decimal parse failed");
  ctx->SetError("String to Decimal cast overflowed");
  if (decimal_v2) ctx->SetError("Cannot divide decimal by zero"); \



--
To view, visit http://gerrit.cloudera.org:8080/17168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64ce4ed194af81ef06401ffc1124e12f05b8da98
Gerrit-Change-Number: 17168
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Wed, 10 Mar 2021 03:16:45 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] WIP IMPALA-10564: Return error when inserting an overflowed decimal value

2021-03-09 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/17168 )

Change subject: WIP IMPALA-10564: Return error when inserting an overflowed 
decimal value
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/8326/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/17168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64ce4ed194af81ef06401ffc1124e12f05b8da98
Gerrit-Change-Number: 17168
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Wed, 10 Mar 2021 02:13:05 +
Gerrit-HasComments: No

[Impala-ASF-CR] WIP IMPALA-10564: Return error when inserting an overflowed decimal value

2021-03-09 Thread Wenzhe Zhou (Code Review) 
Wenzhe Zhou has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/17168


Change subject: WIP IMPALA-10564: Return error when inserting an overflowed 
decimal value
..

WIP IMPALA-10564: Return error when inserting an overflowed decimal value

When using CTAS statements or INSERT-SELECT statements to insert rows to
table with decimal columns, Impala insert NULL for overflowed decimal
values, instead of returning error. This issue happens when the data
expression for the decimal column in SELECT sub-query consists at least
one alias.
This issue is similar as IMPALA-6340, but IMPALA-6340 only fixed the
issue for the cases with the data expression for the decimal columns as
constants so that the overflowed decimal values could be detected by
frontend during expression analysis. If there is an alias (variable) in
the data expression for the decimal column, only backend could detect
decimal overflow.

This patch added checking for the error status of ScalarExprEvaluator
in TableWriter when ScalarExprEvaluator return NULL for decimal column.
If there is an error, the query will be failed without inserting NULL
for decimal column.

Tests:
 - Manually ran queries with overflowed decimal values by using CTAS
   and INSERT-SELECT statements. Verified that queries failed as
   expected.
 - TODO: add unit-test.

Change-Id: I64ce4ed194af81ef06401ffc1124e12f05b8da98
---
M be/src/exec/hdfs-text-table-writer.cc
M be/src/exec/parquet/hdfs-parquet-table-writer.cc
2 files changed, 20 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/17168/1
--
To view, visit http://gerrit.cloudera.org:8080/17168
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I64ce4ed194af81ef06401ffc1124e12f05b8da98
Gerrit-Change-Number: 17168
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou 
Gerrit-Reviewer: Aman Sinha 
Gerrit-Reviewer: Thomas Tauber-Marshall