[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 13 Apr 2021 17:59:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats This change updates the exception that is thrown when the user tries to insert into a partition which has unsupported format. The information to make this decision is available during analysis, therefore this commit also moves the check from the planner to the analyzer to have an earlier result. In the analyzer only the FeFsTables have to be checked therefore Kudu tables are not related. Also, there is a difference between static and dynamic partition clauses, for static partition clauses the partition format is available during compile, for dynaminc partition clauses it is only avaialble during runtime. Testing: - Added unit tests - Ran exhaustive tests successfully Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Reviewed-on: http://gerrit.cloudera.org:8080/17300 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql 4 files changed, 78 insertions(+), 21 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 4 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 13 Apr 2021 12:12:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7065/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 13 Apr 2021 12:12:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 2: Code-Review+2 Thanks for the changes! -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 13 Apr 2021 12:11:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 2: (3 comments) Hi Zoltan, thank you for the review. the change has been updated. http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@13 PS1, Line 13: earlier > earlier Done http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@15 PS1, Line 15: Kudu > Actually Iceberg tables are related because they might use Parquet or ORC u Done http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@16 PS1, Line 16: related > related Done -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 13 Apr 2021 07:50:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17300 to look at the new patch set (#2). Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats This change updates the exception that is thrown when the user tries to insert into a partition which has unsupported format. The information to make this decision is available during analysis, therefore this commit also moves the check from the planner to the analyzer to have an earlier result. In the analyzer only the FeFsTables have to be checked therefore Kudu tables are not related. Also, there is a difference between static and dynamic partition clauses, for static partition clauses the partition format is available during compile, for dynaminc partition clauses it is only avaialble during runtime. Testing: - Added unit tests - Ran exhaustive tests successfully Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec --- M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql 4 files changed, 78 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/17300/2 -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 1: Code-Review+2 (3 comments) Nice improvement! http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@13 PS1, Line 13: eaerlier earlier http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@15 PS1, Line 15: Iceberg Actually Iceberg tables are related because they might use Parquet or ORC underneath. FeIcebergTable inherits from FeFsTable and it overrides 'getFileFormats()' so your patch works well on iceberg tables. http://gerrit.cloudera.org:8080/#/c/17300/1//COMMIT_MSG@16 PS1, Line 16: realted related -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 12 Apr 2021 16:44:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17300 ) Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/8541/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Sat, 10 Apr 2021 09:52:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17300 Change subject: IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats .. IMPALA-9732: Improve exceptions of unsupported HdfsTableSink formats This change updates the exception that is thrown when the user tries to insert into a partition which has unsupported format. The information to make this decision is available during analysis, therefore this commit also moves the check from the planner to the analyzer to have an eaerlier result. In the analyzer only the FeFsTables have to be checked, Kudu and Iceberg tables are not realted. Also, there is a difference between static and dynamic partition clauses, for static partition clauses the partition format is available during compile, for dynaminc partition clauses it is only avaialble during runtime. Testing: - Added unit tests - Ran exhaustive tests successfully Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec --- M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql 4 files changed, 78 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/17300/1 -- To view, visit http://gerrit.cloudera.org:8080/17300 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7fa2f949336a422acb4d01c9347b9b2e808e4aec Gerrit-Change-Number: 17300 Gerrit-PatchSet: 1 Gerrit-Owner: Tamas Mate
