[Impala-ASF-CR] IMPALA-1422: support a constant on LHS of IN predicates.
Hello Dimitris Tsirogiannis, Alex Behm, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8322 to look at the new patch set (#10). Change subject: IMPALA-1422: support a constant on LHS of IN predicates. .. IMPALA-1422: support a constant on LHS of IN predicates. Currently, constant expressions for the LHS of the IN predicate are not supported. This patch adds this support as a rewrite in StmtRewriter (where subqueries are rewritten to joins). Since there is a nested-loop variant of left semijoin, support for IN is handled by not erring out. NOT IN is handled by a rewrite to corresponding NOT EXISTS predicate. Support for NOT IN with a correlated subquery is not included in this change. Re-organized the frontend subquery analysis tests to expand coverage. Testing: - added frontend subquery analysis tests - added e2e tests Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb --- M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test A testdata/workloads/functional-query/queries/QueryTest/subquery-in-constant-lhs.test M tests/query_test/test_queries.py 8 files changed, 921 insertions(+), 203 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/8322/10 -- To view, visit http://gerrit.cloudera.org:8080/8322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb Gerrit-Change-Number: 8322 Gerrit-PatchSet: 10 Gerrit-Owner: Vuk ErcegovacGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-1422: support a constant on LHS of IN predicates.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/8322 ) Change subject: IMPALA-1422: support a constant on LHS of IN predicates. .. Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java: http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@368 PS9, Line 368: String wrapperAlias = rhsQuery.getTableAliasGenerator().getNextAlias(); > Please address this alias generation issue as discussed. done. I create generators externally now since I can't create the outer-block before I've created this view. Then I pass them to the outer-block after its created. http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@401 PS9, Line 401: > remove, no point in having a blank line here Done -- To view, visit http://gerrit.cloudera.org:8080/8322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb Gerrit-Change-Number: 8322 Gerrit-PatchSet: 9 Gerrit-Owner: Vuk ErcegovacGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 01 Dec 2017 06:50:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. IMPALA-6255: Add device names to DiskIoMgr thread names This change adds device names to the DiskIoMgr thread names. It will make them easier to identify during debugging. Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Reviewed-on: http://gerrit.cloudera.org:8080/8669 Reviewed-by: Lars VolkerTested-by: Impala Public Jenkins --- M be/src/runtime/io/disk-io-mgr.cc M tests/webserver/test_web_pages.py 2 files changed, 25 insertions(+), 1 deletion(-) Approvals: Lars Volker: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 7 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 6 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 01 Dec 2017 05:51:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3060: draft: Supports IS [NOT] NULL feature for complex type
Kim Jin Chul has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8710 Change subject: IMPALA-3060: draft: Supports IS [NOT] NULL feature for complex type .. IMPALA-3060: draft: Supports IS [NOT] NULL feature for complex type Change-Id: I986e1bae82d5ca7a82fc1bab2e412c4733928dce --- M be/src/exec/hdfs-scanner.cc M be/src/runtime/descriptors.cc M be/src/runtime/types.h M fe/src/main/java/org/apache/impala/analysis/IsNullPredicate.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/catalog/StructType.java 6 files changed, 22 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/8710/1 -- To view, visit http://gerrit.cloudera.org:8080/8710 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I986e1bae82d5ca7a82fc1bab2e412c4733928dce Gerrit-Change-Number: 8710 Gerrit-PatchSet: 1 Gerrit-Owner: Kim Jin Chul
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1553/ -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Fri, 01 Dec 2017 05:23:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 6: Code-Review+2 Changed thread names for out-of-range disks in tests back to the old behavior. -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 6 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 01 Dec 2017 02:14:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2248: Make idle session timeout a query option
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8490 ) Change subject: IMPALA-2248: Make idle_session_timeout a query option .. Patch Set 10: (5 comments) Did an initial pass. Still need to look at the tests in detail. http://gerrit.cloudera.org:8080/#/c/8490/10//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8490/10//COMMIT_MSG@7 PS10, Line 7: IMPALA-2248: Make idle_session_timeout a query option I did an initial pass over the code and wanted to clarify my understanding of the current behaviour: * Does "set idle_session_timeout" in impala-shell have any effect? I wouldn't think so since it just updates a client-side map, right? * The request pool query options overlay doesn't have an effect on this option, does it? Since a session doesn't belong to a request pool. I think unfortunately this will end up being a bit of a special case in terms of behaviour, but I think this is expected. http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/client-request-state.h File be/src/service/client-request-state.h: http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/client-request-state.h@423 PS10, Line 423: void UpdateSessionTimeout(int32_t requested_timeout); I think this assumes that session_->lock_ is held. We should be careful to document assumptions about which locks are and aren't held in this part of the code (there are unfortunately a lot of locks). http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/client-request-state.cc File be/src/service/client-request-state.cc: http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/client-request-state.cc@207 PS10, Line 207: UpdateSessionTimeout(atoi(value.c_str())); It feels a bit weird that we're doing string parsing of a query option here. Maybe we should set the query option first and then get the parsed value? http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/impala-server.h File be/src/service/impala-server.h: http://gerrit.cloudera.org:8080/#/c/8490/10/be/src/service/impala-server.h@436 PS10, Line 436: void SetTimeout(int32_t requested_timeout); The caller must hold 'lock', right? http://gerrit.cloudera.org:8080/#/c/8490/7/be/src/service/impala-server.h File be/src/service/impala-server.h: http://gerrit.cloudera.org:8080/#/c/8490/7/be/src/service/impala-server.h@423 PS7, Line 423: /// session may be correctly expired after a timeout (when ref_count == 0). Typically Need to think about whether this is necessary - should we just use set_query_options? -- To view, visit http://gerrit.cloudera.org:8080/8490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32e2775f80da387b0df4195fe2c5435b3f8e585e Gerrit-Change-Number: 8490 Gerrit-PatchSet: 10 Gerrit-Owner: Zoltan Borok-NagyGerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Fri, 01 Dec 2017 02:13:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1554/ -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 5 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 01 Dec 2017 02:07:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Hello Tim Armstrong, Alex Behm, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8669 to look at the new patch set (#5). Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. IMPALA-6255: Add device names to DiskIoMgr thread names This change adds device names to the DiskIoMgr thread names. It will make them easier to identify during debugging. Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b --- M be/src/runtime/io/disk-io-mgr.cc M tests/webserver/test_web_pages.py 2 files changed, 23 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/8669/5 -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 5 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1553/ -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Fri, 01 Dec 2017 01:50:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Kim Jin Chul has posted comments on this change. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. Patch Set 7: @Jim and Philip, I appreciate your review. -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 7 Gerrit-Owner: Kim Jin ChulGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: John Russell Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Fri, 01 Dec 2017 00:28:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5940: Avoid stack tracing and log spew with Status::Expected()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8689 ) Change subject: IMPALA-5940: Avoid stack tracing and log spew with Status::Expected() .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/8689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie1f7d16e60f7859d662e87642d0f82e1d74183ad Gerrit-Change-Number: 8689 Gerrit-PatchSet: 1 Gerrit-Owner: Michael HoGerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Thu, 30 Nov 2017 23:34:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5940: Avoid stack tracing and log spew with Status::Expected()
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8689 ) Change subject: IMPALA-5940: Avoid stack tracing and log spew with Status::Expected() .. IMPALA-5940: Avoid stack tracing and log spew with Status::Expected() This change converts some callers of Status() to Status::Expected() in the DataStreamMgr to avoid log spew and unnecessary overhead of stack tracing. Change-Id: Ie1f7d16e60f7859d662e87642d0f82e1d74183ad Reviewed-on: http://gerrit.cloudera.org:8080/8689 Reviewed-by: Bharath VissapragadaReviewed-by: Dan Hecht Tested-by: Impala Public Jenkins --- M be/src/runtime/data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.cc 2 files changed, 19 insertions(+), 11 deletions(-) Approvals: Bharath Vissapragada: Looks good to me, but someone else must approve Dan Hecht: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie1f7d16e60f7859d662e87642d0f82e1d74183ad Gerrit-Change-Number: 8689 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG@43 PS4, Line 43: > Yes, these numbers (and all other benchmarks that I've done) are from a rel Thanks. -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Thu, 30 Nov 2017 23:16:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Taras Bobrovytsky has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG@43 PS4, Line 43: > are these numbers all from release build, no debug build? Yes, these numbers (and all other benchmarks that I've done) are from a release build. -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Thu, 30 Nov 2017 23:16:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 4: Code-Review+1 (1 comment) Tim, please do the +2 with your review. http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8404/4//COMMIT_MSG@43 PS4, Line 43: are these numbers all from release build, no debug build? -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Thu, 30 Nov 2017 23:14:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Taras Bobrovytsky has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. IMPALA-5017: Error on decimal overflow Before this patch, decimal operations would either silently overflow (in the case of sum() and avg()), or produce a warning. In this patch, the behaviour is changed so that an error is produced in the case of overflow when DECIMAL_v2 is enabled. Decimal v1 behaviour is unchanged. We introduce overflow checks when computing sum() and avg(). This results in a ~30% performance regression when we are in decimal v2 mode compared to decimal v1. Benchmarks: Query: select sum(dec_38_19) from decimal_tbl Decimal v1: 11.57s Decimal v2: 16.58s Query: select avg(dec_38_19) from decimal_tbl Decimal v1: 12.08s Decimal v2: 17.08s The performance regression is not as bad if we are computing the sum or average of decimal column with a lower precision: Query: select sum(dec_9_5) from decimal_tbl Decimal v1: 11.06s Decimal v2: 13.08s Query: select avg(dec_9_5) from decimal_tbl Decimal v1: 11.56s Decimal v2: 13.57s Testing: - Added several end to end tests. - Updated Expr tests to check for error in case of overflow. Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 --- M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/decimal-operators-ir.cc M be/src/exprs/expr-codegen-test.cc M be/src/exprs/expr-test.cc M testdata/workloads/functional-query/queries/QueryTest/decimal-exprs.test M testdata/workloads/functional-query/queries/QueryTest/decimal.test 6 files changed, 230 insertions(+), 77 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/8404/4 -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 4 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Taras Bobrovytsky has posted comments on this change. ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/8404/3/be/src/exprs/aggregate-functions-ir.cc File be/src/exprs/aggregate-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/8404/3/be/src/exprs/aggregate-functions-ir.cc@415 PS3, Line 415: ctx->SetError("Avg computation overflowed"); > did it help performance? Yes it did. I updated the benchmark results in the commit message. -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 3 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden Gerrit-Comment-Date: Thu, 30 Nov 2017 23:09:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5017: Error on decimal overflow
Taras Bobrovytsky has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/8404 ) Change subject: IMPALA-5017: Error on decimal overflow .. IMPALA-5017: Error on decimal overflow Before this patch, decimal operations would either silently overflow (in the case of sum() and avg()), or produce a warning. In this patch, the behaviour is changed so that an error is produced in the case of overflow when DECIMAL_v2 is enabled. Decimal v1 behaviour is unchanged. We introduce overflow checks when computing sum() and avg(). This results in a ~30% performance regression when we are in decimal v2 mode compared to decimal v1. Benchmarks: Query: select sum(dec_38_19) from decimal_tbl Decimal v1: 13.09s Decimal v2: 17.10s Query: select avg(dec_38_19) from decimal_tbl Decimal v1: 13.60s Decimal v2: 19.10s Query: select avg(dec_9_5) from decimal_tbl Decimal v1: 12.06s Decimal v2: 18.07s Testing: - Added several end to end tests. - Updated Expr tests to check for error in case of overflow. Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 --- M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/decimal-operators-ir.cc M be/src/exprs/expr-codegen-test.cc M be/src/exprs/expr-test.cc M testdata/workloads/functional-query/queries/QueryTest/decimal-exprs.test M testdata/workloads/functional-query/queries/QueryTest/decimal.test 6 files changed, 230 insertions(+), 77 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/8404/3 -- To view, visit http://gerrit.cloudera.org:8080/8404 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id98a92c9a9469ec8cf14e518c741a2dab7053019 Gerrit-Change-Number: 8404 Gerrit-PatchSet: 3 Gerrit-Owner: Taras BobrovytskyGerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: John Sherman Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Taras Bobrovytsky Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zach Amsden
[Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8690 ) Change subject: IMPALA-6068: Scale back fixing functional-types .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1552/ -- To view, visit http://gerrit.cloudera.org:8080/8690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Gerrit-Change-Number: 8690 Gerrit-PatchSet: 4 Gerrit-Owner: Zach AmsdenGerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 22:17:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Hello Philip Zeyliger, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8690 to look at the new patch set (#4). Change subject: IMPALA-6068: Scale back fixing functional-types .. IMPALA-6068: Scale back fixing functional-types I re-created the original patch for IMPALA-6068, but only performed what I believe to be the limited legal transformation of data load: DEPENDENT_LOAD -> DEPENDENT_LOAD_HIVE. Any place that directly uploads via hadoop or hdfs commands was left alone as changing it can't be proven to be correct. Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Testing: Doing a full data load. I mistakenly changed a variable name causing the first two dry-runs to fail. --- M testdata/bin/generate-schema-statements.py M testdata/common/widetable.py M testdata/datasets/functional/functional_schema_template.sql 3 files changed, 61 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/8690/4 -- To view, visit http://gerrit.cloudera.org:8080/8690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Gerrit-Change-Number: 8690 Gerrit-PatchSet: 4 Gerrit-Owner: Zach AmsdenGerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-1422: support a constant on LHS of IN predicates.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/8322 ) Change subject: IMPALA-1422: support a constant on LHS of IN predicates. .. Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java: http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@368 PS9, Line 368: String wrapperAlias = rhsQuery.getTableAliasGenerator().getNextAlias(); Please address this alias generation issue as discussed. http://gerrit.cloudera.org:8080/#/c/8322/9/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@401 PS9, Line 401: remove, no point in having a blank line here -- To view, visit http://gerrit.cloudera.org:8080/8322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb Gerrit-Change-Number: 8322 Gerrit-PatchSet: 9 Gerrit-Owner: Vuk ErcegovacGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 30 Nov 2017 22:04:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1551/ -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 4 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 30 Nov 2017 21:57:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1550/ -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 4 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 30 Nov 2017 21:56:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 4: Code-Review+2 (1 comment) Carrying Tim's +2 http://gerrit.cloudera.org:8080/#/c/8669/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8669/3//COMMIT_MSG@12 PS3, Line 12: Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b > Needs updating? Thanks for catching this, done. -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 4 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 30 Nov 2017 21:56:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Hello Tim Armstrong, Alex Behm, Bikramjeet Vig, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8669 to look at the new patch set (#4). Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. IMPALA-6255: Add device names to DiskIoMgr thread names This change adds device names to the DiskIoMgr thread names. It will make them easier to identify during debugging. Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b --- M be/src/runtime/disk-io-mgr.cc M tests/webserver/test_web_pages.py 2 files changed, 23 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/8669/4 -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 4 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8690 ) Change subject: IMPALA-6068: Scale back fixing functional-types .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1548/ -- To view, visit http://gerrit.cloudera.org:8080/8690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Gerrit-Change-Number: 8690 Gerrit-PatchSet: 3 Gerrit-Owner: Zach AmsdenGerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 21:26:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4978 / IMPALA-5631: [DOCS] Add FQDN known issue
John Russell has posted comments on this change. ( http://gerrit.cloudera.org:8080/7388 ) Change subject: IMPALA-4978 / IMPALA-5631: [DOCS] Add FQDN known issue .. Patch Set 4: OK, I loosened the wording to avoid promising a comprehensive fix. I see IMPALA-4978 is still open, so I presume nobody currently has a system with fixes for both IMPALA-4978 and IMPALA-5253. -- To view, visit http://gerrit.cloudera.org:8080/7388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib039d0102878f1c05470371f581cb258287b9bc0 Gerrit-Change-Number: 7388 Gerrit-PatchSet: 4 Gerrit-Owner: John RussellGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Henry Robinson Gerrit-Reviewer: John Russell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 30 Nov 2017 20:56:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5940: Avoid stack tracing and log spew with Status::Expected()
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/8689 ) Change subject: IMPALA-5940: Avoid stack tracing and log spew with Status::Expected() .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/8689/1/be/src/runtime/krpc-data-stream-mgr.cc File be/src/runtime/krpc-data-stream-mgr.cc: http://gerrit.cloudera.org:8080/#/c/8689/1/be/src/runtime/krpc-data-stream-mgr.cc@201 PS1, Line 201: ErrorMsg msg(TErrorCode::DATASTREAM_RECVR_CLOSED, PrintId(finst_id), dest_node_id); > why doesn't the same argument hold in the non-krpc case. (I don't feel too We didn't feed this error code back to the sender in the non-krpc case yet. Don't think we do anything for IMPALA-3990 in the non-krpc case. -- To view, visit http://gerrit.cloudera.org:8080/8689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie1f7d16e60f7859d662e87642d0f82e1d74183ad Gerrit-Change-Number: 8689 Gerrit-PatchSet: 1 Gerrit-Owner: Michael HoGerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Thu, 30 Nov 2017 19:57:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5940: Avoid stack tracing and log spew with Status::Expected()
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/8689 ) Change subject: IMPALA-5940: Avoid stack tracing and log spew with Status::Expected() .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/8689/1/be/src/runtime/krpc-data-stream-mgr.cc File be/src/runtime/krpc-data-stream-mgr.cc: http://gerrit.cloudera.org:8080/#/c/8689/1/be/src/runtime/krpc-data-stream-mgr.cc@201 PS1, Line 201: ErrorMsg msg(TErrorCode::DATASTREAM_RECVR_CLOSED, PrintId(finst_id), dest_node_id); > Not really as this doesn't provide much information. We need to propagate a why doesn't the same argument hold in the non-krpc case. (I don't feel too strongly so okay to leave it alone but curious). -- To view, visit http://gerrit.cloudera.org:8080/8689 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie1f7d16e60f7859d662e87642d0f82e1d74183ad Gerrit-Change-Number: 8689 Gerrit-PatchSet: 1 Gerrit-Owner: Michael HoGerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Thu, 30 Nov 2017 18:42:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. IMPALA-4506: Do not display some intro message if --quiet is set Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Reviewed-on: http://gerrit.cloudera.org:8080/8613 Reviewed-by: Jim AppleTested-by: Impala Public Jenkins --- M shell/impala_shell.py 1 file changed, 28 insertions(+), 14 deletions(-) Approvals: Jim Apple: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 7 Gerrit-Owner: Kim Jin Chul Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 6 Gerrit-Owner: Kim Jin ChulGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 18:30:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8690 ) Change subject: IMPALA-6068: Scale back fixing functional-types .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1548/ -- To view, visit http://gerrit.cloudera.org:8080/8690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Gerrit-Change-Number: 8690 Gerrit-PatchSet: 3 Gerrit-Owner: Zach AmsdenGerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 17:55:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6068: Scale back fixing functional-types
Hello Philip Zeyliger, David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8690 to look at the new patch set (#3). Change subject: IMPALA-6068: Scale back fixing functional-types .. IMPALA-6068: Scale back fixing functional-types I re-created the original patch for IMPALA-6068, but only performed what I believe to be the limited legal transformation of data load: DEPENDENT_LOAD -> DEPENDENT_LOAD_HIVE. Any place that directly uploads via hadoop or hdfs commands was left alone as changing it can't be proven to be correct. Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Testing: Doing a full data load. I mistakenly changed a variable name causing the first two dry-runs to fail. --- M testdata/bin/generate-schema-statements.py M testdata/common/widetable.py M testdata/datasets/functional/functional_schema_template.sql 3 files changed, 61 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/8690/3 -- To view, visit http://gerrit.cloudera.org:8080/8690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6c242cca209a7138b10ad517076707709b5cd204 Gerrit-Change-Number: 8690 Gerrit-PatchSet: 3 Gerrit-Owner: Zach AmsdenGerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-5310: Part 2: Add SAMPLED NDV() function.
Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/8569 ) Change subject: IMPALA-5310: Part 2: Add SAMPLED_NDV() function. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/8569/3/be/src/thirdparty/mpfit/DISCLAIMER File be/src/thirdparty/mpfit/DISCLAIMER: PS3: > I'm not sure if it counts as a licence under ASF rules. Maybe it'll get the Ok let's wait for the JIRA. The text in here sure makes it sound like the authors intended for anyone to do anything with this code. -- To view, visit http://gerrit.cloudera.org:8080/8569 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia51d56ee67ec6073e92f90bebb4005484138b820 Gerrit-Change-Number: 8569 Gerrit-PatchSet: 3 Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Thu, 30 Nov 2017 16:51:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Remove "incubator-" from URLs.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8698 ) Change subject: Remove "incubator-" from URLs. .. Patch Set 1: Heh :) -- To view, visit http://gerrit.cloudera.org:8080/8698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6a8e10418f75c89a1f0b2111f860417a80dc10c5 Gerrit-Change-Number: 8698 Gerrit-PatchSet: 1 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Jim Apple Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 30 Nov 2017 16:32:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1547/ -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 6 Gerrit-Owner: Kim Jin ChulGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 14:57:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 6 Gerrit-Owner: Kim Jin ChulGerrit-Reviewer: Jim Apple Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 14:57:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4506: Do not display some intro message if --quiet is set
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/8613 ) Change subject: IMPALA-4506: Do not display some intro message if --quiet is set .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/8613 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19c6d00dfbbe805ee9c525b72eb5703840e2f582 Gerrit-Change-Number: 8613 Gerrit-PatchSet: 5 Gerrit-Owner: Kim Jin ChulGerrit-Reviewer: Jim Apple Gerrit-Reviewer: Kim Jin Chul Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 30 Nov 2017 14:57:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5310: Part 2: Add SAMPLED NDV() function.
Jim Apple has posted comments on this change. ( http://gerrit.cloudera.org:8080/8569 ) Change subject: IMPALA-5310: Part 2: Add SAMPLED_NDV() function. .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/8569/3/be/src/thirdparty/mpfit/DISCLAIMER File be/src/thirdparty/mpfit/DISCLAIMER: PS3: > Thanks for bringing this up. Can you be more specific what you are concerne I'm not sure if it counts as a licence under ASF rules. Maybe it'll get the same treatment from the legal team as WTFPL, which is definitely allowed: http://www.wtfpl.net/txt/copying/ -- To view, visit http://gerrit.cloudera.org:8080/8569 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia51d56ee67ec6073e92f90bebb4005484138b820 Gerrit-Change-Number: 8569 Gerrit-PatchSet: 3 Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Jim Apple Gerrit-Comment-Date: Thu, 30 Nov 2017 14:53:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-1422: support a constant on LHS of IN predicates.
Hello Dimitris Tsirogiannis, Alex Behm, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8322 to look at the new patch set (#9). Change subject: IMPALA-1422: support a constant on LHS of IN predicates. .. IMPALA-1422: support a constant on LHS of IN predicates. Currently, constant expressions for the LHS of the IN predicate are not supported. This patch adds this support as a rewrite in StmtRewriter (where subqueries are rewritten to joins). Since there is a nested-loop variant of left semijoin, support for IN is handled by not erring out. NOT IN is handled by a rewrite to corresponding NOT EXISTS predicate. Support for NOT IN with a correlated subquery is not included in this change. Re-organized the frontend subquery analysis tests to expand coverage. Testing: - added frontend subquery analysis tests - added e2e tests Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb --- M fe/src/main/java/org/apache/impala/analysis/InPredicate.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test A testdata/workloads/functional-query/queries/QueryTest/subquery-in-constant-lhs.test M tests/query_test/test_queries.py 7 files changed, 903 insertions(+), 203 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/8322/9 -- To view, visit http://gerrit.cloudera.org:8080/8322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0d69889a3c72e90be9d4ccf47d2816819ae32acb Gerrit-Change-Number: 8322 Gerrit-PatchSet: 9 Gerrit-Owner: Vuk ErcegovacGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-1422: support a constant on LHS of IN predicates.
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/8322 ) Change subject: IMPALA-1422: support a constant on LHS of IN predicates. .. Patch Set 8: (24 comments) http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java File fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java: http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@326 PS8, Line 326:*condition using the LHS constant. The rewrite handles group by, > Not clear what "handles" here means. It doesn't really do anything special thanks for the suggestion.. re-worded. http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@327 PS8, Line 327:*analytic functions, limit, and uncorrelated subqueries. Correlated subqueries > Move last sentence into a TODO like this: Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@330 PS8, Line 330:*TODO: handle correlated NOT IN case > remove (see above comment) Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@340 PS8, Line 340:*So, if C is equal to any x_i, the expression is false. Similarly, if any > Nice description. I'm wondering if we directly translate this insight into yup, simplified. http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@355 PS8, Line 355: Preconditions.checkArgument(rhs.contains(Subquery.class)); > not needed because L356-357 performs the same check Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@398 PS8, Line 398: newSubquery.analyze(rhsQuery.getAnalyzer()); > This seems wrong. It might happen to work but pretty sure this will eventua Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java@714 PS8, Line 714: if (isCorrelatedPredicate(subqueryStmt.getWhereClause(), subqueryTupleIds)) { > containsCorrelatedPredicate()? yes, good catch. further simplified. http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java: http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@277 PS8, Line 277: // NOT IN subquery with a correlated non-equi predicate is ok if the subquery only > [NOT] IN subquery Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@292 PS8, Line 292: AnalyzesOk(String.format("select 1 from functional.allcomplextypes t " + > What about queries without complex types and non-constant LHS in the IN pre there's several above, for example L220 has a non-constant LHS and the types are not complex. L92 loops over a bunch of scalar types (and non-constant LHS). http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@299 PS8, Line 299: AnalysisError(String.format("select * from functional.alltypestiny t where id %s " + > These tests are pretty rough to follow, perhaps we should spent some time b makes sense. the cut in this patch is just to get the tests to exercise the various branches more consistently where applicable. http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@437 PS8, Line 437: // Tests for constant on the left hand side of [NOT] IN. > Since we are doing a double rewrite here, it's interesting to test nested s separated these out. I looked at making constant LHS just another dimension of the existing tests but I think a more comprehensive clean up here is better as a separate change. for now, I've added several more tests. http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@476 PS8, Line 476: > You can omit this part if you like. Error message matching is prefix based. Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@482 PS8, Line 482: "SELECT int_col FROM functional.alltypestiny b WHERE b.id = a.id ORDER BY int_col ASC LIMIT 5"); > long line, you can omit this line in the expected error message Done http://gerrit.cloudera.org:8080/#/c/8322/8/fe/src/test/java/org/apache/impala/analysis/AnalyzeSubqueriesTest.java@520 PS8, Line 520: + "(select * from functional.tinyinttable x where t.id = x.int_col)"); > please move "+" to previous line (easier to read and more consistent with e Done
[Impala-ASF-CR] IMPALA-6255: Add device names to DiskIoMgr thread names
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8669 ) Change subject: IMPALA-6255: Add device names to DiskIoMgr thread names .. Patch Set 2: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/1546/ -- To view, visit http://gerrit.cloudera.org:8080/8669 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I30faeda6db8846e4aad64ce29ca811366d84910b Gerrit-Change-Number: 8669 Gerrit-PatchSet: 2 Gerrit-Owner: Lars VolkerGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 30 Nov 2017 09:20:28 + Gerrit-HasComments: No