[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Anurag Mantripragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/14707 ) Change subject: IMPALA-9081: fix mt_dop validation tests .. Patch Set 5: Code-Review+1 Looks good to me. -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Fri, 15 Nov 2019 01:27:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9092 : Disable show create table tests on Kudu
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14664 ) Change subject: IMPALA-9092 : Disable show create table tests on Kudu .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5024/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I37c0b6d82372bc6380285afcd94f0c1e123f2eda Gerrit-Change-Number: 14664 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 15 Nov 2019 00:28:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9092 : Disable show create table tests on Kudu
Vihang Karajgaonkar has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/14664 ) Change subject: IMPALA-9092 : Disable show create table tests on Kudu .. IMPALA-9092 : Disable show create table tests on Kudu This patch temporarily disables the Kudu tests which fail when we try to bump up the CDP build number due the HMS translation. The tests will be re-enabled back again when the fix for IMPALA-9092 is submitted. Testing Done: 1. Bumped up the CDP build number to 1507246 which has the HMS translation in it. 2. Ran core tests and found the tests which are failing for Kudu due to this issue. 3. Ran the failing tests again and confirm they are not failing anymore. Change-Id: I37c0b6d82372bc6380285afcd94f0c1e123f2eda --- M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M tests/common/skip.py M tests/metadata/test_ddl.py M tests/metadata/test_show_create_table.py M tests/query_test/test_kudu.py 6 files changed, 25 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/14664/4 -- To view, visit http://gerrit.cloudera.org:8080/14664 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I37c0b6d82372bc6380285afcd94f0c1e123f2eda Gerrit-Change-Number: 14664 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14707 ) Change subject: IMPALA-9081: fix mt_dop validation tests .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5023/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 14 Nov 2019 23:11:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14707 ) Change subject: IMPALA-9081: fix mt_dop validation tests .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5022/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 14 Nov 2019 23:10:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14707 ) Change subject: IMPALA-9081: fix mt_dop validation tests .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5216/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Thu, 14 Nov 2019 22:35:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Hello Thomas Tauber-Marshall, Anurag Mantripragada, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14707 to look at the new patch set (#5). Change subject: IMPALA-9081: fix mt_dop validation tests .. IMPALA-9081: fix mt_dop validation tests This undoes the hack of pretending that it's not a test environment for that single test. That had side effects, e.g. for the metadata loading path. Instead we have a special flag to enable the validation code in frontend tests. Note that the plans change to include join build sinks as an expected result of undoing the hack. Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f --- M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation-hdfs-num-rows-est-enabled.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 5 files changed, 87 insertions(+), 52 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/14707/5 -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-9081: fix mt dop validation tests
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14707 to look at the new patch set (#4). Change subject: IMPALA-9081: fix mt_dop validation tests .. IMPALA-9081: fix mt_dop validation tests This undoes the hack of pretending that it's not a test environment for that single test. That had side effects, e.g. for the metadata loading path. Instead we have a special flag to enable the validation code in frontend tests. Note that the plans change to include join build sinks as an expected result of undoing the hack. Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f --- M fe/src/main/java/org/apache/impala/common/RuntimeEnv.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation-hdfs-num-rows-est-enabled.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 5 files changed, 88 insertions(+), 52 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/14707/4 -- To view, visit http://gerrit.cloudera.org:8080/14707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2e8823c562395e13f318d1ad6eed883d2d9d771f Gerrit-Change-Number: 14707 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8525: preads should use hdfsPreadFully rather than hdfsPread
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14635 ) Change subject: IMPALA-8525: preads should use hdfsPreadFully rather than hdfsPread .. Patch Set 6: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5213/ -- To view, visit http://gerrit.cloudera.org:8080/14635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I29ea34897096bc790abdeb98073a47f1c4c10feb Gerrit-Change-Number: 14635 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 21:30:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9128: part 2: dump traces for slow RPCs
Tim Armstrong has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/14668 )
Change subject: IMPALA-9128: part 2: dump traces for slow RPCs
..
IMPALA-9128: part 2: dump traces for slow RPCs
This adds trace events for data stream RPCs and
dumps them when they take longer than
--impala_slow_rpc_threshold_ms.
I needed to modify the KRPC code to do this because it
currently only dumps traces for RPCs with deadlines.
I plan to add some version of this upstream in Kudu
so that we don't diverge our KRPC implementation.
Example output from test_exchange_small_buffer:
I 08:38:53.732910 26509 rpcz_store.cc:265] Call
impala.DataStreamService.TransmitData from 127.0.0.1:42434 (request call id 43)
took 7799ms. Request Metrics: {}
I 08:38:53.732928 26509 rpcz_store.cc:269] Trace:
08:38:45.933412 (+ 0us) impala-service-pool.cc:167] Inserting onto
call queue
08:38:45.933449 (+37us) impala-service-pool.cc:254] Handling call
08:38:45.933470 (+21us) krpc-data-stream-mgr.cc:227] Added early sender
08:38:47.906542 (+1973072us) krpc-data-stream-recvr.cc:327] Enqueuing
deferred RPC
08:38:53.732858 (+5826316us) krpc-data-stream-recvr.cc:506] Processing
deferred RPC
08:38:53.732860 (+ 2us) krpc-data-stream-recvr.cc:399] Deserializing
batch
08:38:53.732888 (+28us) krpc-data-stream-recvr.cc:426] Enqueuing
deserialized batch
08:38:53.732895 (+ 7us) inbound_call.cc:162] Queueing success response
Disabled +-clang-diagnostic-gnu-zero-variadic-macro-arguments because it
had false positives on the TRACE_TO invocations.
Testing:
* Ran exhaustive and ASAN tests
* Ran stress test
Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947
Reviewed-on: http://gerrit.cloudera.org:8080/14668
Reviewed-by: Tim Armstrong
Tested-by: Impala Public Jenkins
---
M .clang-tidy
M be/src/kudu/rpc/rpcz_store.cc
M be/src/runtime/krpc-data-stream-mgr.cc
M be/src/runtime/krpc-data-stream-recvr.cc
M tests/custom_cluster/test_exchange_deferred_batches.py
M tests/custom_cluster/test_exchange_delays.py
6 files changed, 43 insertions(+), 10 deletions(-)
Approvals:
Tim Armstrong: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/14668
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947
Gerrit-Change-Number: 14668
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14668 ) Change subject: IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947 Gerrit-Change-Number: 14668 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 20:18:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9085: [DOCS] Refactored impala s3.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/14627 ) Change subject: IMPALA-9085: [DOCS] Refactored impala_s3.xml .. Patch Set 3: Joe and Sahil, Could you check if this can be approved? -- To view, visit http://gerrit.cloudera.org:8080/14627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib274968a0412b4b8757f31ab674d4b82311de70a Gerrit-Change-Number: 14627 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 14 Nov 2019 18:48:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9128: part 2: dump traces for slow RPCs
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14668 ) Change subject: IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 7: I didn't see a significant perf change in any of the exchange targeted perf benchmarks. -- To view, visit http://gerrit.cloudera.org:8080/14668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947 Gerrit-Change-Number: 14668 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 18:33:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8906: Fix flaky profile observability test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14704 ) Change subject: IMPALA-8906: Fix flaky profile observability test .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7f6172d75294c1dea8f6be086ebb303725c92620 Gerrit-Change-Number: 14704 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Nov 2019 17:56:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/14588 )
Change subject: IMPALA-6660: Change -0/+0 floating point to compare as equal in
hash table
..
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/14588/4/be/src/codegen/codegen-anyval.cc
File be/src/codegen/codegen-anyval.cc:
http://gerrit.cloudera.org:8080/#/c/14588/4/be/src/codegen/codegen-anyval.cc@787
PS4, Line 787: // Check for -0 == +0 equality
: llvm::Value* cmp_zero =
builder_->CreateFCmpOEQ(ConvertToPositiveZero(local_val),
: ConvertToPositiveZero(val), "cmp_zero");
: llvm::Value* cmp_raw_zero =
: builder_->CreateOr(cmp_raw, cmp_zero,
"cmp_raw_with_zero");
Can you check whether this is really needed?
http://gerrit.cloudera.org:8080/#/c/14588/4/be/src/exec/hash-table.cc
File be/src/exec/hash-table.cc:
http://gerrit.cloudera.org:8080/#/c/14588/4/be/src/exec/hash-table.cc@257
PS4, Line 257: if (RawValue::IsFloatingZero(val, expr_type)) {
: val =
const_cast(RawValue::PositiveFloatingZero(expr_type));
: }
: if (RawValue::IsFloatingZero(loc, expr_type)) {
: loc =
const_cast(RawValue::PositiveFloatingZero(expr_type));
: }
: if (RawValue::Eq(loc, val, expr_type)) continue;
Does this actually make a difference? RawValue::Eq simply compares doubles and
floats with == which should return true for 0.0=-0.0
I think that the only functional difference is using CanonicalValue in EvalRow,
because its output is hashed bit by bit, so -0 and 0 are different there.
--
To view, visit http://gerrit.cloudera.org:8080/14588
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6bb1a817c81c452d041238c19cb6c9f602a5d565
Gerrit-Change-Number: 14588
Gerrit-PatchSet: 4
Gerrit-Owner: Norbert Luksa
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Daniel Becker
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Norbert Luksa
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Thu, 14 Nov 2019 16:59:35 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8525: preads should use hdfsPreadFully rather than hdfsPread
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14635 ) Change subject: IMPALA-8525: preads should use hdfsPreadFully rather than hdfsPread .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5213/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/14635 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I29ea34897096bc790abdeb98073a47f1c4c10feb Gerrit-Change-Number: 14635 Gerrit-PatchSet: 6 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 16:57:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9128: part 2: dump traces for slow RPCs
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14668 ) Change subject: IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 7: 17:07:05 E: Failed to fetch http://us-west-2.ec2.archive.ubuntu.com/ubuntu/dists/xenial-updates/main/binary-amd64/Packages.xz Hash Sum mismatch 17:07:05 E: Some index files failed to download. They have been ignored, or old ones used instead. -- To view, visit http://gerrit.cloudera.org:8080/14668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947 Gerrit-Change-Number: 14668 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 15:44:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9128: part 2: dump traces for slow RPCs
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14668 ) Change subject: IMPALA-9128: part 2: dump traces for slow RPCs .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5212/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7af4b45c43ec731d742d3696112c5f800849947 Gerrit-Change-Number: 14668 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Nov 2019 15:44:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8906: Fix flaky profile observability test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14704 ) Change subject: IMPALA-8906: Fix flaky profile observability test .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5211/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/14704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7f6172d75294c1dea8f6be086ebb303725c92620 Gerrit-Change-Number: 14704 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Nov 2019 13:23:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8906: Fix flaky profile observability test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14704 ) Change subject: IMPALA-8906: Fix flaky profile observability test .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5021/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7f6172d75294c1dea8f6be086ebb303725c92620 Gerrit-Change-Number: 14704 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Nov 2019 13:22:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8906: Fix flaky profile observability test
Tamas Mate has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14704 ) Change subject: IMPALA-8906: Fix flaky profile observability test .. IMPALA-8906: Fix flaky profile observability test The test_query_profile_contains_query_compilation_metadata_load_events is flaky because only non-zero stats are printed and the test is not prepared to handle lines appearing based on conditions. In general, zero values are printed in the profile as well, this makes profile parsing simpler. Therefore, instead of changing the test the condition has been removed as part of this change and the test is updated accordingly. Change-Id: I7f6172d75294c1dea8f6be086ebb303725c92620 --- M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java M tests/query_test/test_observability.py 3 files changed, 14 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/14704/3 -- To view, visit http://gerrit.cloudera.org:8080/14704 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7f6172d75294c1dea8f6be086ebb303725c92620 Gerrit-Change-Number: 14704 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14588 ) Change subject: IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5020/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14588 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb1a817c81c452d041238c19cb6c9f602a5d565 Gerrit-Change-Number: 14588 Gerrit-PatchSet: 4 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 14 Nov 2019 09:41:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8755: Backend support for Z-ordering
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14080 ) Change subject: IMPALA-8755: Backend support for Z-ordering .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5019/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/14080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab Gerrit-Change-Number: 14080 Gerrit-PatchSet: 12 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Anonymous Coward (520) Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 14 Nov 2019 09:00:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table
Norbert Luksa has posted comments on this change. ( http://gerrit.cloudera.org:8080/14588 ) Change subject: IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/14588/3/be/src/codegen/codegen-anyval.h File be/src/codegen/codegen-anyval.h: http://gerrit.cloudera.org:8080/#/c/14588/3/be/src/codegen/codegen-anyval.h@248 PS3, Line 248: // Replaces negative zero with positive, leaves everything else unchanged. > nit: maybe mention that it is for floating point numbers Done http://gerrit.cloudera.org:8080/#/c/14588/3/be/src/runtime/raw-value.inline.h File be/src/runtime/raw-value.inline.h: http://gerrit.cloudera.org:8080/#/c/14588/3/be/src/runtime/raw-value.inline.h@57 PS3, Line 57: std::abs > is std::abs needed? Done -- To view, visit http://gerrit.cloudera.org:8080/14588 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bb1a817c81c452d041238c19cb6c9f602a5d565 Gerrit-Change-Number: 14588 Gerrit-PatchSet: 3 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 14 Nov 2019 08:54:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table
Norbert Luksa has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/14588 ) Change subject: IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table .. IMPALA-6660: Change -0/+0 floating point to compare as equal in hash table Currently -0/+0 values are not treated as equal. This commit changes this behaviour for hash joins and aggregations (since aggregations follow the behaviour of the join). That way, the canonical form for -0/+0 is changed to +0. Tests: - Added relevant tests. Change-Id: I6bb1a817c81c452d041238c19cb6c9f602a5d565 --- M be/src/codegen/codegen-anyval.cc M be/src/codegen/codegen-anyval.h M be/src/exec/hash-table.cc M be/src/runtime/raw-value.cc M be/src/runtime/raw-value.h M be/src/runtime/raw-value.inline.h M testdata/workloads/functional-query/queries/QueryTest/aggregation.test M testdata/workloads/functional-query/queries/QueryTest/joins.test 8 files changed, 108 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/14588/4 -- To view, visit http://gerrit.cloudera.org:8080/14588 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6bb1a817c81c452d041238c19cb6c9f602a5d565 Gerrit-Change-Number: 14588 Gerrit-PatchSet: 4 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8755: Backend support for Z-ordering
Norbert Luksa has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/14080 ) Change subject: IMPALA-8755: Backend support for Z-ordering .. IMPALA-8755: Backend support for Z-ordering This change depends on gerrit.cloudera.org/#/c/13955/ (Frontend support for Z-ordering) The commit adds a Comparator based on Z-ordering. See in detail: https://en.wikipedia.org/wiki/Z-order_curve The comparator instead of calculating the Z-values of the rows, looks for the column with the most significant dimension, and compares the values of this column only. The most significant dimension will be the one where the compared values have the highest different bits. The algorithm requires values of the same binary representation, but this can be relaxed. All primitive types (including string and floating point types) are supported. Testing: * Added unit tests. * Run manual tests, comparing 4-column values with 4-bit integers, for all possible combinations. Checked the result by calculating the Z-value for each comparison. * Tested performance on various data, getting great results. Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab --- M be/src/exec/exchange-node.cc M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/exec/parquet/hdfs-parquet-table-writer.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partial-sort-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/exec/topn-node.cc M be/src/runtime/data-stream-test.cc M be/src/runtime/sorter.cc M be/src/runtime/sorter.h M be/src/util/CMakeLists.txt A be/src/util/tuple-row-compare-test.cc M be/src/util/tuple-row-compare.cc M be/src/util/tuple-row-compare.h M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java 18 files changed, 1,002 insertions(+), 95 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/14080/12 -- To view, visit http://gerrit.cloudera.org:8080/14080 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0200748ce3e65ebc5d3530f794c0f80aa335a2ab Gerrit-Change-Number: 14080 Gerrit-PatchSet: 12 Gerrit-Owner: Norbert Luksa Gerrit-Reviewer: Anonymous Coward (520) Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Zoltan Borok-Nagy
