[Impala-ASF-CR] IMPALA-7829: Mark a fragment instance as done only after Close() is called
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11939 ) Change subject: IMPALA-7829: Mark a fragment instance as done only after Close() is called .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1381/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I61618854ae3f4e7ef20028dcb0ff5cbcfa8adb01 Gerrit-Change-Number: 11939 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 16 Nov 2018 03:56:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7829: Mark a fragment instance as done only after Close() is called
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11939 Change subject: IMPALA-7829: Mark a fragment instance as done only after Close() is called .. IMPALA-7829: Mark a fragment instance as done only after Close() is called As shown in IMPALA-7828. there is some non-determinism on whether the errors detected in FragmentInstanceState::Close() will show up in the final profile sent to the coordinator. The reason is that the current code marks a fragment instance as "done" after ExecInternal() completes but before Close() is called. There is a window between when the final status report is sent and when Close() finishes. This change fixes the problem by not sending the final report until Close() is called. This has no implication on the first row available time for normal queries. It may slightly lengthen the first row available time for DML queries. Testing done: Updated udf-no-expr-rewrite.test to exercise this test Perf run on an 8 node clusters didn't show any regression: TPCH-300 ++---+-++++ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | ++---+-++++ | TPCH(_300) | parquet / none / none | 23.94 | -2.05% | 12.55 | -2.62% | ++---+-++++ Small concurrency +-+---+-++++ | Workload| File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +-+---+-++++ | TPCDS-UNMODIFIED(_1000) | parquet / none / none | 6.89| -0.66% | 6.62 | +0.41% | +-+---+-++++ Medium concurrency +-+---+-++++ | Workload| File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +-+---+-++++ | TPCDS-UNMODIFIED(_1000) | parquet / none / none | 55.57 | -1.04% | 55.27 | -0.98% | +-+---+-++++ Change-Id: I61618854ae3f4e7ef20028dcb0ff5cbcfa8adb01 --- M be/src/runtime/fragment-instance-state.cc M be/src/runtime/fragment-instance-state.h M testdata/workloads/functional-query/queries/QueryTest/udf-no-expr-rewrite.test 3 files changed, 11 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/11939/1 -- To view, visit http://gerrit.cloudera.org:8080/11939 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I61618854ae3f4e7ef20028dcb0ff5cbcfa8adb01 Gerrit-Change-Number: 11939 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. Patch Set 6: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3465/ -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Nov 2018 03:05:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3466/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 3 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 02:52:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 3 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 02:52:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7857: log more information about statestore failure detection
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11937 ) Change subject: IMPALA-7857: log more information about statestore failure detection .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1379/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11937 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6ff85bee117000e4434dcffd3d1680a79905f14 Gerrit-Change-Number: 11937 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anuj Phadke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 01:39:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 01:27:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1378/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 01:34:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7047. Refreshing partitions should not make an RPC per file
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11227 ) Change subject: IMPALA-7047. Refreshing partitions should not make an RPC per file .. Patch Set 3: Is this ready to get merged? -- To view, visit http://gerrit.cloudera.org:8080/11227 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2051b96599206164aaa06ecbdf64374c46eda956 Gerrit-Change-Number: 11227 Gerrit-PatchSet: 3 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 16 Nov 2018 01:15:16 + Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code"
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11699 ) Change subject: Revert "IMPALA-6910/IMPALA-7070: Increase log level for HDFS S3 code" .. Patch Set 2: Any luck? -- To view, visit http://gerrit.cloudera.org:8080/11699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5ab7f2f6317f1281928af5ea6cf3fd7d0c6e0a09 Gerrit-Change-Number: 11699 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Nov 2018 01:11:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: Zero-length arrays are undefined behavior
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11811 ) Change subject: IMPALA-5031: Zero-length arrays are undefined behavior .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11811/1/be/src/runtime/row-batch-serialize-test.cc File be/src/runtime/row-batch-serialize-test.cc: http://gerrit.cloudera.org:8080/#/c/11811/1/be/src/runtime/row-batch-serialize-test.cc@242 PS1, Line 242: int len = (rand() % MAX_STRING_LEN) + 1; I think we want to generate empty strings as random values. Why not just make buf a non-VLA of MAX_STRING_LEN. -- To view, visit http://gerrit.cloudera.org:8080/11811 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93f7d0b0506e4b6a2ff3303477d887a428431f96 Gerrit-Change-Number: 11811 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jinchul Kim Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Nov 2018 01:11:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5031: signed overflow is undefined behavior
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11917 ) Change subject: IMPALA-5031: signed overflow is undefined behavior .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I146bcf35d34cc0e14be0633427d3e4bd0e5a261e Gerrit-Change-Number: 11917 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 16 Nov 2018 01:09:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Hello David Knupp, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11938 to look at the new patch set (#2). Change subject: IMPALA-7585: support LDAP in run-workload.py .. IMPALA-7585: support LDAP in run-workload.py This patch just threads through the user, password, and ssl settings all the way back to the ImpalaBeeswaxClient. Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba --- M bin/run-workload.py M tests/performance/query_exec_functions.py M tests/performance/query_executor.py M tests/performance/workload_runner.py 4 files changed, 20 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11938/2 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7811: optionally count JVM heap towards process mem limit
Tim Armstrong has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/10928 ) Change subject: IMPALA-7811: optionally count JVM heap towards process mem limit .. IMPALA-7811: optionally count JVM heap towards process mem limit Adds a flag --mem_limit_includes_jvm that alters memory accounting to include the amount of memory we think that the JVM is likely to use. By default this flag is false, so behaviour is unchanged. We're not ready to change the default but I want to check this in to enable experimentation. Two metrics are counted towards the process limit: * The maximum JVM heap size. We count this because the JVM memory consumption can expand up to this threshold at any time. * JVM non-heap committed memory. This can be a non-trivial amount of memory (e.g. I saw 150MB on one production cluster). There isn't a hard upper bound on this memory that I know of but should not row rapidly. This requires adjustments in a couple of other places: * We should not admit query memory in addition to the JVM memory. * The buffer pool is now a percentage of the remaining process limit after the JVM heap, instead of the total process limit. Currently, end-to-end tests fail if run with this flag for two reasons: * The default JVM heap size is 1/4 of physical memory, which means that essentially all of the process memory limit is consumed by the JVM heaps when we running 3 impala daemons per host, unless -Xmx is explicitly set. * If the heap size is limited to 1-2GB like below, then most tests pass but TestInsert.test_insert_large_string fails because IMPALA-4865 lets it create giant strings that eat up all the JVM heap. start-impala-cluster.py \ --impalad_args=--mem_limit_includes_jvm=true --jvm_args="-Xmx1g" Testing: Add a custom cluster test that uses the new option and validates the the memory consumption values. Change-Id: I39dd715882a32fc986755d573bd46f0fd9eefbfc --- M be/src/common/global-flags.cc M be/src/runtime/exec-env.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/scheduler.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M be/src/util/memory-metrics.cc M be/src/util/memory-metrics.h M common/thrift/StatestoreService.thrift M tests/common/custom_cluster_test_suite.py A tests/custom_cluster/test_jvm_mem_tracking.py M www/backends.tmpl 13 files changed, 187 insertions(+), 55 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/10928/8 -- To view, visit http://gerrit.cloudera.org:8080/10928 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I39dd715882a32fc986755d573bd46f0fd9eefbfc Gerrit-Change-Number: 10928 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-7857: log more information about statestore failure detection
Tim Armstrong has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11937 ) Change subject: IMPALA-7857: log more information about statestore failure detection .. IMPALA-7857: log more information about statestore failure detection This adds a couple of log messages for state transitions in the statestore's failure detector. Testing: Ran test_statestore.py and checked for presence of new log messages. Added a new tests to test_statestore that exercises handling of intermittent heartbeat failures (required to produce one of the new log messages). Change-Id: Ie6ff85bee117000e4434dcffd3d1680a79905f14 --- M be/src/statestore/failure-detector.cc M be/src/statestore/failure-detector.h M tests/statestore/test_statestore.py 3 files changed, 44 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/37/11937/2 -- To view, visit http://gerrit.cloudera.org:8080/11937 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie6ff85bee117000e4434dcffd3d1680a79905f14 Gerrit-Change-Number: 11937 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11938 ) Change subject: IMPALA-7585: support LDAP in run-workload.py .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11938/1/bin/run-workload.py File bin/run-workload.py: http://gerrit.cloudera.org:8080/#/c/11938/1/bin/run-workload.py@110 PS1, Line 110: " flake8: E501 line too long (92 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 16 Nov 2018 00:44:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7585: support LDAP in run-workload.py
Jim Apple has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11938 Change subject: IMPALA-7585: support LDAP in run-workload.py .. IMPALA-7585: support LDAP in run-workload.py This patch just threads through the user, password, and ssl settings all the way back to the ImpalaBeeswaxClient. Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba --- M bin/run-workload.py M tests/performance/query_exec_functions.py M tests/performance/query_executor.py M tests/performance/workload_runner.py 4 files changed, 20 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/11938/1 -- To view, visit http://gerrit.cloudera.org:8080/11938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibfa987d8a027f50bc1ba3db5aa355331442a74ba Gerrit-Change-Number: 11938 Gerrit-PatchSet: 1 Gerrit-Owner: Jim Apple
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 5: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1376/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 16 Nov 2018 00:27:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11890 ) Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1377/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 2 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 16 Nov 2018 00:17:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 4: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1375/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Fri, 16 Nov 2018 00:16:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Paul Rogers has posted comments on this change. (
http://gerrit.cloudera.org:8080/11890 )
Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
..
Patch Set 2:
(4 comments)
Thanks for the reviews. Addressed the comments, including removing the new (but
disabled) tests. Will include those later along with the fixes that they
motivated.
http://gerrit.cloudera.org:8080/#/c/11890/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11890/1//COMMIT_MSG@10
PS1, Line 10: Th
> ?
Done
http://gerrit.cloudera.org:8080/#/c/11890/1//COMMIT_MSG@26
PS1, Line 26: Testing: this is a test, ran the test to ensure it still passes.
:
> Should we add these tests separately along with the jira fixes? Not sure wh
Sure.
http://gerrit.cloudera.org:8080/#/c/11890/1/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java
File fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java:
http://gerrit.cloudera.org:8080/#/c/11890/1/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java@438
PS1, Line 438: l FALSE, return implicit NULL ELSE.
: RewritesOk("decode(1, 1 + 1, id, 1 + 2, 3)", rules, "NULL");
: // Multiple TRUE, first one becomes ELSE.
> nit: format to fewer lines
Done
http://gerrit.cloudera.org:8080/#/c/11890/1/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java@484
PS1, Line 484: ("coalesce(1,
> nit: more meaningful name?
Done
--
To view, visit http://gerrit.cloudera.org:8080/11890
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083
Gerrit-Change-Number: 11890
Gerrit-PatchSet: 2
Gerrit-Owner: Paul Rogers
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Paul Rogers
Gerrit-Comment-Date: Thu, 15 Nov 2018 23:41:08 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging
Hello Bharath Vissapragada, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11890 to look at the new patch set (#2). Change subject: IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging .. IMPALA-7821: Refactor ExprRewriteRulesTest for easier debugging Impala provides a single test class ExprRewriteRulesTest for all rewrite rules. Each rule class has an associated test function. The SimplifyConditionalRule class is complex, and the result test function is quite large. When doing work to modify a particular conditional rewrite, it became clear that debugging would be much easier if each detailed rewrite had its own test rather than using one huge test function. This ticket asks to break up the big function. (JUnit does not care about small vs. large functions.) This patch splits the function separate from later work that will modify the tests themselves. Also unified test function naming, using the standard "javaStyle" rather than the unusual "CppStyle" that was used. Testing: this is a test, ran the test to ensure it still passes. Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 --- M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java 1 file changed, 76 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/11890/2 -- To view, visit http://gerrit.cloudera.org:8080/11890 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I99a28b170bc0132424041e51f61ebe5c848c4083 Gerrit-Change-Number: 11890 Gerrit-PatchSet: 2 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7823: Clean up Java warnings, fix minor issues
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11893 ) Change subject: IMPALA-7823: Clean up Java warnings, fix minor issues .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/11893/4/fe/src/main/java/org/apache/impala/analysis/CollectionStructType.java File fe/src/main/java/org/apache/impala/analysis/CollectionStructType.java: http://gerrit.cloudera.org:8080/#/c/11893/4/fe/src/main/java/org/apache/impala/analysis/CollectionStructType.java@28 PS4, Line 28: import com.google.common.base.Preconditions; This turns out to be an excellent change. Thanks! -- To view, visit http://gerrit.cloudera.org:8080/11893 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4b174d6448371c55e98487c6d4212c74c6b0c79e Gerrit-Change-Number: 11893 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Anonymous Coward (168) Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Thu, 15 Nov 2018 23:27:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] Added a note in impala scan bytes limit.xml
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11936 ) Change subject: [DOCS] Added a note in impala_scan_bytes_limit.xml .. [DOCS] Added a note in impala_scan_bytes_limit.xml - Added a note that queries can scan over the limit of this option at times. Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Reviewed-on: http://gerrit.cloudera.org:8080/11936 Tested-by: Impala Public Jenkins Reviewed-by: Bikramjeet Vig --- M docs/topics/impala_scan_bytes_limit.xml 1 file changed, 8 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Verified Bikramjeet Vig: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11936 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Gerrit-Change-Number: 11936 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] Added a note in impala scan bytes limit.xml
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11936 ) Change subject: [DOCS] Added a note in impala_scan_bytes_limit.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11936 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Gerrit-Change-Number: 11936 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 23:13:23 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Added a note in impala scan bytes limit.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11936 ) Change subject: [DOCS] Added a note in impala_scan_bytes_limit.xml .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/151/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11936 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Gerrit-Change-Number: 11936 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 23:10:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11240 ) Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml File docs/topics/impala_scan_bytes_limit.xml: http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml@63 PS2, Line 63: time > yes, also re-word it to refer to a scan limit. Something like: Done -- To view, visit http://gerrit.cloudera.org:8080/11240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb Gerrit-Change-Number: 11240 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 23:03:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] Added a note in impala scan bytes limit.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11936 ) Change subject: [DOCS] Added a note in impala_scan_bytes_limit.xml .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/151/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11936 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Gerrit-Change-Number: 11936 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 23:03:37 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Added a note in impala scan bytes limit.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11936 Change subject: [DOCS] Added a note in impala_scan_bytes_limit.xml .. [DOCS] Added a note in impala_scan_bytes_limit.xml - Added a note that queries can scan over the limit of this option at times. Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f --- M docs/topics/impala_scan_bytes_limit.xml 1 file changed, 8 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/11936/1 -- To view, visit http://gerrit.cloudera.org:8080/11936 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id20f952622ce553d9c1f47e469d97a3b4c19683f Gerrit-Change-Number: 11936 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11240 ) Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml File docs/topics/impala_scan_bytes_limit.xml: http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml@63 PS2, Line 63: time > Bikram, yes, also re-word it to refer to a scan limit. Something like: the query will be automatically terminated after it scans more data than the scan bytes limit -- To view, visit http://gerrit.cloudera.org:8080/11240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb Gerrit-Change-Number: 11240 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 22:52:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Hello Bharath Vissapragada, Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11881 to look at the new patch set (#5). Change subject: IMPALA-7807: Analysis test fixture .. IMPALA-7807: Analysis test fixture Refactors the existing ExprRewriteRulesTest to pull out the framework functionality into a "test fixture" class which can be used for a variety of tests. The fixture allows more variation, and access to more intermediate state, than does the existing function-based framework. For example, the fixture allows setting per-query options, allows doing full or partial (no rewrite) analysis, provides access to things like the Analyzer, and more. The fixture pulls in code from FrontEndBase to handle the low-level tasks of running a query, and from ExprRewriteRulesTest for the rewrite-specific aspects. For now, the FrontEndBase base class was left unchanged. ExprRewriteRulesTest is refactored to use the new fixture in order to illustrate usage of the test fixture. The key value of this work is to allow greater detail when testing future change requests. In order to keep the refactoring as simple and clean, no new test cases were added here Testing: since this is a test fixture, the refactored ExprRewriteRulesTest implicitly tests the fixture code. No "production" (non-test) code was changed in this patch. Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c --- A fe/src/test/java/org/apache/impala/analysis/AnalysisFixture.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java A fe/src/test/java/org/apache/impala/analysis/QueryFixture.java A fe/src/test/java/org/apache/impala/analysis/RewriteFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java 6 files changed, 503 insertions(+), 89 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/11881/5 -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 5 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11881 ) Change subject: IMPALA-7807: Analysis test fixture .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/11881/4/fe/src/test/java/org/apache/impala/analysis/QueryFixture.java File fe/src/test/java/org/apache/impala/analysis/QueryFixture.java: http://gerrit.cloudera.org:8080/#/c/11881/4/fe/src/test/java/org/apache/impala/analysis/QueryFixture.java@72 PS4, Line 72: parsedStmt_ = this.analysisFixture_.frontend_.parse(stmt_, analysCtx_.getQueryOptions()); line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/11881/4/fe/src/test/java/org/apache/impala/analysis/QueryFixture.java@118 PS4, Line 118:new StmtMetadataLoader(this.analysisFixture_.frontend_, analysCtx_.getQueryCtx().session.database, null); line too long (112 > 90) -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 22:15:47 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7807: Analysis test fixture
Hello Bharath Vissapragada, Fredy Wijaya, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11881 to look at the new patch set (#4). Change subject: IMPALA-7807: Analysis test fixture .. IMPALA-7807: Analysis test fixture Refactors the existing ExprRewriteRulesTest to pull out the framework functionality into a "test fixture" class which can be used for a variety of tests. The fixture allows more variation, and access to more intermediate state, than does the existing function-based framework. For example, the fixture allows setting per-query options, allows doing full or partial (no rewrite) analysis, provides access to things like the Analyzer, and more. The fixture pulls in code from FrontEndBase to handle the low-level tasks of running a query, and from ExprRewriteRulesTest for the rewrite-specific aspects. For now, the FrontEndBase base class was left unchanged. ExprRewriteRulesTest is refactored to use the new fixture in order to illustrate usage of the test fixture. The key value of this work is to allow greater detail when testing future change requests. In order to keep the refactoring as simple and clean, no new test cases were added here Testing: since this is a test fixture, the refactored ExprRewriteRulesTest implicitly tests the fixture code. No "production" (non-test) code was changed in this patch. Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c --- A fe/src/test/java/org/apache/impala/analysis/AnalysisFixture.java M fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java A fe/src/test/java/org/apache/impala/analysis/QueryFixture.java A fe/src/test/java/org/apache/impala/analysis/RewriteFixture.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/testutil/TestUtils.java 6 files changed, 500 insertions(+), 89 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/11881/4 -- To view, visit http://gerrit.cloudera.org:8080/11881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id69c99b284960f16394f61072d38dd00269bc10c Gerrit-Change-Number: 11881 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. IMPALA-5821: Add query with implicit casts to extended explain output. If explain_level is at 'extended' level or higher, then enhance the output from the explain command. (1) Show the analyzed sql in the explain header, this is the rewritten sql, which includes implicit casts, and literals are printed with a cast so that their type is visible. (2) When predicates are shown in the plan these are shown in the same format. The toSql() method can be called on a ParseNode tree to return the sql corresponding ot the tree. In the past toSQl() has been enhanced to print rewritten sql by partially overloading toSql() [with toSql(boolean)]. This current change requires changing toSQl() in many places as NumericLiteral can appear at different points in ia parse tree. To avoid many new fragile overloads of toSql() I added toSql(ToSqlOptions), where ToSqlOptions is an enum which controls the form of the Sql that is returned. This changes many files but is safer and means that any future options to toSql() can be added painlessly. If SHOW_IMPLICIT_CASTS is passed to toSql() then - in CastExpr print the implicit cast - in NumericLiteral print the literal with a cast to show the type Add a PlannerTestOption directive that will force the query text showing implicit casts to be included in the PLAN section of a .test file. The analyzed query text is wrapped at 80 characters. Note that the analyzed query cannot always be executed as queries rewritten to use LEFT SEMI JOIN are not legal sql. In addition some space characters may be removed from the query for prettier display. Documentation of this change will be done as IMPALA-7718 EXAMPLE OUTPUT: [localhost:21000] default> set explain_level=2; EXPLAIN_LEVEL set to 2 [localhost:21000] default> explain select * from functional_kudu.alltypestiny where bigint_col < 1000 / 100; Query: explain select * from functional_kudu.alltypestiny where bigint_col < 1000 / 100 Max Per-Host Resource Reservation: Memory=0B Threads=2 Per-Host Resource Estimates: Memory=10MB Codegen disabled by planner Analyzed query: SELECT * FROM functional_kudu.alltypestiny WHERE CAST(bigint_col AS DOUBLE) < CAST(10 AS DOUBLE) "" F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=4.88MB mem-reservation=0B thread-reservation=2 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 00:SCAN KUDU [functional_kudu.alltypestiny] predicates: CAST(bigint_col AS DOUBLE) < CAST(10 AS DOUBLE) mem-estimate=4.88MB mem-reservation=0B thread-reservation=1 tuple-ids=0 row-size=97B cardinality=1 in pipelines: 00(GETNEXT) Fetched 16 row(s) in 0.03s TESTING: All end-to-end tests pass. Added a new test in ExprRewriterTest which prints sql with implict casts for some interesting queries. Add a unit test for the code which wraps text at 60 characters. The output of some Planner Tests in .test files has been updated to include the Analyzed sql that is printed when explain_level is at at least 'extended' level. Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 Reviewed-on: http://gerrit.cloudera.org:8080/11719 Tested-by: Impala Public Jenkins Reviewed-by: Thomas Marshall --- M fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddDropRangePartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableAddPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java M fe/src/main/java/org/apache/impala/analysis/AlterViewStmt.java M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java M fe/src/main/java/org/apache/impala/analysis/AnalyticExpr.java M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java M fe/src/main/java/org/apache/impala/analysis/ArithmeticExpr.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/BetweenPredicate.java M fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java M fe/src/main/java/org/apache/impala/analysis/BoolLiteral.java M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/CastExpr.java M fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDataSrcStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDbStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateDropRoleStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateOrAlterViewStmtBase.java M fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java M
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 Gerrit-Change-Number: 11719 Gerrit-PatchSet: 9 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 15 Nov 2018 21:32:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. Patch Set 9: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 Gerrit-Change-Number: 11719 Gerrit-PatchSet: 9 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 15 Nov 2018 21:31:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7836: [DOCS] Format changes in impala topn bytes limit.xml
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11935 ) Change subject: IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml .. IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Reviewed-on: http://gerrit.cloudera.org:8080/11935 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_topn_bytes_limit.xml 1 file changed, 9 insertions(+), 10 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Gerrit-Change-Number: 11935 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 22: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h@41 PS20, Line 41: > Actually I feel like a negative stride is even valid and has reasonable sem That makes sense to me, thanks. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 22 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 21:30:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7836: [DOCS] Format changes in impala topn bytes limit.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11935 ) Change subject: IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/150/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Gerrit-Change-Number: 11935 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 21:29:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7836: [DOCS] Format changes in impala topn bytes limit.xml
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11935 ) Change subject: IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Gerrit-Change-Number: 11935 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 21:20:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7836: [DOCS] Format changes in impala topn bytes limit.xml
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11935 ) Change subject: IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/150/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Gerrit-Change-Number: 11935 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 15 Nov 2018 21:19:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7836: [DOCS] Format changes in impala topn bytes limit.xml
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11935 Change subject: IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml .. IMPALA-7836: [DOCS] Format changes in impala_topn_bytes_limit.xml Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 --- M docs/topics/impala_topn_bytes_limit.xml 1 file changed, 9 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11935/1 -- To view, visit http://gerrit.cloudera.org:8080/11935 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I731b26fe2c225e706454f16cd3b6de697ec70fe2 Gerrit-Change-Number: 11935 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7392: [DOCS] SCAN BYTES LIMIT query option documented
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11240 ) Change subject: IMPALA-7392: [DOCS] SCAN_BYTES_LIMIT query option documented .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml File docs/topics/impala_scan_bytes_limit.xml: http://gerrit.cloudera.org:8080/#/c/11240/2/docs/topics/impala_scan_bytes_limit.xml@63 PS2, Line 63: time Bikram, Should this "time" be removed? -- To view, visit http://gerrit.cloudera.org:8080/11240 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6430e06cabe21b8080239f3225d3bfdd5cc502cb Gerrit-Change-Number: 11240 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 20:57:06 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h@41 PS20, Line 41: DCHECK_GE > Can the stride ever be zero? If not, DCHECK_GT? Actually I feel like a negative stride is even valid and has reasonable semantics - if you wanted to write an array backwards for example. So I'll just remove the DCHECK. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 20 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 20:34:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/20/be/src/util/mem-util.h@41 PS20, Line 41: DCHECK_GE Can the stride ever be zero? If not, DCHECK_GT? -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 20 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 20:06:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 21: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1373/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 21 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 19:36:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 20: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1372/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 20 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 19:10:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 21: (1 comment) Rebased http://gerrit.cloudera.org:8080/#/c/8319/21/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/8319/21/be/src/exec/parquet-column-readers.cc@948 PS21, Line 948: // TODO: converting timestamps after validating them can move them out of range. We Had to resolve a conflict here while rebasing. I solved it by duplicating the comment. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 21 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 19:01:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Hello Lars Volker, Zoltan Borok-Nagy, Csaba Ringhofer, Alex Behm, Mostafa Mokhtar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8319 to look at the new patch set (#21). Change subject: IMPALA-4123: Columnar decoding in Parquet .. IMPALA-4123: Columnar decoding in Parquet The idea is to optimise the common case where there are long runs of NULL or non-NULL values (i.e. the def level is repeated). We can detect this cheaply by keying the decoding loop in the column reader off the state of the def level RLE decoder - if there's a long run of repeated levels, we can skip checking the def level for every value. We still fall back to decoding, caching and reading value-by-value a batch of def levels whenever the next def level is not in a repeated run. We still use the old approach for decoding rep levels. There might be some benefit to using the same approach for rep levels *if* repeated def and rep level runs line up. These changes should unlock further optimizations because more time is spent in simple kernel functions, e.g. UnpackAndDecode32Values() for dictionary decompression, which is very optimisable using SIMD etc. Snappy decompression now seems to be the main CPU bottleneck for decoding snappy-compressed Parquet. Perf: Running TPC-H scale factor 60 on uncompressed and snappy parquet both showed a ~4% speedup overall. Microbenchmarks on uncompressed parquet show scans only doing dictionary decoding on uncompressed Parquet is ~75% faster: set mt_dop=1; select min(l_returnflag) from lineitem; Testing: We have alltypes agg with a mix of null and non-null. Many tables have long runs of non-null values. Added new test data and coverage: * a test table manynulls with long runs of null values. * a large CHAR test table * missing coverage for materialising pos slot in flattened nested types scan. * Extended dict test to test longer runs. * A larger version of complextypestbl with interesting collection shapes - NULL collections, empty collections, etc, particularly runs of collections with the same shape. * Test interaction of timestamp validation with conversion * Ran code coverage build to confirm all code paths are tested * ASAN and exhaustive runs. TODO: * Before merging, run fuzz test for longer Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/exec/parquet-common.h M be/src/runtime/tuple.cc M be/src/runtime/tuple.h M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/dict-test.cc A be/src/util/mem-util.h M be/src/util/rle-encoding.h M testdata/bin/generate-schema-statements.py M testdata/data/README A testdata/data/out_of_range_timestamp2_hive_211.parquet A testdata/data/out_of_range_timestamp_hive_211.parquet M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/chars.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-position.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-local-tz-conversion.test A testdata/workloads/functional-query/queries/QueryTest/scanners-many-nulls.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/query_test/test_scanners.py 27 files changed, 884 insertions(+), 133 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/8319/21 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 21 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3465/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 18:49:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 6 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 18:49:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1371/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 18:48:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 19: (6 comments) http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@313 PS19, Line 313: void ReadPositions(int64_t num_to_read, int tuple_size, uint8_t* tuple_mem) RESTRICT; > Is there a reason that *tuple_mem is not restricted here? Done http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@745 PS19, Line 745: return NeedsConversionInline() ? > This might be more readable as an if-statement Done http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@897 PS19, Line 897: // TODO: converting timestamps after validating them can move them out of range. We > Do we have a Jira for this already? I'm not sure - Csaba, do you know if there is one? http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h File be/src/runtime/tuple.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h@55 PS19, Line 55: to > nit: extra word Done http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h@73 PS19, Line 73: to > nit: double 'to' Done http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/util/mem-util.h@34 PS19, Line 34: T* current; > can you initialize to nullptr (or add a brief comment why we don't)? Instead switched to an explicit constructor that asserts that it's non-NULL. -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 19 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 18:46:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Hello Lars Volker, Zoltan Borok-Nagy, Csaba Ringhofer, Alex Behm, Mostafa Mokhtar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/8319 to look at the new patch set (#20). Change subject: IMPALA-4123: Columnar decoding in Parquet .. IMPALA-4123: Columnar decoding in Parquet The idea is to optimise the common case where there are long runs of NULL or non-NULL values (i.e. the def level is repeated). We can detect this cheaply by keying the decoding loop in the column reader off the state of the def level RLE decoder - if there's a long run of repeated levels, we can skip checking the def level for every value. We still fall back to decoding, caching and reading value-by-value a batch of def levels whenever the next def level is not in a repeated run. We still use the old approach for decoding rep levels. There might be some benefit to using the same approach for rep levels *if* repeated def and rep level runs line up. These changes should unlock further optimizations because more time is spent in simple kernel functions, e.g. UnpackAndDecode32Values() for dictionary decompression, which is very optimisable using SIMD etc. Snappy decompression now seems to be the main CPU bottleneck for decoding snappy-compressed Parquet. Perf: Running TPC-H scale factor 60 on uncompressed and snappy parquet both showed a ~4% speedup overall. Microbenchmarks on uncompressed parquet show scans only doing dictionary decoding on uncompressed Parquet is ~75% faster: set mt_dop=1; select min(l_returnflag) from lineitem; Testing: We have alltypes agg with a mix of null and non-null. Many tables have long runs of non-null values. Added new test data and coverage: * a test table manynulls with long runs of null values. * a large CHAR test table * missing coverage for materialising pos slot in flattened nested types scan. * Extended dict test to test longer runs. * A larger version of complextypestbl with interesting collection shapes - NULL collections, empty collections, etc, particularly runs of collections with the same shape. * Test interaction of timestamp validation with conversion * Ran code coverage build to confirm all code paths are tested * ASAN and exhaustive runs. TODO: * Before merging, run fuzz test for longer Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef --- M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h M be/src/exec/parquet-common.h M be/src/runtime/tuple.cc M be/src/runtime/tuple.h M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/dict-encoding.h M be/src/util/dict-test.cc A be/src/util/mem-util.h M be/src/util/rle-encoding.h M testdata/bin/generate-schema-statements.py M testdata/data/README A testdata/data/out_of_range_timestamp2_hive_211.parquet A testdata/data/out_of_range_timestamp_hive_211.parquet M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/chars.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-scanner-position.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch.test A testdata/workloads/functional-query/queries/QueryTest/out-of-range-timestamp-local-tz-conversion.test A testdata/workloads/functional-query/queries/QueryTest/scanners-many-nulls.test M tests/custom_cluster/test_hive_parquet_timestamp_conversion.py M tests/query_test/test_scanners.py 27 files changed, 881 insertions(+), 133 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/8319/20 -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 20 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 15 Nov 2018 18:40:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7367: Pack StringValue and CollectionValue slots
Pooja Nilangekar has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/11599 ) Change subject: IMPALA-7367: Pack StringValue and CollectionValue slots .. IMPALA-7367: Pack StringValue and CollectionValue slots This change packs StringValue and CollectionValue slots to ensure they now occupy 12 bytes instead of 16 bytes. This reduces the memory requirements and improves the performance. Since Kudu tuples are populated using a memcopy, 4 bytes of padding was added to StringSlots in Kudu tables. Testing: Ran core tests. Added static asserts to ensure the value sizes are as expected. Performance tests on TPCH-40 produced 3.96% improvement. Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a --- M be/src/exec/text-converter.inline.h M be/src/exprs/expr-test.cc M be/src/exprs/scalar-expr.cc M be/src/runtime/collection-value.h M be/src/runtime/descriptors.cc M be/src/runtime/string-value.h M be/src/runtime/types.h M be/src/util/static-asserts.cc M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/catalog/PrimitiveType.java M fe/src/main/java/org/apache/impala/catalog/Type.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/fk-pk-join-detection.test M testdata/workloads/functional-planner/queries/PlannerTest/join-order.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering-disabled.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/sort-expr-materialization.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test M testdata/workloads/functional-planner/queries/PlannerTest/tablesample.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-kudu.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test 32 files changed, 1,704 insertions(+), 1,713 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/11599/5 -- To view, visit http://gerrit.cloudera.org:8080/11599 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I32f3b06622c087e4aa288e8db1bf4581b10d386a Gerrit-Change-Number: 11599 Gerrit-PatchSet: 5 Gerrit-Owner: Pooja Nilangekar Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5821: Add query with implicit casts to extended explain output.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11719 ) Change subject: IMPALA-5821: Add query with implicit casts to extended explain output. .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3464/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I55c3bdacc295137f66b2316a912fc347da30d6b0 Gerrit-Change-Number: 11719 Gerrit-PatchSet: 9 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 15 Nov 2018 17:35:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 19: (5 comments) I did a pass out of curiosity and only had a few nits. http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@313 PS19, Line 313: void ReadPositions(int64_t num_to_read, int tuple_size, uint8_t* tuple_mem) RESTRICT; Is there a reason that *tuple_mem is not restricted here? http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@745 PS19, Line 745: return NeedsConversionInline() ? This might be more readable as an if-statement http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/exec/parquet-column-readers.cc@897 PS19, Line 897: // TODO: converting timestamps after validating them can move them out of range. We Do we have a Jira for this already? http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h File be/src/runtime/tuple.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h@55 PS19, Line 55: to nit: extra word http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/util/mem-util.h@34 PS19, Line 34: T* current; can you initialize to nullptr (or add a brief comment why we don't)? -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 19 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 17:21:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 19: Code-Review+1 (1 comment) I read it through again and haven't found anything. I let Csaba give the final +2 http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h File be/src/runtime/tuple.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h@78 PS19, Line 78: /// if the tuple is logically NULL. Thanks for updating this! -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 19 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 14:52:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4123: Columnar decoding in Parquet
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/8319 ) Change subject: IMPALA-4123: Columnar decoding in Parquet .. Patch Set 19: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h File be/src/runtime/tuple.h: http://gerrit.cloudera.org:8080/#/c/8319/19/be/src/runtime/tuple.h@73 PS19, Line 73: to nit: double 'to' http://gerrit.cloudera.org:8080/#/c/8319/17/be/src/util/mem-util.h File be/src/util/mem-util.h: http://gerrit.cloudera.org:8080/#/c/8319/17/be/src/util/mem-util.h@41 PS17, Line 41: // memcpy() is necessary because 'current' may not be aligned. > Ok, wow, that comment is out of date and has some wrong info. Fixed it in t Thanks for improving the comment! -- To view, visit http://gerrit.cloudera.org:8080/8319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8c03006981c46ef0dae30602f2b73c253d9b49ef Gerrit-Change-Number: 8319 Gerrit-PatchSet: 19 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Mostafa Mokhtar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Thu, 15 Nov 2018 13:49:29 + Gerrit-HasComments: Yes
