[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Tamas Mate has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output This change modifies the output of the SHOW TABLE STATS and SHOW PARTITIONS for Kudu tables. - PARTITIONS: the #Row column has been removed - TABLE STATS: instead of showing partition informations it returns a resultset similar to HDFS table stats, #Rows, #Partitions, Size, Format and Location Example outputs can be seen in the doc changes. Testing: kudu_stats.test is modified to verify the new result set kudu_partition_ddl.test is modified to verify the new partitions style Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 --- M docs/topics/impala_compute_stats.xml M docs/topics/impala_show.xml M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/service/Frontend.java M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test 7 files changed, 189 insertions(+), 148 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/15199/7 -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 7 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5481/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 7 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 17 Mar 2020 06:53:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 5: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/565/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 05:15:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 5: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/565/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 05:08:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15457 ) Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/564/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 17 Mar 2020 05:07:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Kristine Hahn has uploaded a new patch set (#5) to the change originally created by Alex Rodoni. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. IMPALA-9029: [DOCS] Impala 3.4 Release Notes -Added broadcast_bytes_limit query option Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc --- M docs/shared/impala_common.xml M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_known_issues.xml M docs/topics/impala_new_features.xml M docs/topics/impala_txtfile.xml 5 files changed, 223 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/14863/5 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15457 ) Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/564/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 17 Mar 2020 04:59:35 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15457 to look at the new patch set (#2). Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 --- M docs/impala.ditamap A docs/topics/impala_broadcast_bytes_limit.xml 2 files changed, 65 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/15457/2 -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 17 Mar 2020 04:55:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. IMPALA-6267: MT scanners check filters per split Refactor the code for checking each scan range against filters so that it can be shared between the MT and non-MT scan node implementations. Move it into StartNextScanRange(), which has the advantage that we can skip issuing then cancelling the I/O for the range. Testing: Added a regression test for the code path that failed for multithreaded scans before this fix. Looped the test for a couple of hours to flush out flakiness. Fix some runtime filter tests where the mt_dop from the dimensions was not applied. Fix test_wait_time_cancellation() to work with mt_dop > 0, where filters are waited for in Open() instead of GetNext(), which means that the query does not get into the RUNNING state while waiting for filters. Instead use the profile to detect that execution started. Ran core tests. Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Reviewed-on: http://gerrit.cloudera.org:8080/15411 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins --- M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/hdfs-scan-node.cc M tests/query_test/test_runtime_filters.py 5 files changed, 84 insertions(+), 28 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15457 ) Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/563/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 17 Mar 2020 04:32:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Kristine Hahn has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15457 Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 --- M docs/impala.ditamap A docs/topics/impala_broadcast_bytes_limit.xml 2 files changed, 52 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/15457/1 -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn
[Impala-ASF-CR] IMPALA-9517: [DOCS] Document broadcast bytes limit query option
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15457 ) Change subject: IMPALA-9517: [DOCS] Document broadcast_bytes_limit query option .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/563/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/15457 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2f7eaacd5a885a7a5292d7694241d58e4f7b6282 Gerrit-Change-Number: 15457 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 17 Mar 2020 04:24:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3343, IMPALA-9489: Make impala-shell compatible with python 3.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15132 ) Change subject: IMPALA-3343, IMPALA-9489: Make impala-shell compatible with python 3. .. Patch Set 19: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5502/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15132 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb75e162bac0faeae3e12106c15da39cbfb8b462 Gerrit-Change-Number: 15132 Gerrit-PatchSet: 19 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 17 Mar 2020 03:16:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9156: share broadcast join builds
Bikramjeet Vig has posted comments on this change. (
http://gerrit.cloudera.org:8080/15096 )
Change subject: IMPALA-9156: share broadcast join builds
..
Patch Set 16: Code-Review+2
(6 comments)
http://gerrit.cloudera.org:8080/#/c/15096/16/be/src/exec/partitioned-hash-join-builder.h
File be/src/exec/partitioned-hash-join-builder.h:
http://gerrit.cloudera.org:8080/#/c/15096/16/be/src/exec/partitioned-hash-join-builder.h@650
PS16, Line 650: anr
nit: and
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/exec/partitioned-hash-join-builder.cc
File be/src/exec/partitioned-hash-join-builder.cc:
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/exec/partitioned-hash-join-builder.cc@708
PS15, Line 708: probe_barrier_->Wait
> maybe add this wait time to the join node's idle/wait time. This would help
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/runtime/runtime-state.cc
File be/src/runtime/runtime-state.cc:
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/runtime/runtime-state.cc@268
PS15, Line 268: for (CyclicBarrier* cb2 : cancellation_cbs_) {
: // Don't add if already present.
: if (cb == cb2) return;
: }
> I figured that this would always be small enough that the linear search wou
makes sense
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/util/cyclic-barrier.h
File be/src/util/cyclic-barrier.h:
http://gerrit.cloudera.org:8080/#/c/15096/15/be/src/util/cyclic-barrier.h@46
PS15, Line 46: template
> This is actually a best practice, cause lambda functions have an "un-nameab
Got it. Thanks for the explanation
http://gerrit.cloudera.org:8080/#/c/15096/16/fe/src/main/java/org/apache/impala/planner/JoinNode.java
File fe/src/main/java/org/apache/impala/planner/JoinNode.java:
http://gerrit.cloudera.org:8080/#/c/15096/16/fe/src/main/java/org/apache/impala/planner/JoinNode.java@193
PS16, Line 193: fragments
nit: fragment
http://gerrit.cloudera.org:8080/#/c/15096/16/tests/query_test/test_spilling.py
File tests/query_test/test_spilling.py:
http://gerrit.cloudera.org:8080/#/c/15096/16/tests/query_test/test_spilling.py@150
PS16, Line 150: splitsb
nit: typo
--
To view, visit http://gerrit.cloudera.org:8080/15096
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4c67e4b2c87ed0fba648f1e1710addb885d66dc7
Gerrit-Change-Number: 15096
Gerrit-PatchSet: 16
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 17 Mar 2020 03:14:17 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9467: [DOCS] live progress enabled by default in interactive mode
Alice Fan has posted comments on this change. ( http://gerrit.cloudera.org:8080/15442 ) Change subject: IMPALA-9467: [DOCS] live_progress enabled by default in interactive mode .. Patch Set 2: Code-Review+1 Thanks Kris! -- To view, visit http://gerrit.cloudera.org:8080/15442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e624b7bb916ecb5aeb4f007c0610807f7b18cf Gerrit-Change-Number: 15442 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Alice Fan Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 17 Mar 2020 02:50:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3343, IMPALA-9489: Make impala-shell compatible with python 3.
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/15132 )
Change subject: IMPALA-3343, IMPALA-9489: Make impala-shell compatible with
python 3.
..
Patch Set 19:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15132/19/bin/impala-shell.sh
File bin/impala-shell.sh:
http://gerrit.cloudera.org:8080/#/c/15132/19/bin/impala-shell.sh@55
PS19, Line 55: PYTHONPATH=${PYTHONPATH} exec "${IMPALA_PYTHON_EXECUTABLE}"
${SHELL_HOME}/impala_shell.py "$@"
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/15132
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb75e162bac0faeae3e12106c15da39cbfb8b462
Gerrit-Change-Number: 15132
Gerrit-PatchSet: 19
Gerrit-Owner: David Knupp
Gerrit-Reviewer: David Knupp
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Tue, 17 Mar 2020 02:32:15 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3343, IMPALA-9489: Make impala-shell compatible with python 3.
David Knupp has uploaded a new patch set (#19). (
http://gerrit.cloudera.org:8080/15132 )
Change subject: IMPALA-3343, IMPALA-9489: Make impala-shell compatible with
python 3.
..
IMPALA-3343, IMPALA-9489: Make impala-shell compatible with python 3.
This patch makes the impala-shell code cross-compatible with python 2 and
python 3. The goal is wind up with a version of the shell that will pass
python e2e tests irrepsective of the version of python used to launch the shell,
under the assumption that the test framework itself will continue to run with
python 2.7.x.
There are a few isolated tests that weren't able to pass under both versions,
and the reasons have been documented in comments in the test themselves.
Notable changes for reviewers to consider:
- With regard to validating the patch, my assumption is that simply passing
the existing set of e2e shell tests is sufficient to confirm that the shell
is functioning properly. No new tests were added.
- Many of the simpler changes derive from the fact that a few built-in functions
and/or types have either been removed or have else changed in python 3.x,
E.g., xrange and basestring are both gone, dict.iteritems() has been removed,
dict.items() behaves differently, the unicode() function and the method
str.decode() have both been removed, etc.
Also, catching exceptions using "Exception, e" no longer works, and (as most
know), using print() as a function is required now.
- A new pytest command line option was added in conftest.py to enable a user
to specify a path to an alternate impala-shell executable to test. It's
possible to use this to point to an instance of the impala-shell that was
installed as a standalone python package in a separate virtualenv.
Example usage:
USE_THRIFT11_GEN_PY=true impala-py.test --shell_executable=//bin/impala-shell -sv shell/test_shell_commandline.py
The target virtualenv may be based on either python3 or python2. However,
this has no effect on the version of python used to run the test framework,
which remains tied to python 2.7.x for the foreseeable future.
- The $IMPALA_HOME/bin/impala-shell.sh now sets up the impala-shell python
environment independenty from bin/set-pythonpath.sh. (See IMPALA-9489)
- thrift_sasl.py was updated to match the current public alpha, 0.4a1
- The wording of the header changed a bit to include the python version
used to run the shell.
Starting Impala Shell with no authentication using Python 3.7.5
Opened TCP connection to localhost:21000
...
OR
Starting Impala Shell with LDAP-based authentication using Python 2.7.12
Opened TCP connection to localhost:21000
...
- By far, the biggest hassle has been juggling str versus unicode versus
bytes data types. Python 2.x was fairly loose and inconsistent in
how it dealt with strings. As a quick demo of what I mean:
Python 2.7.12 (default, Nov 12 2018, 14:36:49)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> d = 'like a duck'
>>> d == str(d) == bytes(d) == unicode(d) == d.encode('utf-8') ==
d.decode('utf-8')
True
...and yet there are weird unexpected gotchas.
>>> d.decode('utf-8') == d.encode('utf-8')
True
>>> d.encode('utf-8') == bytearray(d, 'utf-8')
True
>>> d.decode('utf-8') == bytearray(d, 'utf-8') # fails the eq property?
False
As a result of this, the way we handled strings in the impala-shell code had
become equally loose and inconsistent -- mainly in the form of frequent and
liberal use of str.encode() and str.decode() -- but things still just worked.
In python3, there's a much clearer distinction between strings and bytes, and
as such, much tighter type consistency is expected by standard libs like
subprocess, re, sqlparse, prettytable, etc., which are used throughout the
shell. Even simple calls that worked in python 2.x:
>>> import re
>>> re.findall('foo', b'foobar')
['foo']
...can throw exceptions in python 3.x:
>>> import re
>>> re.findall('foo', b'foobar')
Traceback (most recent call last):
File "", line 1, in
File "/data0/systest/venvs/py3/lib/python3.7/re.py", line 223, in findall
return _compile(pattern, flags).findall(string)
TypeError: cannot use a string pattern on a bytes-like object
Exceptions like this resulted in a many, if not most shell tests failing
under python 3.
At first, I tried to go one-by-one to the site of each failure, and correct
by checking instance type and re-encoding as necessary, but this only led to
even more str.encode() calls littering the code, which just seemed like a
code-smell. (Wiki "code smell" if you don't know the term.)
What ultimately seemed like a better approach was to try to weed out as many
existing spurious str.encode() and str.decode() calls as I could, and try to
implement what is has co
[Impala-ASF-CR] IMPALA-9489: Set impala-shell and impala-python env to use thrift 11
David Knupp has abandoned this change. ( http://gerrit.cloudera.org:8080/15417 ) Change subject: IMPALA-9489: Set impala-shell and impala-python env to use thrift 11 .. Abandoned This is has ben rolled into another patch. -- To view, visit http://gerrit.cloudera.org:8080/15417 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I5b612500e88adf078ef60e4f226a7e4e79f1ec9c Gerrit-Change-Number: 15417 Gerrit-PatchSet: 3 Gerrit-Owner: David Knupp Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 4: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/562/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 02:27:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 4: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/562/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 02:19:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Kristine Hahn has uploaded a new patch set (#4) to the change originally created by Alex Rodoni. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. IMPALA-9029: [DOCS] Impala 3.4 Release Notes Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc --- M docs/shared/impala_common.xml M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_known_issues.xml M docs/topics/impala_new_features.xml M docs/topics/impala_txtfile.xml 5 files changed, 217 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/14863/4 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn
[Impala-ASF-CR] IMPALA-9438 Implement atomic operations for aarch64
huangtianhua...@gmail.com has abandoned this change. ( http://gerrit.cloudera.org:8080/15316 ) Change subject: IMPALA-9438 Implement atomic operations for aarch64 .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/15316 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I84e907c1cd9b09d3329e6c836d492dba5f49f5ae Gerrit-Change-Number: 15316 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 3: Verified-1 Build Failed https://jenkins.impala.io/job/gerrit-docs-auto-test/561/ : Doc tests failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 02:11:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Kristine Hahn has uploaded a new patch set (#3) to the change originally created by Alex Rodoni. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. IMPALA-9029: [DOCS] Impala 3.4 Release Notes Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc --- M docs/shared/impala_common.xml M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_known_issues.xml M docs/topics/impala_new_features.xml M docs/topics/impala_txtfile.xml 5 files changed, 217 insertions(+), 224 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/14863/3 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/561/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 02:04:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8690: Add LIRS cache eviction algorithm
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15306 ) Change subject: IMPALA-8690: Add LIRS cache eviction algorithm .. Patch Set 15: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15306 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I670fa4b2b7c93998130dc4e8b2546bb93e9a84f8 Gerrit-Change-Number: 15306 Gerrit-PatchSet: 15 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 17 Mar 2020 01:43:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 2: Verified-1 Build Failed https://jenkins.impala.io/job/gerrit-docs-auto-test/560/ : Doc tests failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 01:01:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/560/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn Gerrit-Comment-Date: Tue, 17 Mar 2020 00:54:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Kristine Hahn has uploaded a new patch set (#2) to the change originally created by Alex Rodoni. ( http://gerrit.cloudera.org:8080/14863 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. IMPALA-9029: [DOCS] Impala 3.4 Release Notes Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc --- M docs/topics/impala_incompatible_changes.xml M docs/topics/impala_known_issues.xml M docs/topics/impala_new_features.xml 3 files changed, 214 insertions(+), 221 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/14863/2 -- To view, visit http://gerrit.cloudera.org:8080/14863 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4385749de35f8379ecf6566fe515ed500b42d6cc Gerrit-Change-Number: 14863 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Kristine Hahn
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 17 Mar 2020 00:00:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5480/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 17 Mar 2020 00:00:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15451 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 1: Two things: 1. Can we add a line to impala_new_features.xml saying that we added support for ZSTD and DEFLATE compression codecs for text files? 2. From a git perspective, this is using a pattern that I'm not used to. We use cherrypicks rather than merge commands. Here's how I would like the change to proceed: - Cherrypick Alex's original change on top of current master branch (see dropdown "Download" on the right side and copy the cherrypick command). For me the command is: git fetch ssh://joemcdonn...@gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/14863/1 && git cherry-pick FETCH_HEAD - Make your changes on top of it. Do git add / git commit, then "git rebase origin/master -i" and fixup your new commit with Alex's original commit. Reword the commit message if needed. Leave the Change-Id the same. - Then upload that to gerrit with "git push --no-thin asf-gerrit HEAD:refs/for/master". That should do an upload to Alex's original review. If any of that doesn't make sense, reach out to me and I can help you through it. -- To view, visit http://gerrit.cloudera.org:8080/15451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31e9fcc1e1aa98c784b2c597a6df5aeb75be44c5 Gerrit-Change-Number: 15451 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Mon, 16 Mar 2020 22:34:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 16 Mar 2020 21:53:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. Patch Set 6: (4 comments) Thank you for the detailed review. Addressed the comments, let me know if I should add anything else. http://gerrit.cloudera.org:8080/#/c/15199/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15199/6//COMMIT_MSG@9 PS6, Line 9: S > typo Done http://gerrit.cloudera.org:8080/#/c/15199/6/docs/topics/impala_show.xml File docs/topics/impala_show.xml: http://gerrit.cloudera.org:8080/#/c/15199/6/docs/topics/impala_show.xml@987 PS6, Line 987: Impala only computes the number of rows for the whole Kudu table, partition level > Maybe move this down with the 'show partitions' example below Done http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java@102 PS6, Line 102: " must target an HDFS or Kudu " : + "table: " > nit: cleaner to just move the whole string to the next line instead of brea Done http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java File fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java: http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java@136 PS6, Line 136: STRING > Looks like hdfs 'show stats' uses BIGINT for #Rows, lets be consistent with Done -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 16 Mar 2020 21:11:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15451 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/559/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/15451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31e9fcc1e1aa98c784b2c597a6df5aeb75be44c5 Gerrit-Change-Number: 15451 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 16 Mar 2020 20:51:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8690: Add LIRS cache eviction algorithm
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15306 ) Change subject: IMPALA-8690: Add LIRS cache eviction algorithm .. Patch Set 15: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5479/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15306 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I670fa4b2b7c93998130dc4e8b2546bb93e9a84f8 Gerrit-Change-Number: 15306 Gerrit-PatchSet: 15 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 16 Mar 2020 20:48:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15451 ) Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/559/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/15451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31e9fcc1e1aa98c784b2c597a6df5aeb75be44c5 Gerrit-Change-Number: 15451 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 16 Mar 2020 20:43:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9029: [DOCS] Impala 3.4 Release Notes
Kristine Hahn has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15451 Change subject: IMPALA-9029: [DOCS] Impala 3.4 Release Notes .. IMPALA-9029: [DOCS] Impala 3.4 Release Notes - Adds features to the end of the new features list: constraints, enhanced external Kudu table, Ranger column masking, Apachi Hudi support, ORC reads enabled Change-Id: I31e9fcc1e1aa98c784b2c597a6df5aeb75be44c5 --- M docs/topics/impala_new_features.xml 2 files changed, 36 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/15451/1 -- To view, visit http://gerrit.cloudera.org:8080/15451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I31e9fcc1e1aa98c784b2c597a6df5aeb75be44c5 Gerrit-Change-Number: 15451 Gerrit-PatchSet: 1 Gerrit-Owner: Kristine Hahn
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. Patch Set 6: (4 comments) Patch is looking good, just a few more minor things and this can go in http://gerrit.cloudera.org:8080/#/c/15199/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15199/6//COMMIT_MSG@9 PS6, Line 9: S typo http://gerrit.cloudera.org:8080/#/c/15199/6/docs/topics/impala_show.xml File docs/topics/impala_show.xml: http://gerrit.cloudera.org:8080/#/c/15199/6/docs/topics/impala_show.xml@987 PS6, Line 987: Impala only computes the number of rows for the whole Kudu table, partition level Maybe move this down with the 'show partitions' example below http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java File fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java: http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java@102 PS6, Line 102: " must target an HDFS or Kudu " : + "table: " nit: cleaner to just move the whole string to the next line instead of breaking it up into two strings http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java File fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java: http://gerrit.cloudera.org:8080/#/c/15199/6/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java@136 PS6, Line 136: STRING Looks like hdfs 'show stats' uses BIGINT for #Rows, lets be consistent with that, and the same for "#Partitions". -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 16 Mar 2020 18:18:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR](asf-site) IMPALA-9462: Point to downloads.apache.org for current release
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15443 ) Change subject: IMPALA-9462: Point to downloads.apache.org for current release .. Patch Set 1: I was thinking I would update this when I upload the Impala 3.4 artifacts. Impala 3.3 will move to having an archive.apache.org URL and Impala 3.4 will go directly to downloads.apache.org. -- To view, visit http://gerrit.cloudera.org:8080/15443 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: asf-site Gerrit-MessageType: comment Gerrit-Change-Id: Iccc15ef785529d477958012a3f5c83715580a26d Gerrit-Change-Number: 15443 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 16 Mar 2020 17:43:20 + Gerrit-HasComments: No
[Impala-ASF-CR] Update version to 4.0.0-SNAPSHOT
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15449 ) Change subject: Update version to 4.0.0-SNAPSHOT .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5501/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15449 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4fddb36b09f68d7a98950ecb1f4179f089a634dd Gerrit-Change-Number: 15449 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 16 Mar 2020 16:56:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 16 Mar 2020 16:54:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5478/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 16 Mar 2020 16:54:35 + Gerrit-HasComments: No
[Impala-ASF-CR] Update version to 4.0.0-SNAPSHOT
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15449 Change subject: Update version to 4.0.0-SNAPSHOT .. Update version to 4.0.0-SNAPSHOT Change-Id: I4fddb36b09f68d7a98950ecb1f4179f089a634dd --- M bin/save-version.sh 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/15449/1 -- To view, visit http://gerrit.cloudera.org:8080/15449 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4fddb36b09f68d7a98950ecb1f4179f089a634dd Gerrit-Change-Number: 15449 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-9467: [DOCS] live progress enabled by default in interactive mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15442 ) Change subject: IMPALA-9467: [DOCS] live_progress enabled by default in interactive mode .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/558/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/15442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e624b7bb916ecb5aeb4f007c0610807f7b18cf Gerrit-Change-Number: 15442 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Alice Fan Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 16 Mar 2020 15:53:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6267: MT scanners check filters per split
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/15411 ) Change subject: IMPALA-6267: MT scanners check filters per split .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15411 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic40eb4cb2419393e6f7cd7bd019add9224946c4d Gerrit-Change-Number: 15411 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 16 Mar 2020 15:49:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9467: [DOCS] live progress enabled by default in interactive mode
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15442 ) Change subject: IMPALA-9467: [DOCS] live_progress enabled by default in interactive mode .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/558/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/15442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e624b7bb916ecb5aeb4f007c0610807f7b18cf Gerrit-Change-Number: 15442 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Alice Fan Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 16 Mar 2020 15:45:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9467: [DOCS] live progress enabled by default in interactive mode
Hello Tim Armstrong, Alice Fan, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/15442 to look at the new patch set (#2). Change subject: IMPALA-9467: [DOCS] live_progress enabled by default in interactive mode .. IMPALA-9467: [DOCS] live_progress enabled by default in interactive mode The following documents were impacted by the change: - impala_live_progress.xml, revised to explain new behavior - impala_shell_options.xml, added --disable_live_progress option Change-Id: I94e624b7bb916ecb5aeb4f007c0610807f7b18cf --- M docs/shared/impala_common.xml M docs/topics/impala_live_progress.xml M docs/topics/impala_shell_options.xml 3 files changed, 33 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/15442/2 -- To view, visit http://gerrit.cloudera.org:8080/15442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94e624b7bb916ecb5aeb4f007c0610807f7b18cf Gerrit-Change-Number: 15442 Gerrit-PatchSet: 2 Gerrit-Owner: Kristine Hahn Gerrit-Reviewer: Alice Fan Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. Patch Set 15: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5500/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 15 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 16 Mar 2020 15:38:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#15). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 62 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/15 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 15 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Gabor Kaszab has posted comments on this change. (
http://gerrit.cloudera.org:8080/15288 )
Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by
default
..
Patch Set 14:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@113
PS14, Line 113: Create a long select query then check if it is contained in the
response json.
nit: It would be more beneficial to describe the expected functionality here
like: Check if the full query string is displayed in the query list on the
WebUI.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@115
PS14, Line 115: "x " * 450
As this string is used multiple times I'd put this into a variable.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@115
PS14, Line 115: "select \"{0}\""
Might not work but could you try to put the string within single quotes instead
of double quotes so that you don't have to escape the double quotes inside?
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@116
PS14, Line 116: theere
typo
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@118
PS14, Line 118: expected = "select \\\"{0}\\\"".format("x " * 450)
Same comment for single quotes as above.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@129
PS14, Line 129: """Create a long select query then check if it is contained
in the response json."""
This statements isn't true for this test. Might be a copy-paste error.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@131
PS14, Line 131: query = "select \"{0}\"".format("x " * 450)
Same comment for single quotes as above.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@133
PS14, Line 133: theere
typo
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@135
PS14, Line 135: expected = "select \\\"x ..."
Same comment for single quotes as above.
http://gerrit.cloudera.org:8080/#/c/15288/14/tests/custom_cluster/test_web_pages.py@139
PS14, Line 139: if unexpected in response_json:
: assert False, "The response is containing the full query."
Is this needed? L141 will check the expected output anyway that won't be a
prefix of the input because of the "..." appended to the end.
--
To view, visit http://gerrit.cloudera.org:8080/15288
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605
Gerrit-Change-Number: 15288
Gerrit-PatchSet: 14
Gerrit-Owner: Adam Tamas
Gerrit-Reviewer: Adam Tamas
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Norbert Luksa
Gerrit-Reviewer: Tamas Mate
Gerrit-Comment-Date: Mon, 16 Mar 2020 12:55:45 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5499/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 14 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 16 Mar 2020 11:55:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5498/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 13 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 16 Mar 2020 11:43:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 63 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/14 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 14 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. Patch Set 13: (1 comment) http://gerrit.cloudera.org:8080/#/c/15288/13/tests/custom_cluster/test_web_pages.py File tests/custom_cluster/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/15288/13/tests/custom_cluster/test_web_pages.py@141 PS13, Line 141: flake8: W292 no newline at end of file -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 13 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate Gerrit-Comment-Date: Mon, 16 Mar 2020 10:59:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6360: Don't show full query statement on Impala WebUI by default
Adam Tamas has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/15288 ) Change subject: IMPALA-6360: Don't show full query statement on Impala WebUI by default .. IMPALA-6360: Don't show full query statement on Impala WebUI by default In the WebUI's query list the query statements are trimmed, but the full query statement can be seen in the details page. The default statement length is 250 chars and it can be adjusted by the query_stmt_size flag that can be set when the cluster starts. Example: bin/start-impala-cluster.py -s1 --impalad_args --query_stmt_size=10 Testing: -manual testing in the WebUI. -added 'test_query_stmt()' to test_web_pages.py -added test to the custom cluster webserver tests to check without truncate and with custom length truncate Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 --- M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M tests/custom_cluster/test_web_pages.py M tests/webserver/test_web_pages.py M www/queries.tmpl 5 files changed, 63 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/88/15288/13 -- To view, visit http://gerrit.cloudera.org:8080/15288 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib7109a0be5d1022b4f8d6e72441cf5dc1dc42605 Gerrit-Change-Number: 15288 Gerrit-PatchSet: 13 Gerrit-Owner: Adam Tamas Gerrit-Reviewer: Adam Tamas Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Norbert Luksa Gerrit-Reviewer: Tamas Mate
[Impala-ASF-CR] IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output
Tamas Mate has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/15199 ) Change subject: IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output .. IMPALA-5308: Resolve confusing Kudu SHOW TABLE STATS output This change modifies the output of the SHOW STABLE STATS and SHOW PARTITIONS for Kudu tables. - PARTITIONS: the #Row column has been removed - TABLE STATS: instead of showing partition informations it returns a resultset similar to HDFS table stats, #Rows, #Partitions, Size, Format and Location Example outputs can be seen in the doc changes. Testing: kudu_stats.test is modified to verify the new result set kudu_partition_ddl.test is modified to verify the new partitions style Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 --- M docs/topics/impala_compute_stats.xml M docs/topics/impala_show.xml M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/service/Frontend.java M testdata/workloads/functional-query/queries/QueryTest/kudu_partition_ddl.test M testdata/workloads/functional-query/queries/QueryTest/kudu_stats.test 7 files changed, 186 insertions(+), 145 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/15199/6 -- To view, visit http://gerrit.cloudera.org:8080/15199 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ice4b8df65f0a53fe14b8fbe35d82c9887ab9a041 Gerrit-Change-Number: 15199 Gerrit-PatchSet: 6 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-6505: Min-Max predicate push down in ORC scanner
Quanlong Huang has posted comments on this change. (
http://gerrit.cloudera.org:8080/15403 )
Change subject: IMPALA-6505: Min-Max predicate push down in ORC scanner
..
Patch Set 1:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/15403/1/be/src/exec/hdfs-orc-scanner.cc
File be/src/exec/hdfs-orc-scanner.cc:
http://gerrit.cloudera.org:8080/#/c/15403/1/be/src/exec/hdfs-orc-scanner.cc@875
PS1, Line 875: UtcToUnixTime
> According to ORC spec the timestamp stats are stored as millisecs (which is
Looks like it's an ORC bug that the ORC writers (both Java and C++ versions)
don't handle the rounding issue of max:
https://github.com/apache/orc/blob/fea154436c37c81a16b13d879b510096cfaa2946/java/core/src/java/org/apache/orc/impl/writer/TimestampTreeWriter.java#L108
https://github.com/apache/orc/blob/fea154436c37c81a16b13d879b510096cfaa2946/c%2B%2B/src/ColumnWriter.cc#L1800
http://gerrit.cloudera.org:8080/#/c/15403/1/be/src/exec/hdfs-orc-scanner.cc@902
PS1, Line 902: type.GetByteSize()
I think this should be "literal_expr->type().GetByteSize()" since it's
consistent with the type of "val".
http://gerrit.cloudera.org:8080/#/c/15403/1/be/src/exec/hdfs-orc-scanner.cc@914
PS1, Line 914: static_cast((dv16->value() << 64) >> 64)
Can we cast int128_t to uint64_t directly?
http://gerrit.cloudera.org:8080/#/c/15403/1/be/src/exec/hdfs-orc-scanner.cc@984
PS1, Line 984:
row_reader_options_.setSearchArgument(std::move(final_sarg));
Could you add a VLOG_FILE level logging for 'final_sarg'? It would be helpful
for debugging.
http://gerrit.cloudera.org:8080/#/c/15403/1/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/15403/1/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@543
PS1, Line 543: } else if (op == BinaryPredicate.Operator.EQ) {
ORC lib supports pushing down EQ predicates. I think we don't need to transform
them into LE+GE predicates. But I'm ok leaving this as a further optimization.
--
To view, visit http://gerrit.cloudera.org:8080/15403
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I136622413db21e0941d238ab6aeea901a6464845
Gerrit-Change-Number: 15403
Gerrit-PatchSet: 1
Gerrit-Owner: Norbert Luksa
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Norbert Luksa
Gerrit-Reviewer: Quanlong Huang
Gerrit-Comment-Date: Mon, 16 Mar 2020 08:17:50 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9398: Fix shell history duplication when cmdloop breaks
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15345 ) Change subject: IMPALA-9398: Fix shell history duplication when cmdloop breaks .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5497/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15345 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If4faf46134f44d91e56748642f47d448707db53c Gerrit-Change-Number: 15345 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 16 Mar 2020 08:07:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9398: Fix shell history duplication when cmdloop breaks
Tamas Mate has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/15345 ) Change subject: IMPALA-9398: Fix shell history duplication when cmdloop breaks .. IMPALA-9398: Fix shell history duplication when cmdloop breaks This change adds a new condition to avoid re-reading the impala-shell history when the cmdloop is broken. The loop can break due to exceptions such as KeyboardInterrupt. Testing: - The change was tested manually on local dev env - Added a new EE shell test to verify the history after SIGINT Change-Id: If4faf46134f44d91e56748642f47d448707db53c --- M shell/impala_shell.py M tests/shell/test_shell_interactive.py 2 files changed, 39 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/15345/2 -- To view, visit http://gerrit.cloudera.org:8080/15345 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If4faf46134f44d91e56748642f47d448707db53c Gerrit-Change-Number: 15345 Gerrit-PatchSet: 2 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Fang-Yu Rao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tamas Mate Gerrit-Reviewer: Vincent Tran
