[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13665 ) Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 12 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 08 Jul 2019 22:52:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8744: Fix test session expiration.py to work with Python 2.6
Michael Ho has uploaded this change for review. (
http://gerrit.cloudera.org:8080/13819
Change subject: IMPALA-8744: Fix test_session_expiration.py to work with Python
2.6
..
IMPALA-8744: Fix test_session_expiration.py to work with Python 2.6
This patches fixes a formatting string to use "{0}" rather than "{}"
as Python 2.6 doesn't support "{}"
Testing done:
- Ran the test on Centos6 platform with Python 2.6
Change-Id: I3a914a9ca881181d47bee68641b9c7affbfc0d54
---
M tests/custom_cluster/test_session_expiration.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/13819/1
--
To view, visit http://gerrit.cloudera.org:8080/13819
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3a914a9ca881181d47bee68641b9c7affbfc0d54
Gerrit-Change-Number: 13819
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Ho
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13672 ) Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3843/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 04:34:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4585/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:05:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 3 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:05:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Michael Ho has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() This change fixes some callers of RpcMgr::GetProxy() tp pass the hostname instead of the resolved IP adddress. A new DCHECK is added to make sure calleres are not passing the resolved IP address as hostname. The hostname passed to RpcMgr::GetProxy() is used to construct the principal name of the remote destination when Kerberos is enabled. Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 --- M be/src/rpc/rpc-mgr.h M be/src/runtime/coordinator-backend-state.cc M be/src/service/client-request-state.cc 3 files changed, 13 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/13818/2 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 2: Code-Review+2 Carry Todd's +2 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:04:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8729: [DOCS] Describe on-demand metadata feature
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/13802 ) Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature .. Patch Set 2: Code-Review+2 (4 comments) http://gerrit.cloudera.org:8080/#/c/13802/2/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/13802/2/docs/topics/impala_metadata.xml@52 PS2, Line 52: coordinators nit: coordinator http://gerrit.cloudera.org:8080/#/c/13802/2/docs/topics/impala_metadata.xml@60 PS2, Line 60: new version of metadata I think calling it new version of metadata is confusing, may be say, with this new feature or something along those lines? http://gerrit.cloudera.org:8080/#/c/13802/2/docs/topics/impala_metadata.xml@60 PS2, Line 60: of impaladhttp://gerrit.cloudera.org:8080/#/c/13802/2/docs/topics/impala_metadata.xml@71 PS2, Line 71: : This is a preview feature in and not generally : available. IMO, we could get rid of this. -- To view, visit http://gerrit.cloudera.org:8080/13802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Gerrit-Change-Number: 13802 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 09 Jul 2019 05:04:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13665 ) Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories .. Patch Set 13: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 13 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Tue, 09 Jul 2019 05:04:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Hello Andrew Sherman, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13672 to look at the new patch set (#4). Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. IMPALA-8584: Add cookie support to the HTTP HS2 server This patch modifies the HTTP HS2 server to accept cookies for authentication in order to avoid having to authenticate every request through LDAP. It adds a flag, --max_cookie_lifetime_s, that determines how long generated cookies are valid for. Setting the flag to 0 disables cookie support. The cookies include a SHA256 HMAC signature that it used to verify them. They also have a timestamp that is used to determine if they have expired. If a cookie is successfully verified and hasn't expired, the username contained in the cookie is set on the connection. Each impalad uses its own key to generate the signature, so clients that reconnect to a different impalad will have to reauthenticate. On a single impalad cookies are valid across sessions and connections. A new cookie is generated and sent back with the Set-Cookie header on each request that was not authenticated with a cookie. Cookies are of the form: impala.hs2.auth=;HttpOnly;MaxAge= where: cookie = &&& and 'signature' is the SHA256 HMAC of the rest of the cookie The 'Secure' flag, which indicates to clients that the cookie should only be sent over secure connections, is omitted if '--ldap_passwords_in_clear_ok' is true. This is intended only for testing. It also adds the metrics: impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-success impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-failure Testing: - Added tests to the FE LDAP tests that use the metrics to verify successful and failed cookie attempts. Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 --- M be/src/rpc/CMakeLists.txt M be/src/rpc/TAcceptQueueServer.cpp M be/src/rpc/auth-provider.h M be/src/rpc/authentication.cc A be/src/rpc/cookie-util.cc A be/src/rpc/cookie-util.h M be/src/rpc/thrift-server.cc M be/src/rpc/thrift-server.h M be/src/service/impala-hs2-server.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/transport/THttpServer.cpp M be/src/transport/THttpServer.h M be/src/util/openssl-util.cc M be/src/util/openssl-util.h M common/thrift/metrics.json M fe/src/test/java/org/apache/impala/customcluster/LdapHS2Test.java M fe/src/test/java/org/apache/impala/customcluster/LdapJdbcTest.java M fe/src/test/java/org/apache/impala/service/JdbcTestBase.java 19 files changed, 517 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/13672/4 -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 1: Code-Review+2 (1 comment) +2 pending one issue with the DCHECK http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.inline.h File be/src/rpc/rpc-mgr.inline.h: http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.inline.h@44 PS1, Line 44: DCHECK(!IsResolvedAddress(MakeNetworkAddress(hostname))) << hostname << "\n" I don't think this DCHECK is valid -- it's OK to run things in an environment without DNS, if you give the hosts principals like impala/1.2.3.4. -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 03:33:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h File be/src/rpc/rpc-mgr.h: http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h@148 PS1, Line 148: has to match > if Kerberos is enabled Done http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h@149 PS1, Line 149: f Kerberos is enabled. 'P' must descend fr > The "usually" here seems to suggest that the DCHECK could fail in some case Removed in PS2. Please see replies in rpc-mgr.inline.h http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.inline.h File be/src/rpc/rpc-mgr.inline.h: http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.inline.h@44 PS1, Line 44: // Talk to self via loopback. > I don't think this DCHECK is valid -- it's OK to run things in an environme Yes, was having a hard time coming up with a DCHECK which works for all cases. I suspect Impala may fail elsewhere if DNS is disabled but there is nothing from stopping a user from configuring the hostname as 1.2.3.4 for whatever reason. Anyhow, removing it for now. FWIW, I confirmed that this DCHECK actually catches the bug which this patch is trying to fix. -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:03:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] Changed the example for the INITCAP function
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13816 Change subject: [DOCS] Changed the example for the INITCAP function .. [DOCS] Changed the example for the INITCAP function Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf --- M docs/topics/impala_string_functions.xml 1 file changed, 5 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/13816/1 -- To view, visit http://gerrit.cloudera.org:8080/13816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Gerrit-Change-Number: 13816 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] [DOCS] Changed the example for the INITCAP function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13816 ) Change subject: [DOCS] Changed the example for the INITCAP function .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/382/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Gerrit-Change-Number: 13816 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 23:25:24 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Changed the example for the INITCAP function
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13816 ) Change subject: [DOCS] Changed the example for the INITCAP function .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Gerrit-Change-Number: 13816 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 23:25:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13665 ) Change subject: IMPALA-8663 : FileMetadataLoader should skip hidden and tmp directories .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4584/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c4a22908304fe9e377d77d6c18d401c3f3294aa Gerrit-Change-Number: 13665 Gerrit-PatchSet: 13 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Mon, 08 Jul 2019 23:25:34 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Changed the example for the INITCAP function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13816 ) Change subject: [DOCS] Changed the example for the INITCAP function .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/382/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Gerrit-Change-Number: 13816 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 23:35:55 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Changed the example for the INITCAP function
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13816 ) Change subject: [DOCS] Changed the example for the INITCAP function .. [DOCS] Changed the example for the INITCAP function Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Reviewed-on: http://gerrit.cloudera.org:8080/13816 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_string_functions.xml 1 file changed, 5 insertions(+), 2 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic957e71afcde23952f1751e66203ad9992dbadaf Gerrit-Change-Number: 13816 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13672 ) Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. Patch Set 3: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3841/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 02:02:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3840/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 02:03:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8747: Fix string formatting on HS2 connection setup failure
Joe McDonnell has uploaded this change for review. (
http://gerrit.cloudera.org:8080/13815
Change subject: IMPALA-8747: Fix string formatting on HS2 connection setup
failure
..
IMPALA-8747: Fix string formatting on HS2 connection setup failure
In impala_test_suite.py, the logging for HS2 connection failure
uses "{0}" replacement codes when the logging facility expects it
to use "%"-style codes. This fixes it to call .format() directly.
Change-Id: I4789712fcd71c887a9c6c890cd22a4f382faa3a5
---
M tests/common/impala_test_suite.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/15/13815/1
--
To view, visit http://gerrit.cloudera.org:8080/13815
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4789712fcd71c887a9c6c890cd22a4f382faa3a5
Gerrit-Change-Number: 13815
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-8747: Fix string formatting on HS2 connection setup failure
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13815 ) Change subject: IMPALA-8747: Fix string formatting on HS2 connection setup failure .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3839/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13815 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4789712fcd71c887a9c6c890cd22a4f382faa3a5 Gerrit-Change-Number: 13815 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 23:06:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8543: More diagnostics for TAcceptQueueServer
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13790 ) Change subject: IMPALA-8543: More diagnostics for TAcceptQueueServer .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.cpp File be/src/rpc/TAcceptQueueServer.cpp: http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.cpp@406 PS1, Line 406: 5 * 60 * MICROS_PER_SEC, 1 maybe label these, eg: /* highest_trackable_value */ http://gerrit.cloudera.org:8080/#/c/13790/1/common/thrift/metrics.json File common/thrift/metrics.json: http://gerrit.cloudera.org:8080/#/c/13790/1/common/thrift/metrics.json@955 PS1, Line 955: connection > client connection? here and elsewhere In fact, rather than having the label always match the description on these, it might be helpful to be a little more explicit, eg: "Stats on the amount of time client connections to the Beeswax API spend waiting for the connection to be setup" or similar -- To view, visit http://gerrit.cloudera.org:8080/13790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36 Gerrit-Change-Number: 13790 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 09 Jul 2019 00:18:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Hello Andrew Sherman, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13672 to look at the new patch set (#3). Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server .. IMPALA-8584: Add cookie support to the HTTP HS2 server This patch modifies the HTTP HS2 server to accept cookies for authentication in order to avoid having to authenticate every request through LDAP. It adds a flag, --max_cookie_lifetime_s, that determines how long generated cookies are valid for. Setting the flag to 0 disables cookie support. The cookies include a SHA256 HMAC signature that it used to verify them. They also have a timestamp that is used to determine if they have expired. If a cookie is successfully verified and hasn't expired, the username contained in the cookie is set on the connection. Each impalad uses its own key to generate the signature, so clients that reconnect to a different impalad will have to reauthenticate. On a single impalad cookies are valid across sessions and connections. A new cookie is generated and sent back with the Set-Cookie header on each request that was not authenticated with a cookie. Cookies are of the form: impala.hs2.auth=;HttpOnly;MaxAge= where: cookie = &&& and 'signature' is the SHA256 HMAC of the rest of the cookie The 'Secure' flag, which indicates to clients that the cookie should only be sent over secure connections, is omitted if '--ldap_passwords_in_clear_ok' is true. This is intended only for testing. It also adds the metrics: impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-success impala.thrift-server.hiveserver2-http-frontend.total-cookie-auth-failure Testing: - Added tests to the FE LDAP tests that use the metrics to verify successful and failed cookie attempts. Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 --- M be/src/rpc/CMakeLists.txt M be/src/rpc/TAcceptQueueServer.cpp M be/src/rpc/auth-provider.h M be/src/rpc/authentication.cc A be/src/rpc/cookie-util.cc A be/src/rpc/cookie-util.h M be/src/rpc/thrift-server.cc M be/src/rpc/thrift-server.h M be/src/service/impala-hs2-server.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/transport/THttpServer.cpp M be/src/transport/THttpServer.h M be/src/util/openssl-util.cc M be/src/util/openssl-util.h M common/thrift/metrics.json M fe/src/test/java/org/apache/impala/customcluster/LdapHS2Test.java M fe/src/test/java/org/apache/impala/customcluster/LdapJdbcTest.java M fe/src/test/java/org/apache/impala/service/JdbcTestBase.java 19 files changed, 517 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/13672/3 -- To view, visit http://gerrit.cloudera.org:8080/13672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4 Gerrit-Change-Number: 13672 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Tauber-Marshall Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8584: Add cookie support to the HTTP HS2 server
Thomas Tauber-Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/13672 )
Change subject: IMPALA-8584: Add cookie support to the HTTP HS2 server
..
Patch Set 3:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/13672/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13672/2//COMMIT_MSG@7
PS2, Line 7: IMPALA-8584: Add cookie support to the HTTP HS2 server
> Some questions about the design here:
It turns out that Hive doesn't store the cookies, it just uses a SHA256 HMAC,
as you suggested. This approach has various advantages, so I switched to that.
This solves your first concern - each client that authenticates without using a
cookie with get its own cookie generated and sent back to it, and cookies
aren't tied to sessions or connections.
For your second question, I think that people generally use sticky sessions.
The hs2 sessions themselves aren't shared via the statestore, so reconnecting
to a new impalad requires calling OpenSession(). So, my patch doesn't share the
key. Of course, if we think sharing the key is an important optimization it
would be straightforward to add.
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/authentication.cc@546
PS2, Line 546: ,
> Any idea what the source of this entropy is? Is it subject to entropy delet
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/authentication.cc@562
PS2, Line 562: GENERAL_CALLBACKS[0].id = SASL_CB_LOG;
> Do we want any expiration on the cookie? Or is the default (session cookie)
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.h
File be/src/rpc/cookie-mgr.h:
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.h@56
PS2, Line 56:
> nit: the cookie
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.h@69
PS2, Line 69:
> nit: spelling -> cookies
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.cc
File be/src/rpc/cookie-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.cc@54
PS2, Line 54:
> Should this be * 100 ?
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/rpc/cookie-mgr.cc@61
PS2, Line 61:
> hrm, this is odd. YOu can't just do .emplace(username, Cookie(cookie, UnixM
Done
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/transport/THttpServer.cpp
File be/src/transport/THttpServer.cpp:
http://gerrit.cloudera.org:8080/#/c/13672/2/be/src/transport/THttpServer.cpp@104
PS2, Line 104: } else if (THRIFT_strncasecmp(header, "Cookie", sz) == 0 &&
useCookies_) {
> the Cookie header can pass multiple cookies, separated by ';'. It doesn't s
Done
--
To view, visit http://gerrit.cloudera.org:8080/13672
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I647c06f94ef91aa3b6413e91576c4ec506ed57f4
Gerrit-Change-Number: 13672
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Tue, 09 Jul 2019 01:22:33 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8744: Fix test session expiration.py to work with Python 2.6
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13819 ) Change subject: IMPALA-8744: Fix test_session_expiration.py to work with Python 2.6 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3842/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13819 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a914a9ca881181d47bee68641b9c7affbfc0d54 Gerrit-Change-Number: 13819 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 09 Jul 2019 02:23:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h File be/src/rpc/rpc-mgr.h: http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h@148 PS1, Line 148: has to match if Kerberos is enabled http://gerrit.cloudera.org:8080/#/c/13818/1/be/src/rpc/rpc-mgr.h@149 PS1, Line 149: which usually is not a resolved IP address The "usually" here seems to suggest that the DCHECK could fail in some cases? -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 03:37:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:07:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:07:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4586/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:07:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Hello Thomas Tauber-Marshall, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13818 to look at the new patch set (#4). Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() This change fixes some callers of RpcMgr::GetProxy() tp pass the hostname instead of the resolved IP adddress. The hostname passed to RpcMgr::GetProxy() is used to construct the principal name of the remote destination when Kerberos is enabled. Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 --- M be/src/rpc/rpc-mgr.h M be/src/runtime/coordinator-backend-state.cc M be/src/service/client-request-state.cc 3 files changed, 13 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/13818/4 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8732: Use a serialized descriptor table in TQueryCtx
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/13772 ) Change subject: IMPALA-8732: Use a serialized descriptor table in TQueryCtx .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/coordinator-backend-state.cc File be/src/runtime/coordinator-backend-state.cc: http://gerrit.cloudera.org:8080/#/c/13772/1/be/src/runtime/coordinator-backend-state.cc@200 PS1, Line 200: sidecar.__set_query_ctx(query_ctx()); > Makes me wonder if we should factor out the descriptor table from query_ctx After reading the code some more, the entire query_ctx() is shared across all executors, so we should be able to put that into its own sidecar. There is some code that relies on being able to read the thrift structures directly (e.g. RpcMgr::DoRpcWithRetry(), which is called from Coordinator::BackendState::Cancel()). I'm looking into whether there is a simpler fix that just serializes TQueryCtx. The DescriptorTable is definitely the piece that can grow largest. -- To view, visit http://gerrit.cloudera.org:8080/13772 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I458aa62dd4d1e4e4a7b1869a604623a69f3b2d9a Gerrit-Change-Number: 13772 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Tue, 09 Jul 2019 00:59:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13818 Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() This change fixes some callers of RpcMgr::GetProxy() tp pass the hostname instead of the resolved IP adddress. A new DCHECK is added to make sure calleres are not passing the resolved IP address as hostname. The hostname passed to RpcMgr::GetProxy() is used to construct the principal name of the remote destination when Kerberos is enabled. Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 --- M be/src/rpc/rpc-mgr.h M be/src/rpc/rpc-mgr.inline.h M be/src/runtime/coordinator-backend-state.cc M be/src/service/client-request-state.cc 4 files changed, 17 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/18/13818/1 -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3844/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:43:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8748: Always pass hostname to RpcMgr::GetProxy()
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13818 ) Change subject: IMPALA-8748: Always pass hostname to RpcMgr::GetProxy() .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3845/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I85b661c8c3b3b67bfc1ce9e29aecb90a862666c0 Gerrit-Change-Number: 13818 Gerrit-PatchSet: 4 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Tue, 09 Jul 2019 05:46:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8710: Increase allowed bit width to 64 for bit packing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13809 ) Change subject: IMPALA-8710: Increase allowed bit width to 64 for bit packing .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3835/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b Gerrit-Change-Number: 13809 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 13:50:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Hello Vihang Karajgaonkar, Sudhanshu Arora, Zoltan Borok-Nagy, Sahil Takiar, Todd Lipcon, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13558 to look at the new patch set (#5). Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. IMPALA-8593: Prohibit write operations for bucketed tables This patch adds a method to check if a table bucketed. For Hive 3, integrates with HMS translation layer for capabilities checks. Implements methods ensureTableWriteSupported and ensureTableReadSupported. Set default capabilities for tables. Tests: Added unit tests to ParserTest and AnalyzerTest. Added bucketed tables which are required by IMPALA-8439. Ran core tests(Hive 2 and Hive 3) ToDo: Integrate checking bucketed tables capabilities and creating error messages with HMS translation after Hive provides the required functions. Enable capabilities checking for Kudu tables. Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/catalog/MetaStoreClientPool.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M tests/metadata/test_ddl.py M tests/metadata/test_show_create_table.py 19 files changed, 390 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/13558/5 -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Yongzhi Chen has posted comments on this change. (
http://gerrit.cloudera.org:8080/13558 )
Change subject: IMPALA-8593: Prohibit write operations for bucketed tables
..
Patch Set 5:
Ran Hive 2 and Hive 3 core tests. For hive 3 tests, there are 4 failures and
are the same failures as upstream master :
Test Result (4 failures / -7)
custom_cluster.test_hive_parquet_codec_interop.TestParquetInterop.test_hive_impala_interop[protocol:
beeswax | exec_option: {'sync_ddl': 1, 'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none]
generate_junitxml.buildall.run-custom-cluster-tests
org.apache.impala.catalog.events.MetastoreEventsProcessorTest.testInsertEventOnRemovedTable
org.apache.impala.catalog.events.MetastoreEventsProcessorTest.testInsertEvents
https://master-02.jenkins.cloudera.com/job/impala-private-parameterized/5367/#showFailuresLink
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/20/#showFailuresLink
So there is no regression.
Because of recent cdp bumping from 107 to 195, Impala has to add capabilities
when creating tables. Most impala created table have OBJCAPABILITIES except for
kudu tables (seems that kudu managed table does not store extra table
properties). Therefore, kudu table will not check capabilities for now.
--
To view, visit http://gerrit.cloudera.org:8080/13558
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec
Gerrit-Change-Number: 13558
Gerrit-PatchSet: 5
Gerrit-Owner: Yongzhi Chen
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Sudhanshu Arora
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vihang Karajgaonkar
Gerrit-Reviewer: Yongzhi Chen
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Mon, 08 Jul 2019 13:48:11 +
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6663 Expose current DDL metrics on WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13806 ) Change subject: IMPALA-6663 Expose current DDL metrics on WebUI .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/13806/3/fe/src/main/java/org/apache/impala/catalog/Table.java File fe/src/main/java/org/apache/impala/catalog/Table.java: http://gerrit.cloudera.org:8080/#/c/13806/3/fe/src/main/java/org/apache/impala/catalog/Table.java@186 PS3, Line 186: CatalogUsageMonitor.INSTANCE.getCatalogTableMetrics().updateFrequentlyAccessedTables(this); line too long (97 > 90) http://gerrit.cloudera.org:8080/#/c/13806/3/fe/src/main/java/org/apache/impala/catalog/Table.java@193 PS3, Line 193: CatalogUsageMonitor.INSTANCE.getCatalogTableMetrics().updateHighFileCountTables(this); line too long (92 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13806 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a Gerrit-Change-Number: 13806 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 13:52:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6663 Expose current DDL metrics on WebUI
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13806 Change subject: IMPALA-6663 Expose current DDL metrics on WebUI .. IMPALA-6663 Expose current DDL metrics on WebUI This change adds two new tables to the Catalog WebUI's '/catalog' page. These tables are exposing the number of running catalog operations, which helps to better understand the catalog load and reason when the catalog is overloaded. A new 'monitor' package is created to collect the monitoring sub-classes. The new counters are called just after JNI methods are invoked, so it can notice the exceptions. The existing thrift structure is extended with the new counters, these values are unwrapped and displayed on the WebUI. Before/after pictures of the /catalog page can be found in the Jira. Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a --- M be/src/catalog/catalog-server.cc M common/thrift/JniCatalog.thrift M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Table.java A fe/src/main/java/org/apache/impala/catalog/monitor/CatalogOpRequestCounter.java R fe/src/main/java/org/apache/impala/catalog/monitor/CatalogTableMetrics.java A fe/src/main/java/org/apache/impala/catalog/monitor/CatalogUsageMonitor.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M www/catalog.tmpl 11 files changed, 360 insertions(+), 26 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/13806/3 -- To view, visit http://gerrit.cloudera.org:8080/13806 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a Gerrit-Change-Number: 13806 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate
[Impala-ASF-CR] IMPALA-6663 Expose current DDL metrics on WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13806 ) Change subject: IMPALA-6663 Expose current DDL metrics on WebUI .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3837/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13806 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ed76f134bad6d3b3d4dce132365a53a01e9512a Gerrit-Change-Number: 13806 Gerrit-PatchSet: 3 Gerrit-Owner: Tamas Mate Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 14:32:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/13558/5/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/13558/5/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@20 PS5, Line 20: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_NONE; line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13558/5/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@21 PS5, Line 21: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_READONLY; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/13558/5/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@22 PS5, Line 22: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_READWRITE; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/13558/5/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@23 PS5, Line 23: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_WRITEONLY; line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 08 Jul 2019 13:34:27 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3836/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 08 Jul 2019 14:14:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8710: Increase allowed bit width to 64 for bit packing
Daniel Becker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13809 Change subject: IMPALA-8710: Increase allowed bit width to 64 for bit packing .. IMPALA-8710: Increase allowed bit width to 64 for bit packing Fixed the buffer overflow that the previous attempt introduced. Increasing the allowed bit width for bit packing and bit unpacking to 64 bits. This will be needed to support the Parquet delta encoding. Added new methods to BitWriter and BatchedBitReader handling Uleb and ZigZag integers for 64 bits, also needed by delta encoding. The performance of bit unpacking is either the same or better with the new implementation, depending on bit width. Updated the results in bit-packing-benchmark.cc. Testing: - Modified bit packing and unpacking tests to test bit widths up to 64 bits. - Tests covering the additions in BitWriter and BatchedBitReader. Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b --- M be/src/benchmarks/bit-packing-benchmark.cc M be/src/util/bit-packing.inline.h 2 files changed, 188 insertions(+), 171 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/13809/1 -- To view, visit http://gerrit.cloudera.org:8080/13809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b Gerrit-Change-Number: 13809 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Becker
[Impala-ASF-CR] IMPALA-8710: Increase allowed bit width to 64 for bit packing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13809 ) Change subject: IMPALA-8710: Increase allowed bit width to 64 for bit packing .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3832/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b Gerrit-Change-Number: 13809 Gerrit-PatchSet: 1 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 10:20:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3834/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 14 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 08 Jul 2019 10:36:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Hello Bharath Vissapragada, Fredy Wijaya, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13748
to look at the new patch set (#14).
Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution
..
IMPALA-8571[WIP]: improve QueryEventHook execution
This commit hardens guarantees around QueryEventHook execution by
adding the following features:
*hook timeout/cancellation*
A timeout for hook execution can be configured through the backend
flag `query_event_hook_timeout_s`, which specified a timeout value
in seconds. If a hook has not completed execution within this timeout
(measured from hook submission, not execution) then the hook task
will be cancelled in order to free up resources.
*hook rejection*
The hook execution engine now has a fixed-capacity work queue whose
capacity can be configured through the backend flag
`query_event_hook_queue_capacity`. This queue is used to store
hook tasks that are submitted when there are no free threads
available for hook execution. All hook tasks submitted when the
queue is at capacity will be rejected and logged without affecting
the result of the query.
*hook performance metrics*
The following hook metrics are captured:
*${hookClass}.${method}.execution.rejections*
Counter indicating how many submitted tasks have been rejected
due to a full work queue
*${hookClass}.${method}.execution.exceptions*
Counter indicating how many tasks have thrown an exception
during execution
*${hookClass}.${method}.execution.timeouts*
Counter indicating how many tasks have been cancelled due to
not completing within {@code hookTimeout_s} of submission.
*${hookClass}.${method}.submissions*
Counter indicating the number of times ${hookClass}.${method}
has been submitted for execution.
*${hookClass}.${method}.execution.time*
Timer indicating the amount of time ${hookClass}.${method}
has taken to complete, whether normally or by error (e.g. timeout or
exception).
Because tasks can potentially be cancelled before beginning execution, the
Timer.getCount() of this timer may be less than
`${hookClass}.${method}.submissions`
*${hookClass}.${method}.queued.time*
Timer indicating the amount of time between hook task submission
and hook task execution. Since a task may be cancelled before it even begins
execution, {@link Timer#getCount()} of this timer may be less than
`${hookClass}.${method}.submissions`
Testing:
- added unit tests for new features
- re-ran existing E2E tests
Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
---
M be/src/service/impala-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
A fe/src/main/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutor.java
M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
A
fe/src/test/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutorTest.java
M fe/src/test/java/org/apache/impala/hooks/QueryEventHookManagerTest.java
M query-event-hook-api/src/main/java/org/apache/impala/hooks/QueryEventHook.java
10 files changed, 818 insertions(+), 109 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13748/14
--
To view, visit http://gerrit.cloudera.org:8080/13748
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
Gerrit-Change-Number: 13748
Gerrit-PatchSet: 14
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3833/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 13 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 08 Jul 2019 10:26:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Hello Fredy Wijaya, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13748
to look at the new patch set (#13).
Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution
..
IMPALA-8571[WIP]: improve QueryEventHook execution
This commit hardens guarantees around QueryEventHook execution by
adding the following features:
*hook timeout/cancellation*
A timeout for hook execution can be configured through the backend
flag `query_event_hook_timeout_s`, which specified a timeout value
in seconds. If a hook has not completed execution within this timeout
(measured from hook submission, not execution) then the hook task
will be cancelled in order to free up resources.
*hook rejection*
The hook execution engine now has a fixed-capacity work queue whose
capacity can be configured through the backend flag
`query_event_hook_queue_capacity`. This queue is used to store
hook tasks that are submitted when there are no free threads
available for hook execution. All hook tasks submitted when the
queue is at capacity will be rejected and logged without affecting
the result of the query.
*hook performance metrics*
The following hook metrics are captured:
*${hookClass}.${method}.execution.rejections*
Counter indicating how many submitted tasks have been rejected
due to a full work queue
*${hookClass}.${method}.execution.exceptions*
Counter indicating how many tasks have thrown an exception
during execution
*${hookClass}.${method}.execution.timeouts*
Counter indicating how many tasks have been cancelled due to
not completing within {@code hookTimeout_s} of submission.
*${hookClass}.${method}.submissions*
Counter indicating the number of times ${hookClass}.${method}
has been submitted for execution.
*${hookClass}.${method}.execution.time*
Timer indicating the amount of time ${hookClass}.${method}
has taken to complete, whether normally or by error (e.g. timeout or
exception).
Because tasks can potentially be cancelled before beginning execution, the
Timer.getCount() of this timer may be less than
`${hookClass}.${method}.submissions`
*${hookClass}.${method}.queued.time*
Timer indicating the amount of time between hook task submission
and hook task execution. Since a task may be cancelled before it even begins
execution, {@link Timer#getCount()} of this timer may be less than
`${hookClass}.${method}.submissions`
Testing:
- added unit tests for new features
- re-ran existing E2E tests
Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
---
M be/src/service/impala-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
A fe/src/main/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutor.java
M fe/src/main/java/org/apache/impala/hooks/QueryEventHookManager.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
A
fe/src/test/java/org/apache/impala/hooks/FixedCapacityQueryHookExecutorTest.java
M fe/src/test/java/org/apache/impala/hooks/QueryEventHookManagerTest.java
M query-event-hook-api/src/main/java/org/apache/impala/hooks/QueryEventHook.java
10 files changed, 817 insertions(+), 109 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/13748/13
--
To view, visit http://gerrit.cloudera.org:8080/13748
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643
Gerrit-Change-Number: 13748
Gerrit-PatchSet: 13
Gerrit-Owner: radford nguyen
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: radford nguyen
[Impala-ASF-CR] IMPALA-8571[WIP]: improve QueryEventHook execution
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13748 ) Change subject: IMPALA-8571[WIP]: improve QueryEventHook execution .. Patch Set 13: (3 comments) http://gerrit.cloudera.org:8080/#/c/13748/13/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/13748/13/be/src/service/impala-server.cc@282 PS13, Line 282: DEFINE_int32(query_event_hook_queue_capacity, 16, "Maximum number of QueryEventHook tasks " line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13748/13/be/src/service/impala-server.cc@283 PS13, Line 283: "to queue before tasks begin to get rejected. Tasks are queued when there are no free " line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13748/13/be/src/util/backend-gflag-util.cc File be/src/util/backend-gflag-util.cc: http://gerrit.cloudera.org:8080/#/c/13748/13/be/src/util/backend-gflag-util.cc@162 PS13, Line 162: cfg.__set_query_event_hook_use_daemon_threads(FLAGS_query_event_hook_use_daemon_threads); line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb88422f7cfe86947d11ce57d2b4c63e57d1b643 Gerrit-Change-Number: 13748 Gerrit-PatchSet: 13 Gerrit-Owner: radford nguyen Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 08 Jul 2019 09:47:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8729: [DOCS] Describe on-demand metadata feature
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13802 ) Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/379/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Gerrit-Change-Number: 13802 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 18:21:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8729: [DOCS] Describe on-demand metadata feature
Hello Bharath Vissapragada, Balazs Jeszenszky, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13802 to look at the new patch set (#2). Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature .. IMPALA-8729: [DOCS] Describe on-demand metadata feature - Overview of on-demand metadata. - Config flags to enable/disable on-demand metadata. Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 --- M docs/topics/impala_metadata.xml 1 file changed, 103 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/13802/2 -- To view, visit http://gerrit.cloudera.org:8080/13802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Gerrit-Change-Number: 13802 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8729: [DOCS] Describe on-demand metadata feature
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13802 ) Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13802/1/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/13802/1/docs/topics/impala_metadata.xml@61 PS1, Line 61: and cache it locally. > Since you mention lack of eviction in the problem statement, I think we cou Done http://gerrit.cloudera.org:8080/#/c/13802/1/docs/topics/impala_metadata.xml@65 PS1, Line 65: metadata fetches > This is true between the coordinator and catalog only. The catalogd still h Done -- To view, visit http://gerrit.cloudera.org:8080/13802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Gerrit-Change-Number: 13802 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 18:12:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format
Jiawei Wang has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13801 ) Change subject: IMPALA-5149: Provide query profile in JSON format .. IMPALA-5149: Provide query profile in JSON format Today there is a text and Thrift version of the query profile, but it would be useful to have a JSON version for portability and machine consumption. The ultimate goal is to have a Download JSON format profile link along with the other two formats. Design Document: https://docs.google.com/document/d/ 15P_Lmjf1rlZUD4PZLXdUwTE8Lv_ME3NQ9Yf4MITzO2M/edit Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1 --- M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/util/runtime-profile-counters.h M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M common/thrift/RuntimeProfile.thrift M www/query_profile.tmpl 11 files changed, 364 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/13801/2 -- To view, visit http://gerrit.cloudera.org:8080/13801 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1 Gerrit-Change-Number: 13801 Gerrit-PatchSet: 2 Gerrit-Owner: Jiawei Wang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-5149: Provide query profile in JSON format
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13801 ) Change subject: IMPALA-5149: Provide query profile in JSON format .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3838/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13801 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8181ac818bf22207ca1deabd9220c397ae723ec1 Gerrit-Change-Number: 13801 Gerrit-PatchSet: 2 Gerrit-Owner: Jiawei Wang Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 18:12:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8729: [DOCS] Describe on-demand metadata feature
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13802 ) Change subject: IMPALA-8729: [DOCS] Describe on-demand metadata feature .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/379/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13802 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Gerrit-Change-Number: 13802 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Balazs Jeszenszky Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 18:42:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8543: More diagnostics for TAcceptQueueServer
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13790 ) Change subject: IMPALA-8543: More diagnostics for TAcceptQueueServer .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.h File be/src/rpc/TAcceptQueueServer.h: http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.h@119 PS1, Line 119: cnxns_setup_time_metric_ cnxns_setup_time_us_metric_ or note the units in the comment http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.cpp File be/src/rpc/TAcceptQueueServer.cpp: http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.cpp@255 PS1, Line 255: timer.Reset(); Maybe make this two separate variables, eg. setup_timer and thread_wait_timer, or at least leave a comment here and at the call to Start() eg: // Start timing for connection setup // Start timing for getting the service thread http://gerrit.cloudera.org:8080/#/c/13790/1/be/src/rpc/TAcceptQueueServer.cpp@293 PS1, Line 293: string Can this one also be made const? http://gerrit.cloudera.org:8080/#/c/13790/1/common/thrift/metrics.json File common/thrift/metrics.json: http://gerrit.cloudera.org:8080/#/c/13790/1/common/thrift/metrics.json@955 PS1, Line 955: connection client connection? here and elsewhere -- To view, visit http://gerrit.cloudera.org:8080/13790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36 Gerrit-Change-Number: 13790 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Mon, 08 Jul 2019 19:57:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8589: Re-enable flaky test query event hooks.py
radford nguyen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13713 ) Change subject: IMPALA-8589: Re-enable flaky test_query_event_hooks.py .. Patch Set 9: (1 comment) http://gerrit.cloudera.org:8080/#/c/13713/9/tests/custom_cluster/test_query_event_hooks.py File tests/custom_cluster/test_query_event_hooks.py: http://gerrit.cloudera.org:8080/#/c/13713/9/tests/custom_cluster/test_query_event_hooks.py@49 PS9, Line 49: expected_count=-1) This test no longer checks for the execution of `onQueryComplete`... it only checks that `onImpalaStartup` is called, which is not a query event hook at all, really. I think we still need coverage for `onQueryComplete` because it executes on the thread pool executor, which is different than `onImpalaStartup` -- To view, visit http://gerrit.cloudera.org:8080/13713 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia64550e986b5eba59a1d77657943932bb977d470 Gerrit-Change-Number: 13713 Gerrit-PatchSet: 9 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: radford nguyen Gerrit-Comment-Date: Mon, 08 Jul 2019 20:32:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13814 Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. IMPALA-8745: [DOCS] More details on INITCAP() function Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead --- M docs/topics/impala_string_functions.xml 1 file changed, 38 insertions(+), 53 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/13814/1 -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/380/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 21:08:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 21:10:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/381/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 21:10:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13814 to look at the new patch set (#2). Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. IMPALA-8745: [DOCS] More details on INITCAP() function Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead --- M docs/topics/impala_string_functions.xml 1 file changed, 10 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/13814/2 -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/380/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 21:18:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/381/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Mon, 08 Jul 2019 21:20:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8745: [DOCS] More details on INITCAP() function
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13814 ) Change subject: IMPALA-8745: [DOCS] More details on INITCAP() function .. IMPALA-8745: [DOCS] More details on INITCAP() function Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Reviewed-on: http://gerrit.cloudera.org:8080/13814 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_string_functions.xml 1 file changed, 10 insertions(+), 2 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13814 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iffead7bd8a8dab80992751cdde743165b9720ead Gerrit-Change-Number: 13814 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-5973: Provide query plan in JSON format
Jiawei Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/11974 ) Change subject: IMPALA-5973: Provide query plan in JSON format .. Patch Set 3: Hi, I am trying to pick this up. Is there anyone who is familiar with this ticket and its process knows where should I start? Thanks! -- To view, visit http://gerrit.cloudera.org:8080/11974 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0f07e2223be58b7323e1ea2fa44b62626cdf4944 Gerrit-Change-Number: 11974 Gerrit-PatchSet: 3 Gerrit-Owner: Pranay Singh (320) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jiawei Wang Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Pranay Singh (320) Gerrit-Comment-Date: Mon, 08 Jul 2019 21:35:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8710: Increase allowed bit width to 64 for bit packing
Daniel Becker has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13809 ) Change subject: IMPALA-8710: Increase allowed bit width to 64 for bit packing .. IMPALA-8710: Increase allowed bit width to 64 for bit packing Fixed the buffer overflow that the previous attempt introduced. Increasing the allowed bit width for bit packing and bit unpacking to 64 bits. This will be needed to support the Parquet delta encoding. Added new methods to BitWriter and BatchedBitReader handling Uleb and ZigZag integers for 64 bits, also needed by delta encoding. The performance of bit unpacking is either the same or better with the new implementation, depending on bit width. Updated the results in bit-packing-benchmark.cc. Testing: - Modified bit packing and unpacking tests to test bit widths up to 64 bits. - Tests covering the additions in BitWriter and BatchedBitReader. Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b --- M be/src/benchmarks/bit-packing-benchmark.cc M be/src/util/CMakeLists.txt M be/src/util/bit-packing-test.cc M be/src/util/bit-packing.h M be/src/util/bit-packing.inline.h M be/src/util/bit-stream-utils-test.cc M be/src/util/bit-stream-utils.h M be/src/util/bit-stream-utils.inline.h M be/src/util/rle-encoding.h M be/src/util/rle-test.cc 10 files changed, 430 insertions(+), 209 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/13809/2 -- To view, visit http://gerrit.cloudera.org:8080/13809 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib9276ca291e3d36db16f63f8caf3e9248a18d85b Gerrit-Change-Number: 13809 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins
