[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 7: Code-Review+2 Carrying +2. -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 7 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 26 Oct 2018 04:47:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3362/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 7 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 26 Oct 2018 04:47:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1170/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 26 Oct 2018 03:52:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata
Adam Holley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11794 ) Change subject: IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f Gerrit-Change-Number: 11794 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Fri, 26 Oct 2018 03:19:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Adam Holley has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException The problem was acache consistency issue between impalad and catalogd. Because a Sentry refresh was occuring during an update to privileges from the alter table set owner, impalad had the correct privileges, which allowed the "show grant role" to succeed but the privileges in catalogd were being overwritten from the sentry refresh. Added a delay in the drop call to ensure privileges are updated. This is a workaround to get the tests to pass with the existing behaviour and should be reassessed if IMPALA-7763 is implemented. This would add a lock to possibly prevent this, but will need it's own assessment. Testing: - Ran custom cluster tests 50 times Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 --- M tests/authorization/test_owner_privileges.py 1 file changed, 21 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/11786/4 -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 4 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7662: fix error race when scanner open fails
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11596 ) Change subject: IMPALA-7662: fix error race when scanner open fails .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc File be/src/exec/hdfs-scan-node.cc: http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc@498 PS8, Line 498: scanner > scanner.get() Ignore comment, just saw that boost overloads it -- To view, visit http://gerrit.cloudera.org:8080/11596 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I45a61210ca7d057b048c77d9f2f2695ec450f19b Gerrit-Change-Number: 11596 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 26 Oct 2018 02:25:05 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7662: fix error race when scanner open fails
Bikramjeet Vig has posted comments on this change. (
http://gerrit.cloudera.org:8080/11596 )
Change subject: IMPALA-7662: fix error race when scanner open fails
..
Patch Set 8:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node-base.h
File be/src/exec/hdfs-scan-node-base.h:
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node-base.h@562
PS8, Line 562: open
nit: opened
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc
File be/src/exec/hdfs-scan-node.cc:
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc@491
PS8, Line 491: scanner->Close() call marks a scan range as complete
if this is true for all hdfs scan nodes, maybe add this to the method comment
of HdfsScanner::Close()
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc@498
PS8, Line 498: scanner
scanner.get()
http://gerrit.cloudera.org:8080/#/c/11596/8/be/src/exec/hdfs-scan-node.cc@535
PS8, Line 535: }
L492-498 and L524-531 do pretty much the same thing and have the same comments.
Maybe extract it in a helper method? or will that be an overkill?
http://gerrit.cloudera.org:8080/#/c/11596/8/tests/query_test/test_scanners.py
File tests/query_test/test_scanners.py:
http://gerrit.cloudera.org:8080/#/c/11596/8/tests/query_test/test_scanners.py@733
PS8, Line 733: del vector.get_value('exec_option')['debug_action']
dont we skip the test above if this is set?
--
To view, visit http://gerrit.cloudera.org:8080/11596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I45a61210ca7d057b048c77d9f2f2695ec450f19b
Gerrit-Change-Number: 11596
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Pooja Nilangekar
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Fri, 26 Oct 2018 02:19:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [DOCS] 2 Typos fixed in NVL2 examples
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11795 ) Change subject: [DOCS] 2 Typos fixed in NVL2 examples .. [DOCS] 2 Typos fixed in NVL2 examples Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Reviewed-on: http://gerrit.cloudera.org:8080/11795 Reviewed-by: Alex Rodoni Tested-by: Impala Public Jenkins --- M docs/topics/impala_conditional_functions.xml 1 file changed, 10 insertions(+), 15 deletions(-) Approvals: Alex Rodoni: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Gerrit-Change-Number: 11795 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [DOCS] 2 Typos fixed in NVL2 examples
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11795 ) Change subject: [DOCS] 2 Typos fixed in NVL2 examples .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/118/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/11795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Gerrit-Change-Number: 11795 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 26 Oct 2018 02:14:14 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] 2 Typos fixed in NVL2 examples
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/11795 ) Change subject: [DOCS] 2 Typos fixed in NVL2 examples .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Gerrit-Change-Number: 11795 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 26 Oct 2018 02:06:29 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] 2 Typos fixed in NVL2 examples
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11795 ) Change subject: [DOCS] 2 Typos fixed in NVL2 examples .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/118/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Gerrit-Change-Number: 11795 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 26 Oct 2018 02:06:06 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] 2 Typos fixed in NVL2 examples
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11795 Change subject: [DOCS] 2 Typos fixed in NVL2 examples .. [DOCS] 2 Typos fixed in NVL2 examples Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 --- M docs/topics/impala_conditional_functions.xml 1 file changed, 10 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/11795/1 -- To view, visit http://gerrit.cloudera.org:8080/11795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib3ac978398eb3de1877e3cd26f662a34c3f131d0 Gerrit-Change-Number: 11795 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. IMPALA-7714: remove unsafe code from signal handlers IMPALA-6271 added LOG statements to some signal handlers and an exit() call to a different signal handler. These functions are not async-signal safe. The fixes are: * Use the write system call directly. I tried using glog's RAW_LOG functionality but had major issues getting it to work. * Call _exit() directly instead of exit() so that static destructors are not run. This is the same default behaviour as SIGTERM. This wans't necessary to prevent this specific crash. Testing: Could reproduce the crash by looping tests/custom_cluster/test_local_catalog.py until a minidump was produced. After this fix it did not reproduce after looping for a few hours. Ran exhaustive build. Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Reviewed-on: http://gerrit.cloudera.org:8080/11777 Tested-by: Impala Public Jenkins Reviewed-by: Tim Armstrong --- M be/src/common/init.cc M be/src/util/minidump.cc M tests/custom_cluster/test_breakpad.py 3 files changed, 9 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 7 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Fri, 26 Oct 2018 01:50:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11688 ) Change subject: IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@1169 PS1, Line 1169: // The per-partition schema information is not used by Impala. Can we slim them down even before serializing it at the Catalog server? Might as well save some CPU and RPC costs https://github.com/apache/impala/blob/1dbb3c1be634c2a3e356c3b2ee5deac2436c9815/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java#L1727 -- To view, visit http://gerrit.cloudera.org:8080/11688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c861452cf974970ab511406fe2e175ea3dc668d Gerrit-Change-Number: 11688 Gerrit-PatchSet: 1 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 26 Oct 2018 01:21:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Fri, 26 Oct 2018 01:13:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 6 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Fri, 26 Oct 2018 00:22:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 6: Should there also be an end-to-end test for this ? It may be worth augmenting some of the queries in testdata/workloads/functional-query/queries/QueryTest/analytic-fns.test ? -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 6 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 23:57:25 + Gerrit-HasComments: No
[Impala-ASF-CR] test-with-docker: add --env option to pass through env variables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11730 ) Change subject: test-with-docker: add --env option to pass through env variables .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Gerrit-Change-Number: 11730 Gerrit-PatchSet: 4 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 25 Oct 2018 23:52:57 + Gerrit-HasComments: No
[Impala-ASF-CR] test-with-docker: add --env option to pass through env variables
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11730 ) Change subject: test-with-docker: add --env option to pass through env variables .. test-with-docker: add --env option to pass through env variables Adds simple mechanism to pass environment variables into test-with-docker, which is occasionally useful, especially for development and tinkering with tests. It's typically the right thing to codify the environment variables into tests, but a pass through can be handy. The implementation is simple enough, passing the variables into the docker containers. Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Reviewed-on: http://gerrit.cloudera.org:8080/11730 Reviewed-by: Laszlo Gaal Reviewed-by: Joe McDonnell Tested-by: Impala Public Jenkins --- M docker/test-with-docker.py 1 file changed, 26 insertions(+), 18 deletions(-) Approvals: Laszlo Gaal: Looks good to me, but someone else must approve Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/11730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Gerrit-Change-Number: 11730 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11688 ) Change subject: IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/11688/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11688/1//COMMIT_MSG@14 PS1, Line 14: objects. (At present, we have no automated way to inspect a heap dump.) fwiw I tested changes like this in the past by starting an impalad, querying some table(s) to populate the cache, and then using 'jmap -histo:live' on the impalad. See 1dd4403fdcae665b0f343bf3fb7cb644c1875851's commit message for an example along with a script that can be handy to compare before/after jmap histo dumps. Not super critical to go back and do it for this change if you're busy with other stuff. http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java: http://gerrit.cloudera.org:8080/#/c/11688/1/fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java@71 PS1, Line 71: checkState checkArgument -- To view, visit http://gerrit.cloudera.org:8080/11688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c861452cf974970ab511406fe2e175ea3dc668d Gerrit-Change-Number: 11688 Gerrit-PatchSet: 1 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 23:36:21 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7759: Add Levenshtein edit distance built-in function
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11793 ) Change subject: IMPALA-7759: Add Levenshtein edit distance built-in function .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/11793/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11793/1//COMMIT_MSG@10 PS1, Line 10: distance. Implemented as levenshtein() to match PostgreSQL in If it's identical to the postgres function we could add it to the random query generator in tests/comparison/funcs.py - see for example Length http://gerrit.cloudera.org:8080/#/c/11793/1/be/src/exprs/string-functions-ir.cc File be/src/exprs/string-functions-ir.cc: http://gerrit.cloudera.org:8080/#/c/11793/1/be/src/exprs/string-functions-ir.cc@1113 PS1, Line 1113: // https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance#C++ Long line. We also need to figure out what the license is so that it's clear this is legit. http://gerrit.cloudera.org:8080/#/c/11793/1/be/src/exprs/string-functions-ir.cc@1127 PS1, Line 1127: auto column_start = (decltype(s1len))1; I'd just go with the less clever: int column_start = 1; http://gerrit.cloudera.org:8080/#/c/11793/1/be/src/exprs/string-functions-ir.cc@1129 PS1, Line 1129: auto column = new decltype(s1len)[s1len + 1]; It would be better to use context->Allocate() and Free() instead of new so that we can track the memory, etc. http://gerrit.cloudera.org:8080/#/c/11793/1/be/src/exprs/string-functions-ir.cc@1132 PS1, Line 1132: auto I'd just use int instead of auto in most of these places - it's less clever but more obvious -- To view, visit http://gerrit.cloudera.org:8080/11793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I549d33ab7cebfa10db2934461c8ec91e2cc1cdcb Gerrit-Change-Number: 11793 Gerrit-PatchSet: 1 Gerrit-Owner: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 23:28:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11688 ) Change subject: IMPALA-7501: Slim down metastore Partition objects in LocalCatalog cache .. Patch Set 1: Hey Todd, you asked for this fix. Can you take a look at this change? Thanks! -- To view, visit http://gerrit.cloudera.org:8080/11688 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c861452cf974970ab511406fe2e175ea3dc668d Gerrit-Change-Number: 11688 Gerrit-PatchSet: 1 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 23:25:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11794 ) Change subject: IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1169/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f Gerrit-Change-Number: 11794 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 23:16:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4063: Merge report of query fragment instances per executor
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/11615 ) Change subject: IMPALA-4063: Merge report of query fragment instances per executor .. Patch Set 6: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/11615/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11615/5//COMMIT_MSG@14 PS5, Line 14: In addition, due to the lack of coordination between query : fragment instances, a query may end without collecting the profiles : from all fragment instances when one of them hits an error before : another fragment instance manages to finish Prepare(), leading to : missing profiles for certain fragment instances. > Yes. Uncommented some of the tests in bloom_filters.test. Great. Could you also remove the xfail from test_profile_fragment_instances http://gerrit.cloudera.org:8080/#/c/11615/6/tests/query_test/test_udfs.py File tests/query_test/test_udfs.py: http://gerrit.cloudera.org:8080/#/c/11615/6/tests/query_test/test_udfs.py@290 PS6, Line 290: # Some tests assume determinism or non-determinism, which depends on expr rewrites. This comment is outdated, and below -- To view, visit http://gerrit.cloudera.org:8080/11615 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5f95e026ba05631f33f48ce32da6db39c6f421fa Gerrit-Change-Number: 11615 Gerrit-PatchSet: 6 Gerrit-Owner: Michael Ho Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 22:50:33 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata
Fredy Wijaya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11794 ) Change subject: IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata .. IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata Before this patch, a bug in SentryProxy caused a hang when running invalidate metadata due to privilege version inconsistency. I was able to manually reproduce the issue by doing the following steps: 1. Get all Sentry role privileges for role a: [x, y] --> in SentryProxy 2. Add a sleep statement before getting all Sentry roles to simulate the timing issue--> in SentryProxy 3. Remove role a --> Externally via Sentry CLI 4. Privileges x and y in step 1 do not get removed in the catalog even those they were removed in step 3, which causes the catalog version inconsistency 5. Run invalidate metadata, this will cause it to hang due to catalog version inconsistency The fix is to remove all privileges in the catalog if there are no privileges (null or empty) returned by Sentry. Testing: - Manually tested the patch with by the above steps and did not encounter the hang when issuing invalidate metadata. Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f --- M fe/src/main/java/org/apache/impala/util/SentryProxy.java 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/11794/2 -- To view, visit http://gerrit.cloudera.org:8080/11794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f Gerrit-Change-Number: 11794 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11794 ) Change subject: IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1168/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f Gerrit-Change-Number: 11794 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 22:28:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7759: Add Levenshtein edit distance built-in function
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11793 ) Change subject: IMPALA-7759: Add Levenshtein edit distance built-in function .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1167/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I549d33ab7cebfa10db2934461c8ec91e2cc1cdcb Gerrit-Change-Number: 11793 Gerrit-PatchSet: 1 Gerrit-Owner: Greg Rahn Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 22:23:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11794 Change subject: IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata .. IMPALA-7760: Privilege version inconsistency causes a hang when running invalidate metadata Before this patch, a bug in SentryProxy caused a hang when running invalidate metadata due to privilege version inconsistency. I was able to manually reproduce the issue by doing the following steps: 1. Get all Sentry role privileges for role a: [x, y] --> in SentryProxy 2. Add a sleep statement before getting all Sentry roles to simulate the timing issue--> in SentryProxy 3. Remove role a --> Externally via Sentry CLI 4. Privileges x and y in step 1 do not get removed in the catalog even those they were removed in step 3, which causes the catalog version inconsistency 5. Run invalidate metadata, this will cause it to hang due to catalog version inconsistency The fix is to remove all privileges in the catalog if there are no privileges (null or empty) returned by Sentry. Testing: - Manually tested the patch with by the above steps and did not encounter the hang when issuing invalidate metadata. Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f --- M fe/src/main/java/org/apache/impala/util/SentryProxy.java 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/11794/1 -- To view, visit http://gerrit.cloudera.org:8080/11794 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib1e0db2b1f727476f489c732c4f4e5bc1582429f Gerrit-Change-Number: 11794 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-7759: Add Levenshtein edit distance built-in function
Greg Rahn has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11793 Change subject: IMPALA-7759: Add Levenshtein edit distance built-in function .. IMPALA-7759: Add Levenshtein edit distance built-in function This patch adds new built-in functions to calculate Levenshtein edit distance. Implemented as levenshtein() to match PostgreSQL in both functionality and name and also added le_dst() alias for Netezza, compatibility, but note that levenshtein() differs in functionality in that if either value is NULL or both values are NULL, levenshtein() returns NULL, where Netezza's le_dst() returns the length of the not NULL value or 0 if both values are NULL. Testing: - Added unit tests to expr-test.cc - Manual test on 966289 string pairs and results match PostgreSQL Change-Id: I549d33ab7cebfa10db2934461c8ec91e2cc1cdcb --- M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc M be/src/exprs/string-functions.h M common/function-registry/impala_functions.py 4 files changed, 68 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/11793/1 -- To view, visit http://gerrit.cloudera.org:8080/11793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I549d33ab7cebfa10db2934461c8ec91e2cc1cdcb Gerrit-Change-Number: 11793 Gerrit-PatchSet: 1 Gerrit-Owner: Greg Rahn Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: Hit IMPALA-7523 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 21:21:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3361/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 21:21:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 7: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 7 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 21:17:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. IMPALA-7351: Add estimates to Exchange node Added rough estimates for exchange node and a justification of the method in the in-line comments. Testing: Updated Planner tests. Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Reviewed-on: http://gerrit.cloudera.org:8080/11692 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test 13 files changed, 920 insertions(+), 831 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 8 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/11786/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11786/3//COMMIT_MSG@15 PS3, Line 15: updated. something here is unclear about whether the feature is working as intended but the test was incorrect or if something needs to improve w/ the feature (perhaps link to the jira that discusses solving some of these issues w/ a lock?). -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 21:16:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1166/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 6 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 21:08:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc@192 PS5, Line 192: STDOUT_FILENO > This is just preserving the existing behaviour of logging to the INFO log s Got it, SGTM, thanks for the explanation. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 20:48:29 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3353/ -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 20:46:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 20:36:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3360/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 6 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 20:32:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3359/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 6 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 20:32:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Hello Quanlong Huang, Laszlo Gaal, Jim Apple, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11731 to look at the new patch set (#6). Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. IMPALA-7698: Add centos support to bootstrap_system. Largely, the changes involve conditionalizing some invocations to account for differences between RH and Ubuntu. The trickiest bits were timezone-related test errors (see below), postgresql permissions (need to accept md5 passwords from localhost) and default ulimits (1024 user processes/threads is not enough). To test this, I built using test-with-docker. In additional to the ulimit issue, I ran into the fact that /tmp needed 1777 permissions for the postgresql socket, and entrypoint.sh had a few places that needed special cases. At the moment, the data load ran fine, as did most of the tests. I observed a test that relied on a python2.7-ism fail, which is part of the point of this. In the course of development, I encountered a handful of tests fail with "Encounter parse error: failed to open /usr/share/zoneinfo/GMT-08:00 - No such file or directory.", which was reproduced as follows: [localhost:21000] default> use functional_orc_def; select * from alltypes; ... WARNINGS: Encounter parse error: failed to open /usr/share/zoneinfo/GMT-08:00 - No such file or directory. With Quanlong's help, I learned what was happening. test-with-docker was translating my time zone (America/Los_Angeles) to US/Pacific-New, because realpath(/etc/localtime) = US/Pacific-New. This timezone exists in centos:6, so that wasn't a problem. However, this timezone does not exist in the package "tzdata-java", which is the copy of the timezone information used by Java. (There are bugs here that may have been fixed in centos:7.) As a result, when ORC asks (by using TimeZone.getDefault().getID()) the JDK (src/solaris/native/java/util/TimeZone_md.c) for the default timezone, it can't find the same name as /etc/localtime points to in its repository and defaults to "GMT-08:00". This string then gets written into the ORC files generated by Hive as part of data load, and then the C++ library can't read them. This is fixed by changing "realpath" to "readlink" in test-with-docker.py. centos:7 is not addressed by this change. The move to systemd makes "service sshd start" (and the same for postgresql) not work, and additional care needs to be done to work around that. This change is a joint effort with Laszlo Gaal. Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 --- M bin/bootstrap_system.sh M docker/entrypoint.sh M docker/test-with-docker.py 3 files changed, 166 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/11731/6 -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 6 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3357/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 20:30:48 + Gerrit-HasComments: No
[Impala-ASF-CR] test-with-docker: add --env option to pass through env variables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11730 ) Change subject: test-with-docker: add --env option to pass through env variables .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3356/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Gerrit-Change-Number: 11730 Gerrit-PatchSet: 4 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 25 Oct 2018 19:57:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1165/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 3 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 19:11:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Vuk Ercegovac has posted comments on this change. (
http://gerrit.cloudera.org:8080/11556 )
Change subject: IMPALA-6323 Allow constant analytic window expressions.
..
Patch Set 6:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11556/5/fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java
File fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java:
http://gerrit.cloudera.org:8080/#/c/11556/5/fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java@91
PS5, Line 91: {
remove. as in L89
http://gerrit.cloudera.org:8080/#/c/11556/6/fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java
File fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java:
http://gerrit.cloudera.org:8080/#/c/11556/6/fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java@91
PS6, Line 91: { return true; }
nit: if we're making any changes, pls remove these braces as mentioned earlier.
--
To view, visit http://gerrit.cloudera.org:8080/11556
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c
Gerrit-Change-Number: 11556
Gerrit-PatchSet: 6
Gerrit-Owner: Michal Ostrowski
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michal Ostrowski
Gerrit-Reviewer: Tim Armstrong
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Thu, 25 Oct 2018 19:11:03 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Update .gitignore
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1164/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 19:06:53 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1163/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:59:46 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 3: Code-Review+1 Thanks! I've found a few more additions, many around the use of Eclipse. Probably easiest if I add those after you commit this set. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:56:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7758: Fix LOCATION clause when creating chars formats *
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11789 ) Change subject: IMPALA-7758: Fix LOCATION clause when creating chars_formats_* .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1162/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11789 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I781b484e7a15ccaa5de590563d68b3dca6a658e5 Gerrit-Change-Number: 11789 Gerrit-PatchSet: 1 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:55:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 6: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3423/artifact/Impala/logs_static/logs/ should have the relevant logs -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 6 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 18:54:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull, coalesce to use CASE
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/11760 ) Change subject: IMPALA-7655: Rewrite if, isnull, coalesce to use CASE .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/11760/4/common/function-registry/impala_functions.py File common/function-registry/impala_functions.py: http://gerrit.cloudera.org:8080/#/c/11760/4/common/function-registry/impala_functions.py@531 PS4, Line 531: # Rewritten away in the FE > See IMPALA-7741. nvl2() currently does not appear in the list of built-in f Actually, this is a bit more subtle. Today, we rewrite nvl2() and nullif() at parse time. The goal is to rewrite them in the rewrite rules, along with other conditionals. Doing so allows us to take advantage of analyzed arguments to implement optimizations. To do that, however, we need the function to be defined in this function table. We need it as a FE-only function (which also allows it to appear in the table of built-in functions) so it passes analysis, even though there is no BE implementation. -- To view, visit http://gerrit.cloudera.org:8080/11760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I526654d8546e17b2545c42cc59dab66d9fe1b163 Gerrit-Change-Number: 11760 Gerrit-PatchSet: 4 Gerrit-Owner: Paul Rogers Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 18:52:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7655: Rewrite if, isnull, coalesce to use CASE
Paul Rogers has posted comments on this change. (
http://gerrit.cloudera.org:8080/11760 )
Change subject: IMPALA-7655: Rewrite if, isnull, coalesce to use CASE
..
Patch Set 4:
(12 comments)
Thanks everyone for the comments.
PhilZ's note to use existing optimizations is a good one. A number of changes
are required to make that happen. Researching that turned up quite a few bugs
listed in IMPALA-7655. I plan to push changes for those bugs one by one to ease
reviews. Once those fixes are in master, I'll push a new change set here
rebased on the revised master.
http://gerrit.cloudera.org:8080/#/c/11760/4/common/function-registry/impala_functions.py
File common/function-registry/impala_functions.py:
http://gerrit.cloudera.org:8080/#/c/11760/4/common/function-registry/impala_functions.py@531
PS4, Line 531: # Rewritten away in the FE
> can you explain why this is needed in this change?
See IMPALA-7741. nvl2() currently does not appear in the list of built-in
functions. However, since this improvement is orthogonal to the topic of this
bug, I'll drop this change from here and do a separate change set for
IMPALA-7741.
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java@a92
PS4, Line 92:
> rewrites can be disabled via a query option so cannot be assumed to run. it
Right. Turns out that I had to alter the rule application flow to always
rewrite the conditional functions.
The other choice is to leave the current interpreted implementations and have
two paths based on turning on optimizations or not. In the spirit of "simpler
is better", seems to make sense to have only one implementation, but the
two-implementation solution is also possible.
Though, note, in the original code, nvl2() was always rewritten, there never
was a BE implementation, so that aspect is not really changing.
http://gerrit.cloudera.org:8080/#/c/11760/2/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java
File fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java:
http://gerrit.cloudera.org:8080/#/c/11760/2/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@89
PS2, Line 89:* code gen for , avoiding the need for ad-hoc
implementations
:* of each function.
> The following code looks to me like it'll do re-writes until we're "stuck".
Turns out this is a difference between the simplified test framework (which
applies selected rules once) and the production code (which replies rules
repeatedly.) Easy enough to create a separate test that will apply rules in the
production way so that we take multiple passes to verify all rewrites.
Making this change uncovered quite a few obscure bugs which are recorded in the
JIRA and will be submitted as separate changes for review. Once those are in,
I'll rebase this change onto that code, and implement the chained rewrite test
and logic.
http://gerrit.cloudera.org:8080/#/c/11760/2/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@122
PS2, Line 122: private Expr rewriteIfFn(FunctionCallExpr expr) {
> Is the issue that "null is null" is not getting constant optimization? In m
See earlier note.
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java
File fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java:
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@112
PS4, Line 112:
> to be consistent with the style in the frontend Java code, pls remove these
Really? We create Javadoc comments, but don't use the resulting formatting?
Eclipse, for example, will display the formatting in hovers. Without proper
HTML formatting, the result is a bit of a mess.
Still, if our policy is to use Javadoc comments, but without formatting, I can
revert this stuff.
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@144
PS4, Line 144: instanceof NullLiteral)
> I would prefer .isNullLiteral() as other functions use that.
This is existing code that was moved. However, I agree with your point. I've
gone though and cleaned up places that use this older style to use the newer
predicates, including in cases where we test for true and false.
http://gerrit.cloudera.org:8080/#/c/11760/4/fe/src/main/java/org/apache/impala/rewrite/SimplifyConditionalsRule.java@183
PS4, Line 183: revised.size() - 1;
> Can you add a comment about not adding the last child here? It took me some
Done
[Impala-ASF-CR] IMPALA-7758: Fix LOCATION clause when creating chars formats *
David Knupp has posted comments on this change. (
http://gerrit.cloudera.org:8080/11789 )
Change subject: IMPALA-7758: Fix LOCATION clause when creating chars_formats_*
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11789/1/testdata/bin/load-dependent-tables.sql
File testdata/bin/load-dependent-tables.sql:
http://gerrit.cloudera.org:8080/#/c/11789/1/testdata/bin/load-dependent-tables.sql@42
PS1, Line 42: LOCATION
'${hiveconf:hive.metastore.warehouse.dir}/alltypesmixedformat';
> Is this also a problem?
It doesn't appear to be? That line was committed six years ago. But we should
probably be consistent.
I'll change that line, and kick off another round of tests. Thanks.
--
To view, visit http://gerrit.cloudera.org:8080/11789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I781b484e7a15ccaa5de590563d68b3dca6a658e5
Gerrit-Change-Number: 11789
Gerrit-PatchSet: 1
Gerrit-Owner: David Knupp
Gerrit-Reviewer: David Knupp
Gerrit-Reviewer: Michael Brown
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Thu, 25 Oct 2018 18:43:13 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] test-with-docker: allow built images to be used with "docker run" easily.
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/11781 ) Change subject: test-with-docker: allow built images to be used with "docker run" easily. .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8d6a28d4cb4ab019cd72415024b23374a6d9e2f Gerrit-Change-Number: 11781 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Thu, 25 Oct 2018 18:36:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7743: [DOCS] A new option to load incremental statistics from catalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11790 ) Change subject: IMPALA-7743: [DOCS] A new option to load incremental statistics from catalog .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/117/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8fd9b88138350406065df2f39a48043178759949 Gerrit-Change-Number: 11790 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Oct 2018 18:35:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7743: [DOCS] A new option to load incremental statistics from catalog
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11790 Change subject: IMPALA-7743: [DOCS] A new option to load incremental statistics from catalog .. IMPALA-7743: [DOCS] A new option to load incremental statistics from catalog --pull_incremental_statistics described in the Incremental Stats section. Change-Id: I8fd9b88138350406065df2f39a48043178759949 --- M docs/shared/impala_common.xml M docs/topics/impala_perf_stats.xml 2 files changed, 76 insertions(+), 46 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/11790/1 -- To view, visit http://gerrit.cloudera.org:8080/11790 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8fd9b88138350406065df2f39a48043178759949 Gerrit-Change-Number: 11790 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Michal Ostrowski has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 6: > Patch Set 6: > > The test failures look legit and related to this change: > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3423/ Are the actual test logs for that run available some where? The stuff from the jenkins jobs is very cursory. I've run the tests myself but the errors I get in PlannerTest don't seem like they are due to the change. -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 6 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 18:30:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7758: Fix LOCATION clause when creating chars formats *
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11789 ) Change subject: IMPALA-7758: Fix LOCATION clause when creating chars_formats_* .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11789 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I781b484e7a15ccaa5de590563d68b3dca6a658e5 Gerrit-Change-Number: 11789 Gerrit-PatchSet: 1 Gerrit-Owner: David Knupp Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Michael Brown Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:24:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7758: Fix LOCATION clause when creating chars formats *
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/11789 )
Change subject: IMPALA-7758: Fix LOCATION clause when creating chars_formats_*
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11789/1/testdata/bin/load-dependent-tables.sql
File testdata/bin/load-dependent-tables.sql:
http://gerrit.cloudera.org:8080/#/c/11789/1/testdata/bin/load-dependent-tables.sql@42
PS1, Line 42: LOCATION
'${hiveconf:hive.metastore.warehouse.dir}/alltypesmixedformat';
Is this also a problem?
--
To view, visit http://gerrit.cloudera.org:8080/11789
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I781b484e7a15ccaa5de590563d68b3dca6a658e5
Gerrit-Change-Number: 11789
Gerrit-PatchSet: 1
Gerrit-Owner: David Knupp
Gerrit-Reviewer: David Knupp
Gerrit-Reviewer: Michael Brown
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Thu, 25 Oct 2018 18:25:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] Update .gitignore
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 3: Code-Review+1 Thanks for doing this! LGTM. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:23:58 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1161/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:23:42 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Hello Fredy Wijaya, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11787 to look at the new patch set (#3). Change subject: Update .gitignore .. Update .gitignore A few unversioned artifacts crept in over time without corresponding .gitignore entries. These are the updates based on the git status output on my dev env. Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a --- M .gitignore A common/yarn-extras/.gitignore M fe/.gitignore M fe/src/test/resources/.gitignore A lib/python/.gitignore M testdata/.gitignore M testdata/cluster/.gitignore 7 files changed, 34 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/11787/3 -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Update .gitignore
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 2: I think I addressed the ones you mentioned. I don't use some of those IDEs - intellij and codeblocks it looks like - so I can't confirm that I got all of the generated files for those. -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 25 Oct 2018 18:21:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG@15 PS5, Line 15: had major issues getting it to work > It wasn't printing anything. The first layer of issues was this STRIP_LOG t Another thing was off-putting was this: https://github.com/google/glog/blob/master/src/raw_logging.cc#L82 It's probably not an issue, but it also seems like this code isn't maintained that actively so who knows. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 18:13:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/1160/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 18:11:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG@15 PS5, Line 15: had major issues getting it to work > just curious, whats the issue here? It wasn't printing anything. The first layer of issues was this STRIP_LOG thing: https://github.com/google/glog/blob/master/src/glog/raw_logging.h.in#L91 that compiled it out. Then it still wasn't logging I think because of this guard: https://github.com/google/glog/blob/master/src/raw_logging.cc#L118. At that point I concluded that it wasn't necessarily even going to work for us and was non-trivial to figure out under what circumstances it actually worked. http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG@18 PS5, Line 18: wans't > nit: wasn't I'll fix this if I need to do a new patchset for some reason. Doesn't seem worth restarting the merge for.. http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc@192 PS5, Line 192: STDOUT_FILENO > Wouldn't STDERR be a better place for this log line? Or will it be present This is just preserving the existing behaviour of logging to the INFO log so changing that behaviour is out of scope of the change. I think it makes sense though since this is the usual way of killing the process. Mostly stuff in the ERROR log should be when something unexpected happens. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 18:11:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] test-with-docker: allow built images to be used with "docker run" easily.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11781 ) Change subject: test-with-docker: allow built images to be used with "docker run" easily. .. Patch Set 1: Code-Review+1 LGTM. -- To view, visit http://gerrit.cloudera.org:8080/11781 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib8d6a28d4cb4ab019cd72415024b23374a6d9e2f Gerrit-Change-Number: 11781 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Thu, 25 Oct 2018 18:08:56 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11787 ) Change subject: Update .gitignore .. Patch Set 1: We need to ignore more. These are files that I think should also be ignored. $IMPALA_HOME/ .idea/ Impala.cbp be/ASSEMBLER.cbp lib/python/impala_py_lib.egg-info/ testdata/cluster/cdh6/ -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Fredy Wijaya Gerrit-Comment-Date: Thu, 25 Oct 2018 18:07:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3351/ -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 18:06:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 2: Code-Review+1 Vuk, can you take another look at this? -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 18:04:03 + Gerrit-HasComments: No
[Impala-ASF-CR] test-with-docker: add --env option to pass through env variables
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/11730 ) Change subject: test-with-docker: add --env option to pass through env variables .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Gerrit-Change-Number: 11730 Gerrit-PatchSet: 4 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 25 Oct 2018 18:03:39 + Gerrit-HasComments: No
[Impala-ASF-CR] Update .gitignore
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11787 Change subject: Update .gitignore .. Update .gitignore A few unversioned artifacts crept in over time without corresponding .gitignore entries. These are the updates based on the git status output on my dev env. Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a --- M .gitignore A common/yarn-extras/.gitignore M fe/.gitignore M fe/src/test/resources/.gitignore A lib/python/.gitignore 5 files changed, 24 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/11787/1 -- To view, visit http://gerrit.cloudera.org:8080/11787 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I281ab3b5c98ac32e5d60663562628ffda6606a6a Gerrit-Change-Number: 11787 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1159/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 17:52:34 + Gerrit-HasComments: No
[Impala-ASF-CR] test-with-docker: add --env option to pass through env variables
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/11730 ) Change subject: test-with-docker: add --env option to pass through env variables .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/11730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I03c2feda8edc2983e423f862ed210fabb845714f Gerrit-Change-Number: 11730 Gerrit-PatchSet: 4 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Comment-Date: Thu, 25 Oct 2018 17:52:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG@18 PS5, Line 18: wans't nit: wasn't http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc@192 PS5, Line 192: STDOUT_FILENO Wouldn't STDERR be a better place for this log line? Or will it be present in the INFO and in the ERROR log as well if you do it this way? -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 17:50:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Adam Holley has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException The problem was another cache consistency issue between Impalad and Catalogd. Because a Sentry refresh was occuring during an update to privileges from the alter table set owner, Impalad had the correct privileges, which allowed the "show grant role" to succeed but the privileges in catalogd were being overwritten from the sentry refresh. Added a delay in the drop call to ensure privileges are updated. Testing: - Ran custom cluster tests 50 times Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 --- M tests/authorization/test_owner_privileges.py 1 file changed, 21 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/11786/2 -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-7710: test owner privileges with grant failed with AuthorizationException
Adam Holley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11786 ) Change subject: IMPALA-7710: test_owner_privileges_with_grant failed with AuthorizationException .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11786/1/tests/authorization/test_owner_privileges.py File tests/authorization/test_owner_privileges.py: http://gerrit.cloudera.org:8080/#/c/11786/1/tests/authorization/test_owner_privileges.py@95 PS1, Line 95: return self.count_user_privileges(result) == count: : : def _test_clea > Can be simplified to return self.count_user_privileges_result) Done -- To view, visit http://gerrit.cloudera.org:8080/11786 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5a1babd3dcbb94ffaa1f3e6ef2cebf1a1d391219 Gerrit-Change-Number: 11786 Gerrit-PatchSet: 2 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 25 Oct 2018 17:36:25 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc@192 PS5, Line 192: sys_write(STDOUT_FILENO, msg, strlen(msg)); > One question: Does this mean that we can't log the shutdown message to Impa Yeah it ends up in impalad.INFO since stdout is redirected there. We do something similar with the "Wrote minidump to" message. test_breakpad does test that these errors end up in the file -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 17:25:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1158/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 17:23:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3354/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 7 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 17:21:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 7 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 17:21:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 6: Code-Review+2 > Looks like test_explain_level* needs to be updated. Maybe we can > just regex out the estimate from the expected output Done -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 17:19:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11692 to look at the new patch set (#6). Change subject: IMPALA-7351: Add estimates to Exchange node .. IMPALA-7351: Add estimates to Exchange node Added rough estimates for exchange node and a justification of the method in the in-line comments. Testing: Updated Planner tests. Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd --- M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M testdata/workloads/functional-planner/queries/PlannerTest/kudu-selectivity.test M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test M testdata/workloads/functional-query/queries/QueryTest/explain-level2.test M testdata/workloads/functional-query/queries/QueryTest/explain-level3.test 13 files changed, 920 insertions(+), 831 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/11692/6 -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 6 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Zoram Thanga has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/common/init.cc@192 PS5, Line 192: sys_write(STDOUT_FILENO, msg, strlen(msg)); One question: Does this mean that we can't log the shutdown message to Impala log file? Or does STDOUT_FILENO get duped to impalad.INFO? http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/util/minidump.cc File be/src/util/minidump.cc: http://gerrit.cloudera.org:8080/#/c/11777/5/be/src/util/minidump.cc@102 PS5, Line 102: sys_write(STDOUT_FILENO, msg, strlen(msg)); Same question as in the other .cc file. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Zoram Thanga Gerrit-Comment-Date: Thu, 25 Oct 2018 17:10:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11777/5//COMMIT_MSG@15 PS5, Line 15: had major issues getting it to work just curious, whats the issue here? -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:54:19 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3353/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:50:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11777/2/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/2/be/src/common/init.cc@189 PS2, Line 189: _exit(0); > Should we add a comment why we call _exit() instead of exit()? Done -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:49:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-5004: Switch to sorting node for large TopN queries
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11698 ) Change subject: IMPALA-5004: Switch to sorting node for large TopN queries .. Patch Set 5: (3 comments) Looks good overall http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/analysis/SortInfo.java File fe/src/main/java/org/apache/impala/analysis/SortInfo.java: http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/analysis/SortInfo.java@244 PS5, Line 244: getSortTupleDescriptor().getAvgSerializedSize() * (cardinality + offset)); TODO: can cardinality + offset overflow? http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@324 PS5, Line 324: // SortNode#computeStats Can we factor out the formula into a static helper method in SortNode? http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@337 PS5, Line 337: limit + offset Here and elsewhere: I think limit + offset could overflow. We don't have anything that limits the max values of these, e.g. the following is planned as normal. select * from tpch.lineitem order by l_orderkey limit 9223372036854775807 offset 9223372036854775807 We have a checkedAdd() helper for cases like this, but we probably should add a planner test to make sure that extreme cases like this are tested. -- To view, visit http://gerrit.cloudera.org:8080/11698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I34c9db33c9302b55e9978f53f9c7061f2806c8a9 Gerrit-Change-Number: 11698 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Paul Rogers Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 16:48:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Hello Lars Volker, Philip Zeyliger, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11777 to look at the new patch set (#4). Change subject: IMPALA-7714: remove unsafe code from signal handlers .. IMPALA-7714: remove unsafe code from signal handlers IMPALA-6271 added LOG statements to some signal handlers and an exit() call to a different signal handler. These functions are not async-signal safe. The fixes are: * Use the write system call directly. I tried using glog's RAW_LOG functionality but had major issues getting it to work. * Call _exit() directly instead of exit() so that static destructors are not run. This is the same default behaviour as SIGTERM. This wans't necessary to prevent this specific crash. Testing: Could reproduce the crash by looping tests/custom_cluster/test_local_catalog.py until a minidump was produced. After this fix it did not reproduce after looping for a few hours. Ran exhaustive build. Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 --- M be/src/common/init.cc M be/src/util/minidump.cc M tests/custom_cluster/test_breakpad.py 3 files changed, 9 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/11777/4 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1157/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 16:46:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 3: Code-Review+2 (1 comment) Just a nit http://gerrit.cloudera.org:8080/#/c/11777/2/be/src/common/init.cc File be/src/common/init.cc: http://gerrit.cloudera.org:8080/#/c/11777/2/be/src/common/init.cc@189 PS2, Line 189: // Signal handler for SIGTERM, that prints the message before doing an exit. Should we add a comment why we call _exit() instead of exit()? -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:39:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7351: Add estimates to Exchange node
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11692 ) Change subject: IMPALA-7351: Add estimates to Exchange node .. Patch Set 5: Looks like test_explain_level* needs to be updated. Maybe we can just regex out the estimate from the expected output -- To view, visit http://gerrit.cloudera.org:8080/11692 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5b577f9511abc48b992e814d50bba4959f23f7fd Gerrit-Change-Number: 11692 Gerrit-PatchSet: 5 Gerrit-Owner: Bikramjeet Vig Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 16:35:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3352/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:33:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:33:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7714: remove unsafe code from signal handlers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11777 ) Change subject: IMPALA-7714: remove unsafe code from signal handlers .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/1156/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11777 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I52037d6510b9b34ec33d3a8b5492226aee1b9d92 Gerrit-Change-Number: 11777 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Thu, 25 Oct 2018 16:30:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6323 Allow constant analytic window expressions.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11556 ) Change subject: IMPALA-6323 Allow constant analytic window expressions. .. Patch Set 6: The test failures look legit and related to this change: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3423/ -- To view, visit http://gerrit.cloudera.org:8080/11556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf129026c45120e9470df601268863634037908c Gerrit-Change-Number: 11556 Gerrit-PatchSet: 6 Gerrit-Owner: Michal Ostrowski Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michal Ostrowski Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Thu, 25 Oct 2018 16:29:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3351/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 16:28:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/11731 ) Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/11731/4/bin/bootstrap_system.sh File bin/bootstrap_system.sh: http://gerrit.cloudera.org:8080/#/c/11731/4/bin/bootstrap_system.sh@131 PS4, Line 131: pigz > Nit: pigz isn't listed under Ubuntu. Let's add it. I did the opposite; it doesn't exist in centos:6. Interestingly, yum doesn't fail if a package is missing. --setopt=skip_missing_names_on_install=False doesn't work on centos6. http://gerrit.cloudera.org:8080/#/c/11731/4/bin/bootstrap_system.sh@132 PS4, Line 132: sudo > Nit: sudo yum install sudo? Removed. -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 4 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang Gerrit-Comment-Date: Thu, 25 Oct 2018 16:28:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7698: Add centos support to bootstrap system.
Hello Quanlong Huang, Laszlo Gaal, Jim Apple, Joe McDonnell, Csaba Ringhofer, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11731 to look at the new patch set (#5). Change subject: IMPALA-7698: Add centos support to bootstrap_system. .. IMPALA-7698: Add centos support to bootstrap_system. Largely, the changes involve conditionalizing some invocations to account for differences between RH and Ubuntu. The trickiest bits were timezone-related test errors (see below), postgresql permissions (need to accept md5 passwords from localhost) and default ulimits (1024 user processes/threads is not enough). To test this, I built using test-with-docker. In additional to the ulimit issue, I ran into the fact that /tmp needed 1777 permissions for the postgresql socket, and entrypoint.sh had a few places that needed special cases. At the moment, the data load ran fine, as did most of the tests. I observed a test that relied on a python2.7-ism fail, which is part of the point of this. In the course of development, I encountered a handful of tests fail with "Encounter parse error: failed to open /usr/share/zoneinfo/GMT-08:00 - No such file or directory.", which was reproduced as follows: [localhost:21000] default> use functional_orc_def; select * from alltypes; ... WARNINGS: Encounter parse error: failed to open /usr/share/zoneinfo/GMT-08:00 - No such file or directory. With Quanlong's help, I learned what was happening. test-with-docker was translating my time zone (America/Los_Angeles) to US/Pacific-New, because realpath(/etc/localtime) = US/Pacific-New. This timezone exists in centos:6, so that wasn't a problem. However, this timezone does not exist in the package "tzdata-java", which is the copy of the timezone information used by Java. (There are bugs here that may have been fixed in centos:7.) As a result, when ORC asks (by using TimeZone.getDefault().getID()) the JDK (src/solaris/native/java/util/TimeZone_md.c) for the default timezone, it can't find the same name as /etc/localtime points to in its repository and defaults to "GMT-08:00". This string then gets written into the ORC files generated by Hive as part of data load, and then the C++ library can't read them. This is fixed by changing "realpath" to "readlink" in test-with-docker.py. centos:7 is not addressed by this change. The move to systemd makes "service sshd start" (and the same for postgresql) not work, and additional care needs to be done to work around that. This change is a joint effort with Laszlo Gaal. Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 --- M bin/bootstrap_system.sh M docker/entrypoint.sh M docker/test-with-docker.py 3 files changed, 167 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/11731/5 -- To view, visit http://gerrit.cloudera.org:8080/11731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id54294d7607f51de87a9de373dcfc4a33f4bedf5 Gerrit-Change-Number: 11731 Gerrit-PatchSet: 5 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jim Apple Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Quanlong Huang
