[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 03:56:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7141 (part 1): clean up handling of default/dummy partition
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10711 ) Change subject: IMPALA-7141 (part 1): clean up handling of default/dummy partition .. Patch Set 4: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2699/ -- To view, visit http://gerrit.cloudera.org:8080/10711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15e91b50eb7c2a5e0bac8c33d603d6cd8cbaca2e Gerrit-Change-Number: 10711 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 19 Jun 2018 03:20:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7140 (part 2). Create skeleton for LocalFsTable
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10712 ) Change subject: IMPALA-7140 (part 2). Create skeleton for LocalFsTable .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2703/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10712 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2b184104d92e128250df5a08ac7ffb3dde011a8 Gerrit-Change-Number: 10712 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 19 Jun 2018 03:20:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7140 (part 1). Support fetching schema info in LocalCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10630 ) Change subject: IMPALA-7140 (part 1). Support fetching schema info in LocalCatalog .. Patch Set 9: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I640f27e36198955e057da62a3ce25a858406e496 Gerrit-Change-Number: 10630 Gerrit-PatchSet: 9 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 19 Jun 2018 03:15:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7141 (part 2). Extract interfaces for partition pruning without loading
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10631 ) Change subject: IMPALA-7141 (part 2). Extract interfaces for partition pruning without loading .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2702/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10631 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifdcfd6cffcd298bbf44531e1ec2f47c3a5b7d1fa Gerrit-Change-Number: 10631 Gerrit-PatchSet: 9 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Tue, 19 Jun 2018 03:15:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 03:02:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value The value is chosen to allow only queries that have a reasonable chance of succeeding, albeit with poor performance because of the high number of threads. Testing: Added a test to make sure that the default value rejects a large query. Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Reviewed-on: http://gerrit.cloudera.org:8080/10628 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M common/thrift/ImpalaInternalService.thrift M testdata/workloads/functional-query/queries/QueryTest/thread-limits.test 2 files changed, 24 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10493 ) Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:49:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10493 ) Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ IMPALA-110 will involve refactoring PartitionedAggregationNode by separating out the aggregation logic into a new type called Aggregator, and then supporting multiple Aggregators per node to allow for multiple aggregation classes to be evaluated at the same time. Each Aggregator will need to have its own memory reservation to operate, and we can do this by giving each Aggregator its own BufferPool::ClientHandle instead of using the usual ExecNode::buffer_pool_client_. To facilitate this, this patch refactors all of the buffer_pool_client_ related logic into a new class, ReservationManager, so that eventually each Aggregator can have its own ReservationManager and the logic in ClaimBufferReservation(), ReleaseUnusedReservation(), etc. won't be duplicated. Testing: - Passed a full run of the core tests. Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Reviewed-on: http://gerrit.cloudera.org:8080/10493 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/analytic-eval-node.cc M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/hdfs-scan-node.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partitioned-aggregation-node.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/sort-node.cc M be/src/runtime/CMakeLists.txt A be/src/runtime/reservation-manager.cc A be/src/runtime/reservation-manager.h 13 files changed, 288 insertions(+), 164 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 6: Code-Review+2 Will do a dry run while waiting for Bikram's feedback -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:32:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 5: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/2700/ -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:31:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 6: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2701/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:32:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2700/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:31:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-1760: Implement shutdown command
Tim Armstrong has uploaded a new patch set (#6). (
http://gerrit.cloudera.org:8080/10744 )
Change subject: IMPALA-1760: Implement shutdown command
..
IMPALA-1760: Implement shutdown command
This allows graceful shutdown of executors and partially graceful
shutdown of coordinators (new operations fail, old operations can
continue).
Details:
* In order to allow future admin commands, this is implemented with
function-like syntax and does not add any reserved words.
* ALL privilege is required on the server
* The coordinator impalad that the client is connected to can be shut
down directly with ":shutdown()".
* Remote shutdown of another impalad is supported, e.g. with
":shutdown('hostname')", so that non-coordinators can be shut down
and for the convenience of the client, which does not have to
connect to the specific impalad. There is no assumption that the
other impalad is registered in the statestore; just that the
coordinator can connect to the other daemon's thrift endpoint.
This simplifies things and allows shutdown in various important
cases, e.g. statestore down.
* The shutdown time limit can be overridden to force a quicker or
slower shutdown.
* If shutting down, a banner is shown on the root debug page.
Workflow:
1. (if a coordinator) clients are prevented from submitting
queries to this coordinator via some out-of-band mechanism,
e.g. load balancer
2. the shutdown process is started via ":shutdown()"
3. a bit is set in the statestore and propagated to coordinators,
which stop scheduling fragment instances on this daemon
(if an executor).
4. the quiesce period (which is the AC queueing delay plus
some additional leeway) expires
5. once the daemon is drained (i.e. no fragments, no registered
queries), it shuts itself down.
6. If the daemon does not drain (e.g. rogue clients, long-running
queries), after a longer timeout it will shut down anyway.
What this does do:
* Executors can be shut down without causing a service-wide outage
* Shutting down an executor will not disrupt any short-running queries
and will wait for long-running queries up to a threshold.
* Coordinators can be shut down without query failures only if
there is an out-of-band mechanism to prevent submission of more
queries to the shut down coordinator.
* Long running queries or other issues (e.g. stuck fragments) will
slow down but not prevent eventual shutdown.
Limitations:
* The quiesce period needs to be configured to be greater than the
latency of statestore updates + scheduling + admission +
coordinator startup. Otherwise a coordinator may send a
fragment instance to the shutting down impalad. (We could
automate this configuration as a follow-on)
* The quiesce period means a minimum latency for shutdown,
even if the cluster is idle.
* We depend on the statestore detecting the process going down
if queries are still running on that backend when the timeout
expires. This may still be subject to existing problems,
e.g. IMPALA-2990.
Tests:
* Added parser, analysis and authorization tests.
* End-to-end test of shutting down impalads.
* End-to-end test of shutting down then restarting an executor while
queries are running.
* End-to-end test of shutting down a coordinator
- New queries cannot be started on coord, existing queries continue to run
- Exercises various Beeswax and HS2 operations.
Change-Id: I4d5606ccfec84db4482c1e7f0f198103aad141a0
---
M be/src/runtime/backend-client.h
M be/src/runtime/data-stream-test.cc
M be/src/scheduling/scheduler.cc
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-internal-service.cc
M be/src/service/impala-internal-service.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/testutil/fault-injection-util.h
M be/src/util/default-path-handlers.cc
M common/thrift/Frontend.thrift
M common/thrift/ImpalaInternalService.thrift
M common/thrift/StatestoreService.thrift
M common/thrift/Types.thrift
M common/thrift/generate_error_codes.py
M fe/src/main/cup/sql-parser.cup
A fe/src/main/java/org/apache/impala/analysis/AdminFnStmt.java
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/LimitElement.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationTestV2.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M tests/common/impala_cluster.py
M tests/common/impala_service.py
M tests/custom_cluster/test_restart_services.py
M tests/hs2/hs2_test_suite.py
M tests/hs2/test_fetch_first.py
M www/backends.tmpl
M
[Impala-ASF-CR] IMPALA-2746: part 1: enable LSAN for many backend tests
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10668 ) Change subject: IMPALA-2746: part 1: enable LSAN for many backend tests .. IMPALA-2746: part 1: enable LSAN for many backend tests This turns on leak sanitizer for backend tests that required relatively small modifications to pass. We suppress a few leaks, mainly related to the embedded JVM. Testing: Ran core tests under ASAN. Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Reviewed-on: http://gerrit.cloudera.org:8080/10668 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/CMakeLists.txt M be/src/catalog/CMakeLists.txt M be/src/codegen/CMakeLists.txt M be/src/codegen/instruction-counter-test.cc M be/src/common/CMakeLists.txt M be/src/common/atomic-test.cc M be/src/common/atomic.h M be/src/exec/CMakeLists.txt M be/src/experiments/CMakeLists.txt M be/src/exprs/CMakeLists.txt M be/src/exprs/expr-test.cc M be/src/rpc/CMakeLists.txt M be/src/runtime/CMakeLists.txt M be/src/runtime/bufferpool/CMakeLists.txt M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/data-stream-test.cc M be/src/runtime/io/CMakeLists.txt M be/src/runtime/io/disk-io-mgr-stress.cc M be/src/runtime/io/disk-io-mgr-stress.h M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/mem-tracker.cc M be/src/scheduling/CMakeLists.txt M be/src/service/CMakeLists.txt M be/src/statestore/CMakeLists.txt M be/src/udf/udf.cc M be/src/util/CMakeLists.txt M be/src/util/decompress-test.cc A bin/lsan-suppressions.txt 28 files changed, 297 insertions(+), 242 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Gerrit-Change-Number: 10668 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-2746: part 1: enable LSAN for many backend tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10668 ) Change subject: IMPALA-2746: part 1: enable LSAN for many backend tests .. Patch Set 10: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Gerrit-Change-Number: 10668 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:23:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Reviewed-on: http://gerrit.cloudera.org:8080/10751 Reviewed-by: Alex Rodoni Reviewed-by: Michael Ho Tested-by: Impala Public Jenkins --- M docs/topics/impala_known_issues.xml 1 file changed, 35 insertions(+), 1 deletion(-) Approvals: Alex Rodoni: Looks good to me, approved Michael Ho: Looks good to me, but someone else must approve Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 5: Code-Review+2 Bikram, could you take another quick look to make sure you agree with the new wording? -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 19 Jun 2018 00:00:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Hello Bikramjeet Vig, Dan Hecht, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10731 to look at the new patch set (#5). Change subject: IMPALA-6969: add AC last queued cause to profile .. IMPALA-6969: add AC last queued cause to profile The reason is updated during initial admission and when the query is at the head of the queue but can't be admitted. It is not updated while the query is in the middle of the queue. Together with the async admission change, this makes it possible to determine from the profile why the query has not been admitted yet. Testing: Added admission control tests that check that the string is set for queries queued based both on the query count and the max memory. Looped the tests overnight to confirm non-flakiness. Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M tests/custom_cluster/test_admission_controller.py 3 files changed, 147 insertions(+), 18 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/10731/5 -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc@110 PS3, Line 110: Admission queue details" > that sounds pretty good to me (maybe add "admission" in there). Done -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:53:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7141 (part 1): clean up handling of default/dummy partition
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10711 ) Change subject: IMPALA-7141 (part 1): clean up handling of default/dummy partition .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2699/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10711 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15e91b50eb7c2a5e0bac8c33d603d6cd8cbaca2e Gerrit-Change-Number: 10711 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 18 Jun 2018 23:49:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7140 (part 1). Support fetching schema info in LocalCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10630 ) Change subject: IMPALA-7140 (part 1). Support fetching schema info in LocalCatalog .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2698/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I640f27e36198955e057da62a3ce25a858406e496 Gerrit-Change-Number: 10630 Gerrit-PatchSet: 9 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 23:48:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6119: Fix issue with multiple partitions sharing same location
Bharath Vissapragada has posted comments on this change. (
http://gerrit.cloudera.org:8080/10543 )
Change subject: IMPALA-6119: Fix issue with multiple partitions sharing same
location
..
Patch Set 12:
(4 comments)
Looks pretty close, just some clarifying questions.
http://gerrit.cloudera.org:8080/#/c/10543/12/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:
http://gerrit.cloudera.org:8080/#/c/10543/12/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1227
PS12, Line 1227: partitions != null
Isn't this more like a precondition? Also you could just say
return partitions.size() > 1;
http://gerrit.cloudera.org:8080/#/c/10543/12/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/10543/12/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2245
PS12, Line 2245:
((HdfsTable)tbl).isLocationSharedWithOtherPartitions(part)) {
Should we check if the table is "managed" / "external" in Hive? I'm guessing
this only applies to managed tables since HMS can't drop data partitions for
external tables.
http://gerrit.cloudera.org:8080/#/c/10543/12/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2247
PS12, Line 2247:
Could you format the string to add the partition data here.
Also should we include the the following information?
- Location that is causing the conflict
- List and count(or atleast a subset of them) of conflicting partitions?
Joiner.On(list.subset())...
Seeing this error message, the next steps for the user would be to probably
figure out the conflicting partitions, may be we should include that
information?
http://gerrit.cloudera.org:8080/#/c/10543/12/tests/metadata/test_partition_metadata.py
File tests/metadata/test_partition_metadata.py:
http://gerrit.cloudera.org:8080/#/c/10543/12/tests/metadata/test_partition_metadata.py@206
PS12, Line 206: self.client.execute("insert into table %s partition(j=3)
select 3" % FQ_TBL_NAME)
May be just call refresh instead?
--
To view, visit http://gerrit.cloudera.org:8080/10543
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a54bc8224bcefe65b83de2df58bb84629f2aa4a
Gerrit-Change-Number: 10543
Gerrit-PatchSet: 12
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Bharath Vissapragada
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Sailesh Mukil
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Reviewer: Zoltan Borok-Nagy
Gerrit-Comment-Date: Mon, 18 Jun 2018 23:46:39 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-110 (part 2): Refactor PartitionedAggregationNode
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/10394 ) Change subject: IMPALA-110 (part 2): Refactor PartitionedAggregationNode .. Patch Set 2: > > I think that's probably unnecessary (hopefully this shouldn't be > > too terrible to review, since the logic is entirely unchanged), > > But it sounds like there is some changes in here that aren't purely > code motion. i.e. some cleanups are in here too. so what i'm really > wondering is how can we focus the review on those parts? Maybe you > just want to add comments to indicate which code isn't a straight > copy (and class rename)? Or some other way to highlight the more > interesting parts? There's not really much cleanup, just the changes absolutely necessary to make everything work, eg. defining the Aggregator interface. All of the interesting changes are in the top commit of the branch I posted (https://github.com/twmarshall/impala/commits/agg-refactor). Happy to submit that as a review, or otherwise stage this however you think is most convenient for review. -- To view, visit http://gerrit.cloudera.org:8080/10394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e7bb583f54aa4add3738bde7f57cf3511ac567e Gerrit-Change-Number: 10394 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 23:43:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/324/ -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 18 Jun 2018 23:38:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2697/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:35:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:35:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:34:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. Patch Set 2: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 18 Jun 2018 23:34:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10747 ) Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. Patch Set 6: Code-Review+1 Thanks. Csaba--do you have any more comments? Feel free to GVD once you're happy with it. -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 6 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 18 Jun 2018 23:25:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 18 Jun 2018 23:24:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. Patch Set 1: Fixed a nesting error -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 18 Jun 2018 23:23:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Hello Michael Ho, Sailesh Mukil, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10751 to look at the new patch set (#2). Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 --- M docs/topics/impala_known_issues.xml 1 file changed, 35 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/10751/2 -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc@110 PS3, Line 110: Admission queue details" > I also had the same concern. I didn't want to change the strings for no goo that sounds pretty good to me (maybe add "admission" in there). -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:22:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7151: Rework ephemeral port assignment for be tests
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10740 ) Change subject: IMPALA-7151: Rework ephemeral port assignment for be tests .. Patch Set 2: This is stacked on top of https://gerrit.cloudera.org/#/c/10726/ -- To view, visit http://gerrit.cloudera.org:8080/10740 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fcc9243099f8249380ac5f01ac5eb67cba24cf5 Gerrit-Change-Number: 10740 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Sailesh Mukil Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:20:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10742 ) Change subject: IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD .. Patch Set 2: Code-Review+1 I'm a little late to this party, but thought I would +1. -- To view, visit http://gerrit.cloudera.org:8080/10742 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9452f7e44c7679f86a947cd20115c078757223d8 Gerrit-Change-Number: 10742 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:19:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc@110 PS3, Line 110: Admission queue details" > so the reason given in this entry will always be the initial reason, is tha I also had the same concern. I didn't want to change the strings for no good reason, but maybe "Initial queue cause" would be clearer? http://gerrit.cloudera.org:8080/#/c/10731/3/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/10731/3/tests/custom_cluster/test_admission_controller.py@604 PS3, Line 604: EXPECTED_CAUSE = \ > for these tests, do you think it's also worth verifying that the initial re Yeah probably is. I added checks for that. I'm not validating the full message, just the general cause. -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 23:13:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Hello Bikramjeet Vig, Dan Hecht, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10731 to look at the new patch set (#4). Change subject: IMPALA-6969: add AC last queued cause to profile .. IMPALA-6969: add AC last queued cause to profile The reason is updated during initial admission and when the query is at the head of the queue but can't be admitted. It is not updated while the query is in the middle of the queue. Together with the async admission change, this makes it possible to determine from the profile why the query has not been admitted yet. Testing: Added admission control tests that check that the string is set for queries queued based both on the query count and the max memory. Looped the tests overnight to confirm non-flakiness. Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M tests/custom_cluster/test_admission_controller.py 3 files changed, 139 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/10731/4 -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7046: introduce "global" debug actions
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10690 ) Change subject: IMPALA-7046: introduce "global" debug_actions .. Patch Set 8: i think this is ready for review again. -- To view, visit http://gerrit.cloudera.org:8080/10690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 Gerrit-Change-Number: 10690 Gerrit-PatchSet: 8 Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Comment-Date: Mon, 18 Jun 2018 23:04:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7171: [DOCS] Hints for Kudu insert and upsert
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10737 ) Change subject: IMPALA-7171: [DOCS] Hints for Kudu insert and upsert .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04378e6f2b17d4d6e844192807d946b9045e2927 Gerrit-Change-Number: 10737 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 18 Jun 2018 22:42:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7171: [DOCS] Hints for Kudu insert and upsert
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10737 ) Change subject: IMPALA-7171: [DOCS] Hints for Kudu insert and upsert .. IMPALA-7171: [DOCS] Hints for Kudu insert and upsert Change-Id: I04378e6f2b17d4d6e844192807d946b9045e2927 Reviewed-on: http://gerrit.cloudera.org:8080/10737 Reviewed-by: Thomas Marshall Tested-by: Impala Public Jenkins --- M docs/shared/impala_common.xml M docs/topics/impala_hints.xml M docs/topics/impala_kudu.xml 3 files changed, 31 insertions(+), 15 deletions(-) Approvals: Thomas Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I04378e6f2b17d4d6e844192807d946b9045e2927 Gerrit-Change-Number: 10737 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-6802 (part 4): Clean up authorization tests
Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10442 ) Change subject: IMPALA-6802 (part 4): Clean up authorization tests .. IMPALA-6802 (part 4): Clean up authorization tests The fourth part of this patch is to rewrite the following authorization tests: - describe Testing: - Added new authorization tests - Ran all front-end tests Cherry-picks: not for 2.x Change-Id: Ic4cf3a4751b41908ef81ec35d89a2713d9fa0dc4 Reviewed-on: http://gerrit.cloudera.org:8080/10442 Tested-by: Impala Public Jenkins Reviewed-by: Vuk Ercegovac --- M fe/src/test/java/org/apache/impala/analysis/AuthorizationTestV2.java 1 file changed, 279 insertions(+), 57 deletions(-) Approvals: Impala Public Jenkins: Verified Vuk Ercegovac: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/10442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic4cf3a4751b41908ef81ec35d89a2713d9fa0dc4 Gerrit-Change-Number: 10442 Gerrit-PatchSet: 6 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7180: Pin Impala CDH dependencies
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10748 Change subject: IMPALA-7180: Pin Impala CDH dependencies .. IMPALA-7180: Pin Impala CDH dependencies For profile=3 Pin the CDH dependencies by storing the CDH tarballs and Maven repository in S3. This solves the issue of build coherency between between the the CDH tarballs and Maven dependencies. For profile=2 Pin the CDH dependencies by storing only the CDH tarballs in S3. The Maven repository will still use https://repositories.cloudera.com. So there is still a possibility of a build coherency issue. For each CDH dependency, there is a global build number in each repository URL to indicate the build number that created those CDH dependencies. This informaton can be useful for debugging issues related to CDH dependencies. This patch introduces CDH_HOST and CDH_BUILD_NUMBER environment variables that can be overriden, which can be useful for running an integration job. This patch also fixes dependency issue in Hadoop that transitively depends on snapshot versions of dependencies that no longer exist. See HADOOP-14903. Testing: - Ran all core tests on profile=2 and profile=3 Cherry-picks: not for 2.x Change-Id: I66c0dcb8abdd0d187490a761f129cda3b3500990 --- M bin/bootstrap_toolchain.py M bin/impala-config.sh M common/yarn-extras/pom.xml M fe/pom.xml M impala-parent/pom.xml M testdata/pom.xml M tests/test-hive-udfs/pom.xml 7 files changed, 283 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/10748/2 -- To view, visit http://gerrit.cloudera.org:8080/10748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I66c0dcb8abdd0d187490a761f129cda3b3500990 Gerrit-Change-Number: 10748 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-6802 (part 4): Clean up authorization tests
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10442 ) Change subject: IMPALA-6802 (part 4): Clean up authorization tests .. Patch Set 5: Seems ok to go ahead and merge. There haven't been any related commits that have gone in since you ran the tests. -- To view, visit http://gerrit.cloudera.org:8080/10442 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic4cf3a4751b41908ef81ec35d89a2713d9fa0dc4 Gerrit-Change-Number: 10442 Gerrit-PatchSet: 5 Gerrit-Owner: Adam Holley Gerrit-Reviewer: Adam Holley Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 22:38:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3307: Add support for IANA time-zone db
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9986 ) Change subject: IMPALA-3307: Add support for IANA time-zone db .. Patch Set 19: Are we ready to go ahead and merge? Would be good to run exhaustive tests + ASAN before merging just to be sure we aren't going to break anything. -- To view, visit http://gerrit.cloudera.org:8080/9986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93c1fbffe81f067919706e30db0a34d0e58e7e77 Gerrit-Change-Number: 9986 Gerrit-PatchSet: 19 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Mon, 18 Jun 2018 22:35:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7171: [DOCS] Hints for Kudu insert and upsert
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10737 ) Change subject: IMPALA-7171: [DOCS] Hints for Kudu insert and upsert .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-docs-submit/323/ -- To view, visit http://gerrit.cloudera.org:8080/10737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04378e6f2b17d4d6e844192807d946b9045e2927 Gerrit-Change-Number: 10737 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 18 Jun 2018 22:32:40 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/10747 ) Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. Patch Set 6: (2 comments) Good catch. http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py File tests/shell/test_shell_commandline.py: http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py@626 PS3, Line 626: if impala_shell.poll() is None: > This can actually also throw an exception, if the process already exited fo Done http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py@627 PS3, Line 627: impala_shell.kill() > connection should be None checked. Done -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 6 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 18 Jun 2018 22:16:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Hello Philip Zeyliger, Csaba Ringhofer, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10747 to look at the new patch set (#6). Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening test_shell_commandline.py::TestImpalaShell::test_socket_opening uses netcat to listen to an ephemeral port to verify the expected socket opening behavior of impala-shell. This port number is fixed to 42000. When this port happens to be used by another outbound socket, this test will fail. This change refactors the test to use socket.bind(). The port used in this test is no longer fixed and will be picked automatically. This change also adds the proper cleanup logics to the various subprocess.Popen objects used in the test. Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 --- M tests/shell/test_shell_commandline.py 1 file changed, 38 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/10747/6 -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 6 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/10751 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Mon, 18 Jun 2018 22:14:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7046: introduce "global" debug actions
Hello Bikramjeet Vig, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10690 to look at the new patch set (#8). Change subject: IMPALA-7046: introduce "global" debug_actions .. IMPALA-7046: introduce "global" debug_actions The motivation is to add jitter to backend startup in test_failpoints. The race in IMPALA-7033 can be reproduced by adding jitter to the exec rpcs when some backends fail. Let's add jitter to test_failpoints to get better coverage of exec startup races. This builds on top of the debug action extensions added in the async admission control patch by allowing the new "global" debug actions (i.e. actions that can be used in points outside of the ExecNodes). See the code comments for details. For now, we're only using the SLEEP and JITTER commands, but I've included a FAIL command as well since I'll want to use that to write a test for IMPALA-6788 to simulate exec rpc failure. Note that I don't bother resolving the actions ahead of time (like we do for ExecNode actions). It doesn't seem worth it since the resolution only needs to occur after we've matched the label and I don't expect the same label to be hit many times within a single thread. We can always optimize this later if needed. Instead, just continue compiling it out of Release builds. Testing: - Verified that test_failpoints can reproduce the race in IMPALA-7033 by reverting that fix and testing. - Ran the modified tests and grepped the impalad log to see that the sleeps are still occuring. - Manually verify global FAIL command (in a build with another patch). Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 --- M be/src/runtime/coordinator.cc M be/src/runtime/debug-options.cc M be/src/scheduling/admission-controller.cc M be/src/service/client-request-state.cc M be/src/util/debug-util.cc M be/src/util/debug-util.h M common/thrift/ImpalaService.thrift M tests/custom_cluster/test_admission_controller.py M tests/failure/test_failpoints.py M tests/query_test/test_observability.py 10 files changed, 191 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/10690/8 -- To view, visit http://gerrit.cloudera.org:8080/10690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 Gerrit-Change-Number: 10690 Gerrit-PatchSet: 8 Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10628 ) Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/10628/1/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/10628/1/common/thrift/ImpalaInternalService.thrift@287 PS1, Line 287: 3000 > it'd be good to include some explanation as to how this was chosen. Done http://gerrit.cloudera.org:8080/#/c/10628/1/testdata/workloads/functional-query/queries/QueryTest/thread-limits.test File testdata/workloads/functional-query/queries/QueryTest/thread-limits.test: http://gerrit.cloudera.org:8080/#/c/10628/1/testdata/workloads/functional-query/queries/QueryTest/thread-limits.test@109 PS1, Line 109: SELECT count(*) from functional.alltypes > When I am trying to create a plan with crazy amount of nodes, I use WITH + That's a much better idea, thanks -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:53:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7115: set a default THREAD RESERVATION LIMIT value
Hello Csaba Ringhofer, Dan Hecht, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10628 to look at the new patch set (#3). Change subject: IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value .. IMPALA-7115: set a default THREAD_RESERVATION_LIMIT value The value is chosen to allow only queries that have a reasonable chance of succeeding, albeit with poor performance because of the high number of threads. Testing: Added a test to make sure that the default value rejects a large query. Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 --- M common/thrift/ImpalaInternalService.thrift M testdata/workloads/functional-query/queries/QueryTest/thread-limits.test 2 files changed, 24 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/10628/3 -- To view, visit http://gerrit.cloudera.org:8080/10628 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I31d3fa3f6305c360922649dba53a9026c9563384 Gerrit-Change-Number: 10628 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Dan Hecht
[Impala-ASF-CR] IMPALA-7119: Restart whole minicluster when HDFS replication stalls
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10665 ) Change subject: IMPALA-7119: Restart whole minicluster when HDFS replication stalls .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58ffe301708c78c26ee61aa754a06f46c224c6e2 Gerrit-Change-Number: 10665 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 21:46:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7119: Restart whole minicluster when HDFS replication stalls
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10665 ) Change subject: IMPALA-7119: Restart whole minicluster when HDFS replication stalls .. IMPALA-7119: Restart whole minicluster when HDFS replication stalls After loading data, we wait for HDFS to replicate all of the blocks appropriately. If this takes too long, we restart HDFS. However, HBase can fail if HDFS is restarted and HBase is unable to write its logs. In general, there is no real reason to keep HBase and the other minicluster components running while restarting HDFS. This changes the HDFS health check to restart the whole minicluster and Impala rather than just HDFS. Testing: - Tested with a modified version that always does the restart in the HDFS health check and verified that the tests pass Change-Id: I58ffe301708c78c26ee61aa754a06f46c224c6e2 Reviewed-on: http://gerrit.cloudera.org:8080/10665 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M testdata/bin/create-load-data.sh 1 file changed, 29 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I58ffe301708c78c26ee61aa754a06f46c224c6e2 Gerrit-Change-Number: 10665 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10751 Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed .. IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 --- M docs/topics/impala_known_issues.xml 1 file changed, 33 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/10751/1 -- To view, visit http://gerrit.cloudera.org:8080/10751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9db28d42fccd9711635c6aee66f2aafc758d58d0 Gerrit-Change-Number: 10751 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7161: Fix impala-config.sh's handling of JAVA HOME
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10702 ) Change subject: IMPALA-7161: Fix impala-config.sh's handling of JAVA_HOME .. IMPALA-7161: Fix impala-config.sh's handling of JAVA_HOME It is common for developers to specify JAVA_HOME in bin/impala-config-local.sh, so wait until after it is sourced to validate JAVA_HOME. Also, try harder to auto-detect the system's JAVA_HOME in case it has not been specified in the environment. Here is a run through of different scenarios: 1. Not set in environment, not set in impala-config-local.sh: Didn't work before, now tries to autodetect by looking for javac on the PATH 2. Set in environment, not set in impala-config-local.sh: No change 3. Not set in environment, set in impala-config-local.sh: Didn't work before, now works 4. Set in environment and set in impala-config-local.sh: This used to be potentially inconsistent (i.e. JAVA comes from the environment's JAVA_HOME, but JAVA_HOME is overwritten by impala-config-local.sh), now it always uses the value from impala-config-local.sh. Change-Id: Idf3521b4f44fdbdc841a90fd00c477c9423a75bb Reviewed-on: http://gerrit.cloudera.org:8080/10702 Reviewed-by: Philip Zeyliger Tested-by: Impala Public Jenkins --- M bin/impala-config.sh M docker/entrypoint.sh 2 files changed, 26 insertions(+), 12 deletions(-) Approvals: Philip Zeyliger: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10702 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Idf3521b4f44fdbdc841a90fd00c477c9423a75bb Gerrit-Change-Number: 10702 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang
[Impala-ASF-CR] IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed by default
Alex Rodoni has abandoned this change. ( http://gerrit.cloudera.org:8080/10750 ) Change subject: IMPALA-7182: [DOCS] Insecure clusters with public IPs not allowed by default .. Abandoned -- To view, visit http://gerrit.cloudera.org:8080/10750 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I918c3315845f99ae575e02ca4829560b62187af3 Gerrit-Change-Number: 10750 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7161: Fix impala-config.sh's handling of JAVA HOME
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10702 ) Change subject: IMPALA-7161: Fix impala-config.sh's handling of JAVA_HOME .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10702 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf3521b4f44fdbdc841a90fd00c477c9423a75bb Gerrit-Change-Number: 10702 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 21:35:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10493 ) Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. Patch Set 5: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2696/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:30:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10493 ) Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:30:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-110 (part 2): Refactor PartitionedAggregationNode
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10394 ) Change subject: IMPALA-110 (part 2): Refactor PartitionedAggregationNode .. Patch Set 2: > I think that's probably unnecessary (hopefully this shouldn't be > too terrible to review, since the logic is entirely unchanged), But it sounds like there is some changes in here that aren't purely code motion. i.e. some cleanups are in here too. so what i'm really wondering is how can we focus the review on those parts? Maybe you just want to add comments to indicate which code isn't a straight copy (and class rename)? Or some other way to highlight the more interesting parts? -- To view, visit http://gerrit.cloudera.org:8080/10394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e7bb583f54aa4add3738bde7f57cf3511ac567e Gerrit-Change-Number: 10394 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 21:19:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 3: Code-Review+2 (2 comments) http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc File be/src/scheduling/admission-controller.cc: http://gerrit.cloudera.org:8080/#/c/10731/3/be/src/scheduling/admission-controller.cc@110 PS3, Line 110: Admission queue details" so the reason given in this entry will always be the initial reason, is that right? Will it be clear what this entry means vs the new entry? http://gerrit.cloudera.org:8080/#/c/10731/3/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/10731/3/tests/custom_cluster/test_admission_controller.py@604 PS3, Line 604: EXPECTED_CAUSE = \ for these tests, do you think it's also worth verifying that the initial reason is as expected? -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:16:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10493 ) Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:05:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 3: Carry +1 -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 3 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:05:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/10731/1/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/10731/1/be/src/scheduling/admission-controller.h@485 PS1, Line 485: queueing > nit: dequeueing Done http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py@614 PS1, Line 614: assert num_causes == NUM_QUERIES - 1, \ : "All queries except first should have been queued: " + '\n===\n'.join(profiles) > nit: repeated assert Done http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py@627 PS1, Line 627: # > nit: remove '#' Done -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:05:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Hello Bikramjeet Vig, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10731 to look at the new patch set (#2). Change subject: IMPALA-6969: add AC last queued cause to profile .. IMPALA-6969: add AC last queued cause to profile The reason is updated during initial admission and when the query is at the head of the queue but can't be admitted. It is not updated while the query is in the middle of the queue. Together with the async admission change, this makes it possible to determine from the profile why the query has not been admitted yet. Testing: Added admission control tests that check that the string is set for queries queued based both on the query count and the max memory. Looped the tests overnight to confirm non-flakiness. Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 --- M be/src/scheduling/admission-controller.cc M be/src/scheduling/admission-controller.h M tests/custom_cluster/test_admission_controller.py 3 files changed, 117 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/10731/2 -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig
[Impala-ASF-CR] IMPALA-2746: part 1: enable LSAN for many backend tests
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10668 ) Change subject: IMPALA-2746: part 1: enable LSAN for many backend tests .. Patch Set 9: Ran ASAN tests one more time to make sure that everything still worked. -- To view, visit http://gerrit.cloudera.org:8080/10668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Gerrit-Change-Number: 10668 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:02:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2746: part 1: enable LSAN for many backend tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10668 ) Change subject: IMPALA-2746: part 1: enable LSAN for many backend tests .. Patch Set 10: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Gerrit-Change-Number: 10668 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:02:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-2746: part 1: enable LSAN for many backend tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10668 ) Change subject: IMPALA-2746: part 1: enable LSAN for many backend tests .. Patch Set 10: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2695/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10668 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibdda092a4eb4bc827c75a8c121e5428ec746b7f4 Gerrit-Change-Number: 10668 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 21:02:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/10747 ) Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py File tests/shell/test_shell_commandline.py: http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py@626 PS3, Line 626: impala_shell.kill() This can actually also throw an exception, if the process already exited for some reason. http://gerrit.cloudera.org:8080/#/c/10747/3/tests/shell/test_shell_commandline.py@627 PS3, Line 627: connection.close() connection should be None checked. -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 3 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 18 Jun 2018 20:59:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7046: introduce "global" debug actions
Dan Hecht has posted comments on this change. ( http://gerrit.cloudera.org:8080/10690 ) Change subject: IMPALA-7046: introduce "global" debug_actions .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/10690/6/be/src/util/debug-util.cc File be/src/util/debug-util.cc: http://gerrit.cloudera.org:8080/#/c/10690/6/be/src/util/debug-util.cc@374 PS6, Line 374: 2 > that should be 1 (I messed it up when refactoring). Done -- To view, visit http://gerrit.cloudera.org:8080/10690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 Gerrit-Change-Number: 10690 Gerrit-PatchSet: 6 Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht Gerrit-Comment-Date: Mon, 18 Jun 2018 20:53:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7046: introduce "global" debug actions
Hello Bikramjeet Vig, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10690 to look at the new patch set (#7). Change subject: IMPALA-7046: introduce "global" debug_actions .. IMPALA-7046: introduce "global" debug_actions The motivation is to add jitter to backend startup in test_failpoints. The race in IMPALA-7033 can be reproduced by adding jitter to the exec rpcs when some backends fail. Let's add jitter to test_failpoints to get better coverage of exec startup races. This builds on top of the debug action extensions added in the async admission control patch by allowing the new "global" debug actions (i.e. actions that can be used in points outside of the ExecNodes). See the code comments for details. For now, we're only using the SLEEP and JITTER commands, but I've included a FAIL command as well since I'll want to use that to write a test for IMPALA-6788 to simulate exec rpc failure. Note that I don't bother resolving the actions ahead of time (like we do for ExecNode actions). It doesn't seem worth it since the resolution only needs to occur after we've matched the label and I don't expect the same label to be hit many times within a single thread. We can always optimize this later if needed. Instead, just continue compiling it out of Release builds. Testing: - Verified that test_failpoints can reproduce the race in IMPALA-7033 by reverting that fix and testing. - Ran the modified tests and grepped the impalad log to see that the sleeps are still occuring. - Manually verify global FAIL command (in a build with another patch). Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 --- M be/src/runtime/coordinator.cc M be/src/runtime/debug-options.cc M be/src/scheduling/admission-controller.cc M be/src/service/client-request-state.cc M be/src/util/debug-util.cc M be/src/util/debug-util.h M common/thrift/ImpalaService.thrift M tests/custom_cluster/test_admission_controller.py M tests/failure/test_failpoints.py M tests/query_test/test_observability.py 10 files changed, 191 insertions(+), 69 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/10690/7 -- To view, visit http://gerrit.cloudera.org:8080/10690 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I77663a539be18711a4f12c470ffd7474e3d69388 Gerrit-Change-Number: 10690 Gerrit-PatchSet: 7 Gerrit-Owner: Dan Hecht Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Dan Hecht
[Impala-ASF-CR] IMPALA-7171: [DOCS] Hints for Kudu insert and upsert
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/10737 ) Change subject: IMPALA-7171: [DOCS] Hints for Kudu insert and upsert .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04378e6f2b17d4d6e844192807d946b9045e2927 Gerrit-Change-Number: 10737 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Mon, 18 Jun 2018 20:31:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Hello Philip Zeyliger, Csaba Ringhofer, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10747 to look at the new patch set (#5). Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening test_shell_commandline.py::TestImpalaShell::test_socket_opening uses netcat to listen to an ephemeral port to verify the expected socket opening behavior of impala-shell. This port number is fixed to 42000. When this port happens to be used by another outbound socket, this test will fail. This change refactors the test to use socket.bind(). The port used in this test is no longer fixed and will be picked automatically. This change also adds the proper cleanup logics to the various subprocess.Popen objects used in the test. Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 --- M tests/shell/test_shell_commandline.py 1 file changed, 36 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/10747/5 -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 5 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] IMPALA-6812: Fix flaky Kudu scan tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10503 ) Change subject: IMPALA-6812: Fix flaky Kudu scan tests .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/10503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I70df84f2cbc663107f2ad029565d3c15bdfbd47c Gerrit-Change-Number: 10503 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Mon, 18 Jun 2018 20:19:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6812: Fix flaky Kudu scan tests
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10503 ) Change subject: IMPALA-6812: Fix flaky Kudu scan tests .. IMPALA-6812: Fix flaky Kudu scan tests Many of our Kudu related tests have been flaky with the symptom that scans appear to not return rows that were just inserted. This occurs because our default Kudu scan level of READ_LATEST doesn't make any consistency guarantees. This patch adds a query option 'kudu_read_mode', which overrides the startup flag of the same name, and then set that option to READ_AT_SNAPSHOT for all tests with Kudu inserts and scans, which should give us more consistent test results. Testing: - Passed a full exhaustive run. Does not appear to increase time to run by any significant amount. Change-Id: I70df84f2cbc663107f2ad029565d3c15bdfbd47c Reviewed-on: http://gerrit.cloudera.org:8080/10503 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/kudu-scanner.cc M be/src/exec/kudu-util.cc M be/src/exec/kudu-util.h M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/util/debug-util.cc M be/src/util/debug-util.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M tests/common/test_dimensions.py M tests/custom_cluster/test_kudu.py M tests/metadata/test_ddl.py M tests/query_test/test_kudu.py 13 files changed, 119 insertions(+), 8 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I70df84f2cbc663107f2ad029565d3c15bdfbd47c Gerrit-Change-Number: 10503 Gerrit-PatchSet: 6 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: David Ribeiro Alves Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/10747 ) Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. Patch Set 4: Fixed a mistake the test function's block comment. -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 4 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 18 Jun 2018 20:18:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Vincent Tran has posted comments on this change. ( http://gerrit.cloudera.org:8080/10747 ) Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py File tests/shell/test_shell_commandline.py: http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py@621 PS2, Line 621: accept > I think that this will wait forever by default if no one connects to it. se Good point. Done. http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py@624 PS2, Line 624: impala_shell.kill() : connection.close() > These should be moved to a finally block to call them if there is an except Done -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 2 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran Gerrit-Comment-Date: Mon, 18 Jun 2018 20:12:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Hello Philip Zeyliger, Csaba Ringhofer, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10747 to look at the new patch set (#3). Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening test_shell_commandline.py::TestImpalaShell::test_socket_opening uses netcat to listen to an ephmeral port to verify the expected socket opening behavior of impala-shell. This port number is fixed to 42000. When this port happens to be used by another outbound socket, this test will fail. This change refactors the test to use socket.bind(). The port used in this test is no longer fixed and will be picked automatically. This change also adds the proper cleanup logics to the various subprocess.Popen objects used in the test. Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 --- M tests/shell/test_shell_commandline.py 1 file changed, 35 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/10747/3 -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 3 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Vincent Tran
[Impala-ASF-CR] IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/10742 ) Change subject: IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD .. Patch Set 2: Code-Review+2 The flakiness related to IMPALA-7130 is not fixed yet ( it is on review: https://gerrit.cloudera.org/#/c/10747/ ). I just mentioned it to ensure you that it is not related to your change, so a new jenkins verify job may be successful. -- To view, visit http://gerrit.cloudera.org:8080/10742 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9452f7e44c7679f86a947cd20115c078757223d8 Gerrit-Change-Number: 10742 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 20:02:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/10747 )
Change subject: IMPALA-7181: Fix flaky test
shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening
..
Patch Set 2:
(2 comments)
Some socketing concerns, sorry for not noticing them in the first run.
http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py@621
PS2, Line 621: accept
I think that this will wait forever by default if no one connects to it.
setdefaulttimeout should be to set avoid this ("select" or something similar
can be used too, but that will be more complex I guess).
http://gerrit.cloudera.org:8080/#/c/10747/2/tests/shell/test_shell_commandline.py@624
PS2, Line 624: impala_shell.kill()
: connection.close()
These should be moved to a finally block to call them if there is an exception,
for example from the assert.
--
To view, visit http://gerrit.cloudera.org:8080/10747
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44
Gerrit-Change-Number: 10747
Gerrit-PatchSet: 2
Gerrit-Owner: Vincent Tran
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vincent Tran
Gerrit-Comment-Date: Mon, 18 Jun 2018 19:52:34 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-110 (part 2): Refactor PartitionedAggregationNode
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/10394 ) Change subject: IMPALA-110 (part 2): Refactor PartitionedAggregationNode .. Patch Set 2: > > (14 comments) > > > > While developing this patch, I maintained a branch with a series > of > > commits separating the work out into several mostly mechanical > > steps to keep everything straight. > > > > Does each step function on it's own? it might be easier to just > review and commit each step individually (or at least some grouping > of the steps) so that big mechanical changes are easier to review. They don't really make much sense on their own (eg. the first step just copies partitioned-aggregation-node.(h|cc) like 5 times) though since its mostly all in new files it would be easy to not add them to cmake until the final step so that the intermediate steps will still compile and pass tests. I think that's probably unnecessary (hopefully this shouldn't be too terrible to review, since the logic is entirely unchanged), but if there's s desire for that, I would suggest at least that I squash it down to two commits: mechanical changes and then interesting changes, rather than the 6 commits it is now. -- To view, visit http://gerrit.cloudera.org:8080/10394 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e7bb583f54aa4add3738bde7f57cf3511ac567e Gerrit-Change-Number: 10394 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 19:30:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7174: fix test cancellation for RELEASE builds
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10722 ) Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. Patch Set 4: I thought we weren't maintaining the branch. Sent an email to dev@ to clarify -- To view, visit http://gerrit.cloudera.org:8080/10722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 10722 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 19:19:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7174: fix test cancellation for RELEASE builds
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10722 ) Change subject: IMPALA-7174: fix test_cancellation for RELEASE builds .. Patch Set 4: Cherrypick to 2.x? (Not a clean pick) -- To view, visit http://gerrit.cloudera.org:8080/10722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I41da7b5ac58a468a8ed11969906f63df6d4b Gerrit-Change-Number: 10722 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Mon, 18 Jun 2018 19:12:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10611 ) Change subject: IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition .. Patch Set 7: > Do we plan to port it to 2.x? If not please update > https://github.com/apache/impala/blob/2.x/bin/ignored_commits.json Nevermind. 2.x no longer receives new features and the automatic cherrypicking job has been disabled. -- To view, visit http://gerrit.cloudera.org:8080/10611 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id55f7d2e94d81e66ce720acb6315f15a89621b31 Gerrit-Change-Number: 10611 Gerrit-PatchSet: 7 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 19:08:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Vincent Tran has posted comments on this change. (
http://gerrit.cloudera.org:8080/10747 )
Change subject: IMPALA-7181: Fix flaky test
shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening
..
Patch Set 1:
(3 comments)
Thanks for reviewing!
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@629
PS1, Line 629: # Verify that impala-shell tries to create a socket
against the host:port
: # combination specified by -i when -b is not used
: impala_shell = Popen(shlex.split("%s %s" % (shell_cmd,
args1, )), stdout=devnull,
: stderr=devnull)
: connection, client_address = s.accept()
: data = connection.recv(1024)
: assert expected_output in data
: impala_shell.kill()
: connection.close()
:
: # Verify that impala-shell tries to create a socket
against the host:port
: # combination specified by -i when -b is used
> I would prefer if the duplicated logic in the two cases would be merged to
Good idea. Done.
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@645
PS1, Line 645: expected_output = "PingImpalaService"
> Already set in line 627, no need to set here again.
Yep. Thought I cleaned this up but it snuck back in after a few edits.
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@649
PS1, Line 649: s.close()
> I am a bit worried about exceptions - this could be moved to a finally bloc
Done
--
To view, visit http://gerrit.cloudera.org:8080/10747
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44
Gerrit-Change-Number: 10747
Gerrit-PatchSet: 1
Gerrit-Owner: Vincent Tran
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Reviewer: Vincent Tran
Gerrit-Comment-Date: Mon, 18 Jun 2018 19:05:59 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Hello Philip Zeyliger, Csaba Ringhofer, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10747 to look at the new patch set (#2). Change subject: IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening .. IMPALA-7181: Fix flaky test shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening test_shell_commandline.py::TestImpalaShell::test_socket_opening uses netcat to listen to an ephmeral port to verify the expected socket opening behavior of impala-shell. This port number is fixed to 42000. When this port happens to be used by another outbound socket, this test will fail. This change refactors the test to use socket.bind(). The port used in this test is no longer fixed and will be picked automatically. This change also adds the proper cleanup logics to the various subprocess.Popen objects used in the test. Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 --- M tests/shell/test_shell_commandline.py 1 file changed, 32 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/10747/2 -- To view, visit http://gerrit.cloudera.org:8080/10747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44 Gerrit-Change-Number: 10747 Gerrit-PatchSet: 2 Gerrit-Owner: Vincent Tran Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Philip Zeyliger
[Impala-ASF-CR] IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10742 ) Change subject: IMPALA-7169: Prevent HDFS from checkpointing trash until 3000 AD .. Patch Set 2: > fyi, the build failure is related to > https://issues.apache.org/jira/browse/IMPALA-7130 > > Vincent is working on the test to deflake it. That JIRA is marked fixed yesterday. Can we GVO now? -- To view, visit http://gerrit.cloudera.org:8080/10742 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9452f7e44c7679f86a947cd20115c078757223d8 Gerrit-Change-Number: 10742 Gerrit-PatchSet: 2 Gerrit-Owner: Tianyi Wang Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 19:03:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition
Tianyi Wang has posted comments on this change. ( http://gerrit.cloudera.org:8080/10611 ) Change subject: IMPALA-7128 (part 1) Refactor interfaces for Db, View, Table, Partition .. Patch Set 7: Do we plan to port it to 2.x? If not please update https://github.com/apache/impala/blob/2.x/bin/ignored_commits.json -- To view, visit http://gerrit.cloudera.org:8080/10611 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id55f7d2e94d81e66ce720acb6315f15a89621b31 Gerrit-Change-Number: 10611 Gerrit-PatchSet: 7 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Mon, 18 Jun 2018 18:59:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7140 (part 5): support fetching file info for FS tables
Hello Tianyi Wang, Vuk Ercegovac, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/10749 to review the following change. Change subject: IMPALA-7140 (part 5): support fetching file info for FS tables .. IMPALA-7140 (part 5): support fetching file info for FS tables This adds support for fetching file information and creating file descriptors. With this patch, I'm able to connect and run queries. Most planner tests still fail because of missing column stats resulting in different join orders compared to the existing implementation. Change-Id: I42d67ab754872fad094c7dacdd2e1182de1bf3e8 --- M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java 8 files changed, 185 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/10749/1 -- To view, visit http://gerrit.cloudera.org:8080/10749 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I42d67ab754872fad094c7dacdd2e1182de1bf3e8 Gerrit-Change-Number: 10749 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7140 (part 3): load partitions for FS tables
Hello Tianyi Wang, Vuk Ercegovac, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10713 to look at the new patch set (#4). Change subject: IMPALA-7140 (part 3): load partitions for FS tables .. IMPALA-7140 (part 3): load partitions for FS tables This implements the loading of partitions from the HMS for FS-based tables. The LocalPartitionSpec class implements PrunablePartition based on parsing the partition names returned from the HMS. After partitions are pruned, the remaining partitions can be loaded by name. With this patch, I am able to connect to an impalad running with --use_local_catalog and issue 'show partitions functional.alltypes' with the expected results. A new unit test is added which shares some code with CatalogTest to ensure that partitions can be loaded and parsed properly. Testing partition pruning via end-to-end planning is not quite supported yet: we need to implement 'toThriftDescriptor()' before we can run those end-to-end tests. Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3 --- M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java A fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java A fe/src/main/java/org/apache/impala/catalog/local/LocalPartitionSpec.java M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java 14 files changed, 707 insertions(+), 108 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/10713/4 -- To view, visit http://gerrit.cloudera.org:8080/10713 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3 Gerrit-Change-Number: 10713 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tianyi Wang Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Hello Tim Armstrong, Dan Hecht, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10493 to look at the new patch set (#4). Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ IMPALA-110 will involve refactoring PartitionedAggregationNode by separating out the aggregation logic into a new type called Aggregator, and then supporting multiple Aggregators per node to allow for multiple aggregation classes to be evaluated at the same time. Each Aggregator will need to have its own memory reservation to operate, and we can do this by giving each Aggregator its own BufferPool::ClientHandle instead of using the usual ExecNode::buffer_pool_client_. To facilitate this, this patch refactors all of the buffer_pool_client_ related logic into a new class, ReservationManager, so that eventually each Aggregator can have its own ReservationManager and the logic in ClaimBufferReservation(), ReleaseUnusedReservation(), etc. won't be duplicated. Testing: - Passed a full run of the core tests. Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a --- M be/src/exec/analytic-eval-node.cc M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/hdfs-scan-node.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partitioned-aggregation-node.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/sort-node.cc M be/src/runtime/CMakeLists.txt A be/src/runtime/reservation-manager.cc A be/src/runtime/reservation-manager.h 13 files changed, 288 insertions(+), 164 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/10493/4 -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 4 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Thomas Marshall has posted comments on this change. (
http://gerrit.cloudera.org:8080/10493 )
Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_
..
Patch Set 3:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.h
File be/src/runtime/reservation-manager.h:
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.h@33
PS2, Line 33: with
> with?
Done
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.h@37
PS2, Line 37: ReservationManager() {}
> please document that
Done
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.h@39
PS2, Line 39: lize this ReservationManager with the given values. 'name' is
> you probably meant to use references here.
Done
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.cc
File be/src/runtime/reservation-manager.cc:
http://gerrit.cloudera.org:8080/#/c/10493/2/be/src/runtime/reservation-manager.cc@31
PS2, Line 31: Init(str
> this looks more like an Init() function (and doesn't have to be static give
Done
--
To view, visit http://gerrit.cloudera.org:8080/10493
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a
Gerrit-Change-Number: 10493
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall
Gerrit-Reviewer: Dan Hecht
Gerrit-Reviewer: Thomas Marshall
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Mon, 18 Jun 2018 18:38:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-110 (part 1): Refactor ExecNode::buffer pool client
Hello Tim Armstrong, Dan Hecht, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/10493 to look at the new patch set (#3). Change subject: IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ .. IMPALA-110 (part 1): Refactor ExecNode::buffer_pool_client_ IMPALA-110 will involve refactoring PartitionedAggregationNode by separating out the aggregation logic into a new type called Aggregator, and then supporting multiple Aggregators per node to allow for multiple aggregation classes to be evaluated at the same time. Each Aggregator will need to have its own memory reservation to operate, and we can do this by giving each Aggregator its own BufferPool::ClientHandle instead of using the usual ExecNode::buffer_pool_client_. To facilitate this, this patch refactors all of the buffer_pool_client_ related logic into a new class, ReservationManager, so that eventually each Aggregator can have its own ReservationManager and the logic in ClaimBufferReservation(), ReleaseUnusedReservation(), etc. won't be duplicated. Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a --- M be/src/exec/analytic-eval-node.cc M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-mt.cc M be/src/exec/hdfs-scan-node.cc M be/src/exec/partial-sort-node.cc M be/src/exec/partitioned-aggregation-node.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/sort-node.cc M be/src/runtime/CMakeLists.txt A be/src/runtime/reservation-manager.cc A be/src/runtime/reservation-manager.h 13 files changed, 288 insertions(+), 164 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/10493/3 -- To view, visit http://gerrit.cloudera.org:8080/10493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I75f92c3f4f05adeef11a70f59e0c8ff2d19bc17a Gerrit-Change-Number: 10493 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-7181: Fix flaky test shell/test shell commandline.py::TestImpalaShell::test socket opening
Csaba Ringhofer has posted comments on this change. (
http://gerrit.cloudera.org:8080/10747 )
Change subject: IMPALA-7181: Fix flaky test
shell/test_shell_commandline.py::TestImpalaShell::test_socket_opening
..
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py
File tests/shell/test_shell_commandline.py:
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@629
PS1, Line 629: # Verify that impala-shell tries to create a socket
against the host:port
: # combination specified by -i when -b is not used
: impala_shell = Popen(shlex.split("%s %s" % (shell_cmd,
args1, )), stdout=devnull,
: stderr=devnull)
: connection, client_address = s.accept()
: data = connection.recv(1024)
: assert expected_output in data
: impala_shell.kill()
: connection.close()
:
: # Verify that impala-shell tries to create a socket
against the host:port
: # combination specified by -i when -b is used
I would prefer if the duplicated logic in the two cases would be merged to a
function somehow, but this is optional.
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@645
PS1, Line 645: expected_output = "PingImpalaService"
Already set in line 627, no need to set here again.
http://gerrit.cloudera.org:8080/#/c/10747/1/tests/shell/test_shell_commandline.py@649
PS1, Line 649: s.close()
I am a bit worried about exceptions - this could be moved to a finally block or
the socket could be added to a with statement (see the python 2 version in
https://stackoverflow.com/a/16772515 )
--
To view, visit http://gerrit.cloudera.org:8080/10747
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd64632ded936d49fc404bcac75588dd7886be44
Gerrit-Change-Number: 10747
Gerrit-PatchSet: 1
Gerrit-Owner: Vincent Tran
Gerrit-Reviewer: Csaba Ringhofer
Gerrit-Reviewer: Philip Zeyliger
Gerrit-Comment-Date: Mon, 18 Jun 2018 18:18:18 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7140 (part 3): load partitions for FS tables
Todd Lipcon has posted comments on this change. (
http://gerrit.cloudera.org:8080/10713 )
Change subject: IMPALA-7140 (part 3): load partitions for FS tables
..
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java:
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@188
PS3, Line 188: List
> nit: indentation off?
Done
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java:
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java@67
PS3, Line 67: Li
> nit: put arg on prev. line.
Done
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@338
PS3, Line 338: a
> did these additional changes (and below) come in from a merge?
are you looking at an inter-diff view in gerrit? There aren't any changes in
this patch where this comment is. I did rebase so you're probably seeing the
changes that I rebased on top of.
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java:
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@65
PS3, Line 65: public static final int MAX_OWNER_LENGTH = 128;
> looks like from that same merge?
same - think you're looking at interdiff
http://gerrit.cloudera.org:8080/#/c/10713/3/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@268
PS3, Line 268: msTbl, String partName) throws MetaException,
CatalogException {
> nit: strange wrapping... put the type and arg together if possible.
Was carried over from copy-paste. Looks like I can just import Table here so
I'll do that and clean this up.
--
To view, visit http://gerrit.cloudera.org:8080/10713
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3
Gerrit-Change-Number: 10713
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon
Gerrit-Reviewer: Tianyi Wang
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Vuk Ercegovac
Gerrit-Comment-Date: Mon, 18 Jun 2018 18:14:30 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6969: add AC last queued cause to profile
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/10731 ) Change subject: IMPALA-6969: add AC last queued cause to profile .. Patch Set 1: Code-Review+1 (3 comments) lgtm, just a few nits. http://gerrit.cloudera.org:8080/#/c/10731/1/be/src/scheduling/admission-controller.h File be/src/scheduling/admission-controller.h: http://gerrit.cloudera.org:8080/#/c/10731/1/be/src/scheduling/admission-controller.h@485 PS1, Line 485: queueing nit: dequeueing http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py File tests/custom_cluster/test_admission_controller.py: http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py@614 PS1, Line 614: assert num_causes == NUM_QUERIES - 1, \ : "All queries except first should have been queued: " + '\n===\n'.join(profiles) nit: repeated assert http://gerrit.cloudera.org:8080/#/c/10731/1/tests/custom_cluster/test_admission_controller.py@627 PS1, Line 627: # nit: remove '#' -- To view, visit http://gerrit.cloudera.org:8080/10731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida9b75dc50dfb7a27f59deda91bad6ac838130a1 Gerrit-Change-Number: 10731 Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Bikramjeet Vig Gerrit-Comment-Date: Mon, 18 Jun 2018 18:14:08 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7119: Restart whole minicluster when HDFS replication stalls
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10665 ) Change subject: IMPALA-7119: Restart whole minicluster when HDFS replication stalls .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/10665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58ffe301708c78c26ee61aa754a06f46c224c6e2 Gerrit-Change-Number: 10665 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 18:11:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7119: Restart whole minicluster when HDFS replication stalls
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10665 ) Change subject: IMPALA-7119: Restart whole minicluster when HDFS replication stalls .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2694/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/10665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58ffe301708c78c26ee61aa754a06f46c224c6e2 Gerrit-Change-Number: 10665 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 18:11:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7161: Fix impala-config.sh's handling of JAVA HOME
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10702 ) Change subject: IMPALA-7161: Fix impala-config.sh's handling of JAVA_HOME .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2693/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/10702 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idf3521b4f44fdbdc841a90fd00c477c9423a75bb Gerrit-Change-Number: 10702 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tianyi Wang Gerrit-Comment-Date: Mon, 18 Jun 2018 18:10:53 + Gerrit-HasComments: No
