[Impala-ASF-CR] IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13872 ) Change subject: IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage .. Patch Set 1: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4607/ -- To view, visit http://gerrit.cloudera.org:8080/13872 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I08f1c36ecf54ac277d99e2d2843163eada732e50 Gerrit-Change-Number: 13872 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 17 Jul 2019 05:56:25 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13873 ) Change subject: IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3892/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca Gerrit-Change-Number: 13873 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Jul 2019 02:27:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13873 ) Change subject: IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3891/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca Gerrit-Change-Number: 13873 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 17 Jul 2019 02:19:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 7: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/387/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 7 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 17 Jul 2019 01:44:49 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13873 to look at the new patch set (#2). Change subject: IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations .. IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations Refactors PlanRootSink into a base class with two subclasses: BlockingPlanRootSink and BufferedPlanRootSink. BlockingPlanRootSink encapsulates the current implementation of PlanRootSink and BufferedPlanRootSink encapsulates a new implementation that will buffer RowBatches in memory until they are read by the client. The implementation of BlockingPlanRootSink is left to future work. A new query option called POOL_QUERY_RESULTS controls whether a BlockingPlanRootSink or a BufferedPlanRootSink is used as the DataSink. POOL_QUERY_RESULTS is false by default. Added a few more docs to PlanRootSink and BlockingPlanRootSink to make the implementation easier to understand. Testing: * Since no new functionality has been added no tests were added * Running core tests Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca --- M be/src/exec/CMakeLists.txt A be/src/exec/blocking-plan-root-sink.cc A be/src/exec/blocking-plan-root-sink.h A be/src/exec/buffered-plan-root-sink.cc A be/src/exec/buffered-plan-root-sink.h M be/src/exec/data-sink.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift 12 files changed, 369 insertions(+), 154 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13873/2 -- To view, visit http://gerrit.cloudera.org:8080/13873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca Gerrit-Change-Number: 13873 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations
Impala Public Jenkins has posted comments on this change. (
http://gerrit.cloudera.org:8080/13873 )
Change subject: IMPALA-8656: Re-factor PlanRootSink into blocking and buffered
implementations
..
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/blocking-plan-root-sink.h
File be/src/exec/blocking-plan-root-sink.h:
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/blocking-plan-root-sink.h@51
PS1, Line 51: BlockingPlanRootSink(TDataSinkId sink_id, const RowDescriptor*
row_desc, RuntimeState* state);
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/blocking-plan-root-sink.cc
File be/src/exec/blocking-plan-root-sink.cc:
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/blocking-plan-root-sink.cc@89
PS1, Line 89: // All rows have been sent by the producer, so wake up the
producer so it can set eos to true.
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/blocking-plan-root-sink.cc@116
PS1, Line 116: // Set the shared QueryResultSet pointer 'results_' to the
given 'results' object and wake
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/plan-root-sink.cc
File be/src/exec/plan-root-sink.cc:
http://gerrit.cloudera.org:8080/#/c/13873/1/be/src/exec/plan-root-sink.cc@43
PS1, Line 43: void PlanRootSink::ValidateCollectionSlots(const RowDescriptor&
row_desc, RowBatch* batch) {
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/13873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca
Gerrit-Change-Number: 13873
Gerrit-PatchSet: 1
Gerrit-Owner: Sahil Takiar
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Wed, 17 Jul 2019 01:39:37 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations
Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13873 Change subject: IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations .. IMPALA-8656: Re-factor PlanRootSink into blocking and buffered implementations Refactors PlanRootSink into a base class with two subclasses: BlockingPlanRootSink and BufferedPlanRootSink. BlockingPlanRootSink encapsulates the current implementation of PlanRootSink and BufferedPlanRootSink encapsulates a new implementation that will buffer RowBatches in memory until they are read by the client. The implementation of BlockingPlanRootSink is left to future work. A new query option called POOL_QUERY_RESULTS controls whether a BlockingPlanRootSink or a BufferedPlanRootSink is used as the DataSink. POOL_QUERY_RESULTS is false by default. Added a few more docs to PlanRootSink and BlockingPlanRootSink to make the implementation easier to understand. Testing: * Since no new functionality has been added no tests were added * Running core tests Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca --- M be/src/exec/CMakeLists.txt A be/src/exec/blocking-plan-root-sink.cc A be/src/exec/blocking-plan-root-sink.h A be/src/exec/buffered-plan-root-sink.cc A be/src/exec/buffered-plan-root-sink.h M be/src/exec/data-sink.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift 12 files changed, 366 insertions(+), 154 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/13873/1 -- To view, visit http://gerrit.cloudera.org:8080/13873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8786b1a9af68ab0a8a094970d8f955eb20d04bca Gerrit-Change-Number: 13873 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml File docs/topics/impala_kudu.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml@1157 PS5, Line 1157:Since there may be no one-to-one mapping between Kudu tables and : external tables, only internal tables are automatically : synchronized. > When you create a table in Kudu when the HMS integration is turned on, Kudu Done -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 17 Jul 2019 01:24:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Hello Thomas Tauber-Marshall, Mike Percy, Andrew Wong, Grant Henke, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13776 to look at the new patch set (#7). Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e --- M docs/shared/impala_common.xml M docs/topics/impala_kudu.xml M docs/topics/impala_tables.xml 3 files changed, 86 insertions(+), 64 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/13776/7 -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 7 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 7: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/387/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 7 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 17 Jul 2019 01:24:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8484: Run queries on disjoint executor groups
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/13550 )
Change subject: IMPALA-8484: Run queries on disjoint executor groups
..
Patch Set 9:
(11 comments)
Made it up to test_admission_controller.py. Will resume tomorrow.
http://gerrit.cloudera.org:8080/#/c/13550/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13550/9//COMMIT_MSG@20
PS9, Line 20: will it be considered for admission.
Document the EXECUTOR_GROUP query option?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/cluster-membership-mgr.h
File be/src/scheduling/cluster-membership-mgr.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/cluster-membership-mgr.h@134
PS9, Line 134: /// 'executor_groups'.
Document that it's called from UpdateMembership (since it's not thread-safe)?
Or actually, can this be a static function?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/query-schedule.h
File be/src/scheduling/query-schedule.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/query-schedule.h@269
PS9, Line 269: void set_executor_group(string executor_group) {
It shouldn't be valid to overwrite a previous exector group, right? Could
DCHECK(executor_group_.empty());
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/query-schedule.h@321
PS9, Line 321: string executor_group_;
Brief comment?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/client-request-state.cc@511
PS9, Line 511: // Register the query with the server to support cancellation.
Maybe add to the comment to explain why this is done after admission, i.e.
after admission the set of executors is fixed and an executor failure will
cause a query failure.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/impala-server.h
File be/src/service/impala-server.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/impala-server.h@446
PS9, Line 446: or scheduled per
The "scheduled" bit is potentially confusing, since we have many schedules
internally, some of which aren't used. Maybe omit, or rephrase as "or admitted".
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/impala-server.h@447
PS9, Line 447: /// (query_locations_).
Document the implications from the caller's point of view? E.g. "If one of the
backends fails, the query will be cancelled with an error".
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/impala-server.cc
File be/src/service/impala-server.cc:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/impala-server.cc@296
PS9, Line 296: safe_strto64(group.substr(colon_idx + 1).as_string(),
_desc.min_size);
Can we log a warning/error or abort the process if this parsing fails?
We also use StringParser::StringToInt() in a lot of places already, so I have a
slight preference for sticking with that. this function seems fine though.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/query-options.h
File be/src/service/query-options.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/service/query-options.h@172
PS9, Line 172: QUERY_OPT_FN(executor_group, EXECUTOR_GROUP,
TQueryOptionLevel::REGULAR)\
I'm not sure that I understand the use case for exposing this option. It seems
like in most cases an end-user trying to run a query on a particular group
would be an anti-pattern.
http://gerrit.cloudera.org:8080/#/c/13550/9/tests/custom_cluster/test_admission_controller.py
File tests/custom_cluster/test_admission_controller.py:
http://gerrit.cloudera.org:8080/#/c/13550/9/tests/custom_cluster/test_admission_controller.py@408
PS9, Line 408: print "Error: %s" % str(ex)
Remove or use LOG
http://gerrit.cloudera.org:8080/#/c/13550/9/tests/custom_cluster/test_admission_controller.py@560
PS9, Line 560: handle = client.execute_async("select 2")
I like this idea
--
To view, visit http://gerrit.cloudera.org:8080/13550
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8a1d0900f2a82bd2fc0a906cc094e442cffa189b
Gerrit-Change-Number: 13550
Gerrit-PatchSet: 9
Gerrit-Owner: Lars Volker
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Bikramjeet Vig
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Lars Volker
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Tim Armstrong
Gerrit-Comment-Date: Wed, 17 Jul 2019 01:14:43 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13872 ) Change subject: IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3890/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13872 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I08f1c36ecf54ac277d99e2d2843163eada732e50 Gerrit-Change-Number: 13872 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 17 Jul 2019 00:16:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8484: Run queries on disjoint executor groups
Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/13550 )
Change subject: IMPALA-8484: Run queries on disjoint executor groups
..
Patch Set 9:
(22 comments)
I got through the core of the admission controller changes. Pushing out a batch
of comments
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/runtime/exec-env.h
File be/src/runtime/exec-env.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/runtime/exec-env.h@268
PS9, Line 268: admit_num_limit_
Maybe admit_num_queries_limit? Probably a matter of taste but it would be more
explicit what the number is of.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/runtime/exec-env.cc@289
PS9, Line 289: 8
Consider making it a constant up the top of the file.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h
File be/src/scheduling/admission-controller.h:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@158
PS9, Line 158: but
: /// multiple groups can run queries of the same pool
I found this a little tricky to parse. Here's one alternative phrasing.
A resource pool can have multiple executor groups associated with it. Each
executor group belongs to a single resource pool and will only serve requests
from that pool. I.e. the relationship is 1 resource pool : many executor groups.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@283
PS9, Line 283: struct AdmissionRequest {
Can you mention the lifetime/ownership of the objects referred to here. Assume
this are allow owned directly or indirectly by CLietnRequestState?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@592
PS9, Line 592: const ExecutorGroup* executor_group;
Can this be NULL? Maybe should be a reference if non-nullable. Or if nullable
that is worth documenting in a comment. Maybe this will become clear later on.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@595
PS9, Line 595: typedef std::vector GroupSchedules;
This is mostly a matter of taste, but I have a bit of a bias towards just using
the full type everywhere so there's less indirection.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@628
PS9, Line 628: /// not be valid once the request has been admitted,
cancelled, or rejected.
nit: I found it a little tricky to see the delimitation of the groups of
members. I found the below style of delimination used in some ExecNodes to be
useful in some cases.
/
/// BEGIN: Members that must be Reset()
...
/// END: Members that must be Reset()
/
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@700
PS9, Line 700: /// 'membership_snapshot' has changed.
Document in what cases it might fail?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@707
PS9, Line 707: /// false and sets queue_node->admitted_schedule if the query
can be admitted. Returns
I found the output boolean a little confusing. I don't know if switching the
polarity helps (i.e. false means reject, true means admit or queue). I thought
about suggesting using an enum for the output, but that may be overcomplicating
things.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.h@752
PS9, Line 752: unavailable_reason
'unavailable_reason' for consistency.
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.cc@509
PS9, Line 509: if (num_admitted + 1 > admit_num_limit) {
Maybe: num_admitted >= admin_num_limit
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.cc@523
PS9, Line 523: // (b) The executor group is already at the maximum number of
requests.
This should be "One of the executors is already at its maximum number of
requests", I think
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.cc@561
PS9, Line 561: orth it?
Probably not?
http://gerrit.cloudera.org:8080/#/c/13550/9/be/src/scheduling/admission-controller.cc@738
PS9, Line 738: return Status("AC: Local backend has not yet started");
Is this reachable? I spent a couple of minutes trying to understand how we'd
get into this state but couldn't figure it out - I'm assuming it would have to
be if the server had started but not received the initial cluster membership?
Since it looks like local_be_desc is set in UpdateMembership(). Although I'd
think in that case that we'd
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml@2115 PS4, Line 2115: Kudu service is integrated with the Hive Metastore, this operation : is not supported in Kudu that is integrated with Hive Metastore. : > Am I understanding correctly then that altering a Kudu table between extern Hrm, that's surprising to me because we have tests that test for this: https://github.infra.cloudera.com/QE/cmf/blob/68bc1e95444cdb54c76690df6c0be2bc21e60ed7/systest/test_kudu_hms_enabled_l1.py#L519 running the query: https://github.infra.cloudera.com/QE/cmf/blob/b4944a49437706eebe443ed0b2fc48b65058e898/systest/kudu/kudu_hms_integration_test_base.py#L20 What version of the Kudu HMS plugin are you using locally? Has it been rebuilt since June (when Hao's plugin patch went in)? -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Wed, 17 Jul 2019 00:10:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13872 ) Change subject: IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4607/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13872 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I08f1c36ecf54ac277d99e2d2843163eada732e50 Gerrit-Change-Number: 13872 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Jul 2019 23:37:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13872 Change subject: IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage .. IMPALA-8766: Change cloud dependencies to use hadoop-cloud-storage Currently, fe/pom.xml includes direct dependencies on hadoop-aws, hadoop-azure, and hadoop-azure-datalake. Hadoop has a meta-package hadoop-cloud-storage that provides these dependencies with appropriate exclusions. This would not change Impala's build significantly, but it allows different versions or distributions of Hadoop to add necessary runtime jars to the classpath. For example, distributions of Hadoop that use Apache Knox for identity management can add Knox jars to this meta-package. This switches Impala to use the hadoop-cloud-storage package. This does not change the default configuration, but the USE_CDP_HIVE=true configuration gains Apache Knox jars. Due to the structure of the CDP maven repository, it has dependencies on versions of jars that are not in the CDP maven repository. So, USE_CDP_HIVE=true requires the Hortonworks public maven repository. This is added, but it is only enabled with USE_CDP_HIVE=true and only for releases (no SNAPSHOTs). Testing: - Ran core tests - Manually verified classpath changes Change-Id: I08f1c36ecf54ac277d99e2d2843163eada732e50 --- M fe/pom.xml M impala-parent/pom.xml 2 files changed, 18 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/13872/1 -- To view, visit http://gerrit.cloudera.org:8080/13872 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I08f1c36ecf54ac277d99e2d2843163eada732e50 Gerrit-Change-Number: 13872 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml@2115 PS4, Line 2115: Kudu service is integrated with the Hive Metastore, this operation : is not supported in Kudu that is integrated with Hive Metastore. : > Yes, AFAICT this is right. Thomas, could you elaborate on the experimentati Am I understanding correctly then that altering a Kudu table between external/internal should never work when HMS integration is enabled? I can run Kudu locally with HMS integration turned on, create a table, and alter it back and forth without getting any errors. -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 22:59:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 6: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/386/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 22:30:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml File docs/topics/impala_kudu.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml@1157 PS5, Line 1157:Since there may be no one-to-one mapping between Kudu tables and : external tables, only internal tables are automatically : synchronized. > What does this mean? How does Kudu enforce the presence of an internal tabl When you create a table in Kudu when the HMS integration is turned on, Kudu will create an HMS entry for that table with a MANAGED table type. -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 22:20:01 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Hello Thomas Tauber-Marshall, Mike Percy, Andrew Wong, Grant Henke, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13776 to look at the new patch set (#6). Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e --- M docs/shared/impala_common.xml M docs/topics/impala_kudu.xml M docs/topics/impala_tables.xml 3 files changed, 86 insertions(+), 64 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/13776/6 -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/13776/5/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/shared/impala_common.xml@2115 PS5, Line 2115: This op > nit: lowercase Done http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml File docs/topics/impala_kudu.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml@1157 PS5, Line 1157:Since there may be no one-to-one mapping between Kudu tables and : external tables, only internal tables are automatically : synchronized. > A more user-facing effect of the HMS integration is that Kudu will enforce What does this mean? How does Kudu enforce the presence of an internal table in the HMS? -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 22:03:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 6: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/386/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 6 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 22:03:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8493 IMPALA-8494: [DOCS] GRANT/REVOKE User and Group
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13861 ) Change subject: IMPALA-8493 IMPALA-8494: [DOCS] GRANT/REVOKE User and Group .. Patch Set 1: @Autin, could you review the patch and approve it? If there is someone else to review, please add them here. Thanks! -- To view, visit http://gerrit.cloudera.org:8080/13861 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I99165bcffa76b8da983fdbf905df4035f4afafe9 Gerrit-Change-Number: 13861 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Austin Nobis Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Jul 2019 21:57:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/13776/4/docs/shared/impala_common.xml@2115 PS4, Line 2115: Kudu service is integrated with the Hive Metastore, This operation : is not supported in Kudu that is integrated with Hive Metastore. : > @andrew So this statement is correct, right? Yes, AFAICT this is right. Thomas, could you elaborate on the experimentation you mentioned? http://gerrit.cloudera.org:8080/#/c/13776/5/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/shared/impala_common.xml@2115 PS5, Line 2115: This op nit: lowercase http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml File docs/topics/impala_kudu.xml: http://gerrit.cloudera.org:8080/#/c/13776/5/docs/topics/impala_kudu.xml@1157 PS5, Line 1157:Since there may be no one-to-one mapping between Kudu tables and : external tables, only internal tables are automatically : synchronized. A more user-facing effect of the HMS integration is that Kudu will enforce the presence of an internal table in the HMS for every Kudu table. -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 21:50:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/13776 ) Change subject: IMPALA-8727: [DOCS] Impala-side changes for Kudu HMS integration .. Patch Set 5: @thomas Could you take a look at the new patch I uploaded? Thanks! -- To view, visit http://gerrit.cloudera.org:8080/13776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ieec79ac0bbb860c6394a3bf0617b285a7d23ca9e Gerrit-Change-Number: 13776 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Thomas Tauber-Marshall Gerrit-Comment-Date: Tue, 16 Jul 2019 20:41:06 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1268086
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13865 ) Change subject: Bump CDP_BUILD_NUMBER to 1268086 .. Bump CDP_BUILD_NUMBER to 1268086 The new build number fixes some tests that otherwise fails because of missing Zstd library. Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Reviewed-on: http://gerrit.cloudera.org:8080/13865 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/impala-config.sh 1 file changed, 5 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Gerrit-Change-Number: 13865 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1268086
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13865 ) Change subject: Bump CDP_BUILD_NUMBER to 1268086 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Gerrit-Change-Number: 13865 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 17:05:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Support table capabilities handling with Hive 3
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Support table capabilities handling with Hive 3 .. IMPALA-8593: Support table capabilities handling with Hive 3 This patch adds a method to check if a table bucketed. For Hive 3, integrates with HMS translation layer for capabilities checks. Implements methods ensureTableWriteSupported and ensureTableReadSupported. Set default capabilities for tables. Tests: Added unit tests to ParserTest and AnalyzerTest. Added bucketed tables which are required by IMPALA-8439. Ran core tests(Hive 2 and Hive 3) ToDo: Integrate checking bucketed tables capabilities and creating error messages with HMS translation after Hive provides the required functions. Enable capabilities checking for Kudu tables. When upgrade tables from non-acid to acid, the default capabilities should be changed too. Currently, use the workaround by explicitly setting tblproperties OBJCAPABILITIES with the acid properties. Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Reviewed-on: http://gerrit.cloudera.org:8080/13558 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M tests/metadata/test_ddl.py M tests/metadata/test_show_create_table.py 21 files changed, 418 insertions(+), 11 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 14 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8593: Support table capabilities handling with Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Support table capabilities handling with Hive 3 .. Patch Set 13: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 13 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 15:32:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8751: Skip failing Kudu - HMS integration tests with Hive 3
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13854 ) Change subject: IMPALA-8751: Skip failing Kudu - HMS integration tests with Hive 3 .. Patch Set 1: Also, started https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/25/ -- To view, visit http://gerrit.cloudera.org:8080/13854 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic19b8d93eaa6e8ec886c5704578563fb0871f941 Gerrit-Change-Number: 13854 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 12:36:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8751: Skip failing Kudu - HMS integration tests with Hive 3
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13854 ) Change subject: IMPALA-8751: Skip failing Kudu - HMS integration tests with Hive 3 .. Patch Set 1: Code-Review+1 LGTM, I can give the final +2 if it looks OK to Yongzhi. -- To view, visit http://gerrit.cloudera.org:8080/13854 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic19b8d93eaa6e8ec886c5704578563fb0871f941 Gerrit-Change-Number: 13854 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 12:32:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8758: Improve error message when no executors are online
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13866 ) Change subject: IMPALA-8758: Improve error message when no executors are online .. IMPALA-8758: Improve error message when no executors are online Prior to this change a dedicated coordinator would not create the default executor group when registering its own backend descriptor in the cluster membership. This caused a misleading error message during scheduling when the default executor group could not be found. To improve this, we now always create the default executor group and return an improved error message if it is empty. This change adds a test that validates that a query against a cluster without executors returns the expected error. Change-Id: Ia4428ef833363f52b14dfff253569212427a8e2f Reviewed-on: http://gerrit.cloudera.org:8080/13866 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/scheduling/cluster-membership-mgr.cc M be/src/scheduling/scheduler.cc M bin/start-impala-cluster.py M tests/custom_cluster/test_coordinators.py 4 files changed, 15 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13866 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia4428ef833363f52b14dfff253569212427a8e2f Gerrit-Change-Number: 13866 Gerrit-PatchSet: 3 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8758: Improve error message when no executors are online
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13866 ) Change subject: IMPALA-8758: Improve error message when no executors are online .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13866 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4428ef833363f52b14dfff253569212427a8e2f Gerrit-Change-Number: 13866 Gerrit-PatchSet: 2 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 16 Jul 2019 11:11:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Attila Jeges has posted comments on this change. (
http://gerrit.cloudera.org:8080/13722 )
Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..
Patch Set 9:
(27 comments)
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-expr.h
File be/src/exprs/cast-expr.h:
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-expr.h@28
PS9, Line 28: // Note, that it should be verified at the callsite that
TExprNode.cast_expr is set.
nit: Move this comment in front of L31.
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-functions-ir.cc
File be/src/exprs/cast-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-functions-ir.cc@171
PS9, Line 171: if (val.is_null) return StringVal::null();
Add DCHECK(ctx != nullptr);
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-functions-ir.cc@178
PS9, Line 178: lexical_cast(tv)
It would be better to use ToString() directly rather than involving the
indirection of lexical_cast and operator<<. (just like it's done in L199).
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-functions-ir.cc@191
PS9, Line 191: if (val.is_null) return StringVal::null();
Add DCHECK(ctx != nullptr);
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/exprs/cast-functions-ir.cc@347
PS9, Line 347: if (val.is_null) return DateVal::null();
Add DCHECK(ctx != nullptr);
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.h
File be/src/runtime/datetime-iso-sql-format-parser.h:
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.h@28
PS9, Line 28: //
nit: in most other header files comments are prefixed with ///
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.h@46
PS9, Line 46: index
position
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.h@47
PS9, Line 47: token group
It hasn't been properly defined what "token group" means. Whst's the difference
between a token and a token group?
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.h@48
PS9, Line 48: GetNextTokenGroupFromInput
Naming is a bit confusing as it is used to find the end of the current token
(pointed to by input_str).
Again, instead of "TokenGroup", you might want to just use "Token" in the
wording.
http://gerrit.cloudera.org:8080/#/c/13722/3/be/src/runtime/datetime-iso-sql-format-parser.cc
File be/src/runtime/datetime-iso-sql-format-parser.cc:
http://gerrit.cloudera.org:8080/#/c/13722/3/be/src/runtime/datetime-iso-sql-format-parser.cc@116
PS3, Line 116: // Note the addition
> My assumption was that it is initialized to zero when the 'result' is creat
Maybe add some DCHECKs at the beginning of the function then?
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.cc
File be/src/runtime/datetime-iso-sql-format-parser.cc:
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-parser.cc@47
PS9, Line 47: if (!IsoSqlFormatTokenizer::IsSeparator(*current_pos))
return false;
: // Advance to the end of the separator sequence.
: ++current_pos;
: while (current_pos < end_pos &&
IsoSqlFormatTokenizer::IsSeparator(*current_pos)) {
: ++current_pos;
: }
: // Advance to the end of the separator sequence in the
expected tokens list.
: ++i;
: while (i < dt_ctx.toks.size() && dt_ctx.toks[i].type ==
SEPARATOR) ++i;
:
: // If we reached the end of input or the end of token
sequence, we can return.
: if (current_pos >= end_pos || i >= dt_ctx.toks.size()) {
: return (current_pos >= end_pos && i >=
dt_ctx.toks.size());
: }
:
: // Next token, following the separator sequence.
: tok = _ctx.toks[i];
:
: // The last '-' of a separator sequence might be taken as a
sign for timezone hour.
: if (*(current_pos - 1) == '-' && tok->type ==
TIMEZONE_HOUR) {
: --current_pos;
: }
Please consider moving this block to a separate function.
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-tokenizer.h
File be/src/runtime/datetime-iso-sql-format-tokenizer.h:
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-tokenizer.h@73
PS9, Line 73: judge
nit: decide
http://gerrit.cloudera.org:8080/#/c/13722/9/be/src/runtime/datetime-iso-sql-format-tokenizer.h@75
PS9, Line 75: token group
Again, the wording here and below is a bit confusing. Shouldn't it just be
"token"?
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1268086
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/13865 ) Change subject: Bump CDP_BUILD_NUMBER to 1268086 .. Patch Set 2: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch-cdp-hive/24 only contained failed tests that will be fixed by https://gerrit.cloudera.org/#/c/13854/ -- To view, visit http://gerrit.cloudera.org:8080/13865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Gerrit-Change-Number: 13865 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 10:51:44 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1268086
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13865 ) Change subject: Bump CDP_BUILD_NUMBER to 1268086 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4606/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Gerrit-Change-Number: 13865 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 10:50:45 + Gerrit-HasComments: No
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1268086
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13865 ) Change subject: Bump CDP_BUILD_NUMBER to 1268086 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13865 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9f8783595db8a6cc0b1f205cc77edfbce54fed7 Gerrit-Change-Number: 13865 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 10:50:44 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-5031: Out-of-range enums are undefined behavior
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13804 ) Change subject: IMPALA-5031: Out-of-range enums are undefined behavior .. IMPALA-5031: Out-of-range enums are undefined behavior This eliminates an out-of-range enum value in the end-to-end tests. The interesting part of the backtrace is: exec/parquet/parquet-column-readers.cc:1530:112: runtime error: load of value 38, which is not a valid value for type 'Encoding::type' #0 BaseScalarColumnReader::ReadDataPage() exec/parquet/parquet-column-readers.cc:1530:112 #1 BaseScalarColumnReader::NextPage() exec/parquet/parquet-column-readers.cc:1769:28 #2 bool ScalarColumnReader ::ReadValueBatch(int, int, unsigned char*, int*) exec/parquet/parquet-column-readers.cc:459:12 #3 ScalarColumnReader ::ReadNonRepeatedValueBatch(MemPool*, int, int, unsigned char*, int*) exec/parquet/parquet-column-readers.cc:106:12 #4 HdfsParquetScanner::AssembleRows(vector const&, RowBatch*, bool*) exec/parquet/hdfs-parquet-scanner.cc:1113:42 #5 HdfsParquetScanner::GetNextInternal(RowBatch*) exec/parquet/hdfs-parquet-scanner.cc:456:19 #6 HdfsParquetScanner::ProcessSplit() exec/parquet/hdfs-parquet-scanner.cc:353:21 #7 HdfsScanNode::ProcessSplit(vector const&, MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21 #8 HdfsScanNode::ScannerThread(bool, long) exec/hdfs-scan-node.cc:415:7 #9 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*)::$_0 ::operator()() const exec/hdfs-scan-node.cc:337:13 Change-Id: Ia86de44daaf56a941fb95b15d5dfd7b5a2752129 Reviewed-on: http://gerrit.cloudera.org:8080/13804 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-level-decoder.cc M be/src/exec/parquet/parquet-level-decoder.h M common/thrift/parquet.thrift 4 files changed, 21 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13804 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia86de44daaf56a941fb95b15d5dfd7b5a2752129 Gerrit-Change-Number: 13804 Gerrit-PatchSet: 3 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-5031: Out-of-range enums are undefined behavior
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13804 ) Change subject: IMPALA-5031: Out-of-range enums are undefined behavior .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13804 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia86de44daaf56a941fb95b15d5dfd7b5a2752129 Gerrit-Change-Number: 13804 Gerrit-PatchSet: 2 Gerrit-Owner: Jim Apple Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Tue, 16 Jul 2019 09:38:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Support table capabilities handling with Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Support table capabilities handling with Hive 3 .. Patch Set 13: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 13 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 09:16:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Support table capabilities handling with Hive 3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Support table capabilities handling with Hive 3 .. Patch Set 13: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4605/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 13 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Comment-Date: Tue, 16 Jul 2019 09:16:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3889/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 9 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Jul 2019 08:30:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3888/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Jul 2019 08:26:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Gabor Kaszab has posted comments on this change. (
http://gerrit.cloudera.org:8080/13722 )
Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..
Patch Set 9:
(3 comments)
I have just noticed you had 2 additional comments yesterday. Addressed them as
well.
http://gerrit.cloudera.org:8080/#/c/13722/6/be/src/runtime/datetime-iso-sql-format-parser.cc
File be/src/runtime/datetime-iso-sql-format-parser.cc:
http://gerrit.cloudera.org:8080/#/c/13722/6/be/src/runtime/datetime-iso-sql-format-parser.cc@195
PS6, Line 195: if (input_len == 0) return nullptr;
> if (input_len == 0) return nullptr;
Indeed. Done.
http://gerrit.cloudera.org:8080/#/c/13722/6/be/src/runtime/datetime-iso-sql-format-parser.cc@211
PS6, Line 211:
: const char* end_pos = start_of_token;
: while (end_pos < start_of_token + max_tok_len &&
: !IsoSqlFormatTokenizer::IsSeparator(*end_pos)) {
: ++end_pos;
: }
> 'len' is not really necessary:
Done
http://gerrit.cloudera.org:8080/#/c/13722/8/tests/query_test/test_cast_with_format.py
File tests/query_test/test_cast_with_format.py:
http://gerrit.cloudera.org:8080/#/c/13722/8/tests/query_test/test_cast_with_format.py@120
PS8, Line 120:
> flake8: E303 too many blank lines (2)
Done
--
To view, visit http://gerrit.cloudera.org:8080/13722
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23
Gerrit-Change-Number: 13722
Gerrit-PatchSet: 9
Gerrit-Owner: Gabor Kaszab
Gerrit-Reviewer: Attila Jeges
Gerrit-Reviewer: Gabor Kaszab
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Comment-Date: Tue, 16 Jul 2019 07:51:31 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Hello Attila Jeges, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13722
to look at the new patch set (#9).
Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..
IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
This enhancement introduces FORMAT clause for CAST() operator that is
applicable for casts between string types and timestamp types. Instead
of accepting SimpleDateFormat patterns the FORMAT clause supports
datetime patterns following the ISO:SQL:2016 standard.
Note, the CAST() operator without the FORMAT clause still uses
Impala's implementation of SimpleDateFormat handling. Similarly, the
existing conversion functions such as to_timestamp(), from_timestamp()
etc. remain unchanged and use SimpleDateFormat.
Milestone 1 contains all the format tokens covered by the SQL
standard. Further milestones will add more functionality on top of
this list to cover functionality provided by other RDBMS systems.
List of tokens implemented by this change:
- , YYY, YY, Y: Year tokens
- , RR: Round year tokens
- MM: Month
- DD: Day
- DDD: Day of year
- HH, HH12: Hour of day (1-12)
- HH24: Hour of day (0-23)
- MI: Minute
- SS: Second
- S: Second of day
- FF, FF1, ..., FF9: Fractional second
- AM, PM, A.M., P.M.: Meridiem indicators
- TZH: Timezone hour
- TZM: Timezone minute
- Separators: - . / , ' ; : space
- ISO8601 date indicators (T, Z)
Some notes about the matching algorithm:
- The parsing algorithm uses these tokens in a case insensitive
manner.
- The separators are interchangeable with each other. For example a
'-' separator in the format will match with a '.' character in the
input.
- The length of the separator sequences is handled flexibly meaning
that a single separator character in the format for instance would
match with a multi-separator sequence in the input.
- In a string type to timestamp conversion the timezone offset tokens
are parsed, expected to match with the input but they don't adjust
the result as the input is already expected to be in UTC format.
Usage example:
SELECT CAST('01-02-2019' AS TIMESTAMP FORMAT 'MM-DD-');
SELECT CAST('2019.10.10 13:30:40.123456 +01:30' AS TIMESTAMP
FORMAT '-MM-DD HH24:MI:SS.FF9 TZH:TZM');
SELECT CAST(timestamp_column as STRING
FORMAT " MM HH12 YY") from some_table;
Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/common/init.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/CMakeLists.txt
A be/src/exprs/cast-expr.cc
A be/src/exprs/cast-expr.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/date-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/scalar-expr.cc
M be/src/exprs/scalar-expr.h
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/CMakeLists.txt
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-parse-util.h
M be/src/runtime/date-test.cc
M be/src/runtime/date-value.cc
M be/src/runtime/date-value.h
A be/src/runtime/datetime-iso-sql-format-parser.cc
A be/src/runtime/datetime-iso-sql-format-parser.h
A be/src/runtime/datetime-iso-sql-format-tokenizer.cc
A be/src/runtime/datetime-iso-sql-format-tokenizer.h
D be/src/runtime/datetime-parse-util.h
A be/src/runtime/datetime-parser-common.cc
A be/src/runtime/datetime-parser-common.h
R be/src/runtime/datetime-simple-date-format-parser.cc
A be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/runtime-state.cc
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-parse-util.h
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.cc
M be/src/runtime/timestamp-value.h
M be/src/service/impala-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/testutil/random-vector-generators.h
M be/src/util/dict-test.cc
M be/src/util/min-max-filter-test.cc
M be/src/util/string-parser.h
M common/thrift/Exprs.thrift
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
A
testdata/workloads/functional-query/queries/QueryTest/cast_format_from_table.test
M testdata/workloads/functional-query/queries/QueryTest/date.test
A tests/query_test/test_cast_with_format.py
54 files changed, 3,394 insertions(+), 858 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13722/9
--
To view, visit http://gerrit.cloudera.org:8080/13722
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13722 ) Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1 .. Patch Set 8: (1 comment) http://gerrit.cloudera.org:8080/#/c/13722/8/tests/query_test/test_cast_with_format.py File tests/query_test/test_cast_with_format.py: http://gerrit.cloudera.org:8080/#/c/13722/8/tests/query_test/test_cast_with_format.py@120 PS8, Line 120: d flake8: E303 too many blank lines (2) -- To view, visit http://gerrit.cloudera.org:8080/13722 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23 Gerrit-Change-Number: 13722 Gerrit-PatchSet: 8 Gerrit-Owner: Gabor Kaszab Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Tue, 16 Jul 2019 07:47:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
Hello Attila Jeges, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13722
to look at the new patch set (#8).
Change subject: IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
..
IMPALA-8703: ISO:SQL:2016 datetime patterns - Milestone 1
This enhancement introduces FORMAT clause for CAST() operator that is
applicable for casts between string types and timestamp types. Instead
of accepting SimpleDateFormat patterns the FORMAT clause supports
datetime patterns following the ISO:SQL:2016 standard.
Note, the CAST() operator without the FORMAT clause still uses
Impala's implementation of SimpleDateFormat handling. Similarly, the
existing conversion functions such as to_timestamp(), from_timestamp()
etc. remain unchanged and use SimpleDateFormat.
Milestone 1 contains all the format tokens covered by the SQL
standard. Further milestones will add more functionality on top of
this list to cover functionality provided by other RDBMS systems.
List of tokens implemented by this change:
- , YYY, YY, Y: Year tokens
- , RR: Round year tokens
- MM: Month
- DD: Day
- DDD: Day of year
- HH, HH12: Hour of day (1-12)
- HH24: Hour of day (0-23)
- MI: Minute
- SS: Second
- S: Second of day
- FF, FF1, ..., FF9: Fractional second
- AM, PM, A.M., P.M.: Meridiem indicators
- TZH: Timezone hour
- TZM: Timezone minute
- Separators: - . / , ' ; : space
- ISO8601 date indicators (T, Z)
Some notes about the matching algorithm:
- The parsing algorithm uses these tokens in a case insensitive
manner.
- The separators are interchangeable with each other. For example a
'-' separator in the format will match with a '.' character in the
input.
- The length of the separator sequences is handled flexibly meaning
that a single separator character in the format for instance would
match with a multi-separator sequence in the input.
- In a string type to timestamp conversion the timezone offset tokens
are parsed, expected to match with the input but they don't adjust
the result as the input is already expected to be in UTC format.
Usage example:
SELECT CAST('01-02-2019' AS TIMESTAMP FORMAT 'MM-DD-');
SELECT CAST('2019.10.10 13:30:40.123456 +01:30' AS TIMESTAMP
FORMAT '-MM-DD HH24:MI:SS.FF9 TZH:TZM');
SELECT CAST(timestamp_column as STRING
FORMAT " MM HH12 YY") from some_table;
Change-Id: I19d8d097a45ae6f103b6cd1b2d81aad38dfd9e23
---
M be/src/benchmarks/convert-timestamp-benchmark.cc
M be/src/benchmarks/parse-timestamp-benchmark.cc
M be/src/common/init.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/CMakeLists.txt
A be/src/exprs/cast-expr.cc
A be/src/exprs/cast-expr.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/date-functions-ir.cc
M be/src/exprs/expr-test.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/scalar-expr.cc
M be/src/exprs/scalar-expr.h
M be/src/exprs/timestamp-functions-ir.cc
M be/src/exprs/timestamp-functions.cc
M be/src/exprs/timestamp-functions.h
M be/src/runtime/CMakeLists.txt
M be/src/runtime/date-parse-util.cc
M be/src/runtime/date-parse-util.h
M be/src/runtime/date-test.cc
M be/src/runtime/date-value.cc
M be/src/runtime/date-value.h
A be/src/runtime/datetime-iso-sql-format-parser.cc
A be/src/runtime/datetime-iso-sql-format-parser.h
A be/src/runtime/datetime-iso-sql-format-tokenizer.cc
A be/src/runtime/datetime-iso-sql-format-tokenizer.h
D be/src/runtime/datetime-parse-util.h
A be/src/runtime/datetime-parser-common.cc
A be/src/runtime/datetime-parser-common.h
R be/src/runtime/datetime-simple-date-format-parser.cc
A be/src/runtime/datetime-simple-date-format-parser.h
M be/src/runtime/runtime-state.cc
M be/src/runtime/timestamp-parse-util.cc
M be/src/runtime/timestamp-parse-util.h
M be/src/runtime/timestamp-test.cc
M be/src/runtime/timestamp-value.cc
M be/src/runtime/timestamp-value.h
M be/src/service/impala-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/testutil/random-vector-generators.h
M be/src/util/dict-test.cc
M be/src/util/min-max-filter-test.cc
M be/src/util/string-parser.h
M common/thrift/Exprs.thrift
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/CastExpr.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
A
testdata/workloads/functional-query/queries/QueryTest/cast_format_from_table.test
M testdata/workloads/functional-query/queries/QueryTest/date.test
A tests/query_test/test_cast_with_format.py
54 files changed, 3,395 insertions(+), 858 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13722/8
--
To view, visit http://gerrit.cloudera.org:8080/13722
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3887/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 16 Jul 2019 07:09:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: submit patch set 5 with better tests -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 16 Jul 2019 06:31:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Hello Bharath Vissapragada, Quanlong Huang, Sahil Takiar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13765 to look at the new patch set (#5). Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, the planner makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation(IsNullPredicate is not supported for complex types), AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Ran all core tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test M tests/query_test/test_nested_types.py 3 files changed, 63 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/5 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8425: part 2: avoid chown when building containers
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13699 ) Change subject: IMPALA-8425: part 2: avoid chown when building containers .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5476a97a7a030499a60a6cef67f8c3cdffa7e756 Gerrit-Change-Number: 13699 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 16 Jul 2019 06:06:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8425: part 2: avoid chown when building containers
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13699 ) Change subject: IMPALA-8425: part 2: avoid chown when building containers .. IMPALA-8425: part 2: avoid chown when building containers This reduces the size of an image from 1.36GB to 705MB with a release build on my system. Thanks to Joe McDonnell for the suggestion. Testing: Precommit docker tests are sufficient to validate that the containers are functional. Change-Id: I5476a97a7a030499a60a6cef67f8c3cdffa7e756 Reviewed-on: http://gerrit.cloudera.org:8080/13699 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M docker/impala_base/Dockerfile 1 file changed, 10 insertions(+), 10 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13699 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5476a97a7a030499a60a6cef67f8c3cdffa7e756 Gerrit-Change-Number: 13699 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-8686: docker entrypoint script execs daemon
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13682 ) Change subject: IMPALA-8686: docker entrypoint script execs daemon .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifefbe0a926cf9cfb8acbd37c3f691dc28847dd8b Gerrit-Change-Number: 13682 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 16 Jul 2019 06:04:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8686: docker entrypoint script execs daemon
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13682 ) Change subject: IMPALA-8686: docker entrypoint script execs daemon .. IMPALA-8686: docker entrypoint script execs daemon The script now execs the subprocess, which is required for signals, etc to be handled correctly. Change-Id: Ifefbe0a926cf9cfb8acbd37c3f691dc28847dd8b Reviewed-on: http://gerrit.cloudera.org:8080/13682 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M docker/daemon_entrypoint.sh 1 file changed, 1 insertion(+), 7 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13682 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifefbe0a926cf9cfb8acbd37c3f691dc28847dd8b Gerrit-Change-Number: 13682 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Anurag Mantripragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
