[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Tim Armstrong has abandoned this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Abandoned Cleaning up some old CRs -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 4 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Rahul Shivu Mahadev Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Patch Set 4: (1 comment) May help to add a unit test to exercise retrying logic when sending a filter to the aggregator. http://gerrit.cloudera.org:8080/#/c/11055/4/common/thrift/ImpalaInternalService.thrift File common/thrift/ImpalaInternalService.thrift: http://gerrit.cloudera.org:8080/#/c/11055/4/common/thrift/ImpalaInternalService.thrift@878 PS4, Line 878: optional map> backend_list May help to add a comment on what this is for. -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 4 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Rahul Shivu Mahadev Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 10 Aug 2018 21:50:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/119/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 4 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Rahul Shivu Mahadev Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 31 Jul 2018 21:18:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Rahul Shivu Mahadev has posted comments on this change. (
http://gerrit.cloudera.org:8080/11055 )
Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation
..
Patch Set 3:
(37 comments)
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc
File be/src/runtime/coordinator-backend-state.cc:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@95
PS3, Line 95: x
> naming
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@98
PS3, Line 98: LOG(INFO) << "DebuggingPublishFilter AggregatorAddress " <<
tfs.aggregator_address;
> remove
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@95
PS3, Line 95: for (auto const& x : filter_routing_table) {
: TFilterState tfs;
: x.second.ToThrift(&tfs);
: LOG(INFO) << "DebuggingPublishFilter AggregatorAddress " <<
tfs.aggregator_address;
: rpc_params->filter_routing_table.insert(
: std::pair(x.first, tfs));
: }
> Add a comment about what you're doing here.
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@260
PS3, Line 260: x
> naming. Call it 'filter' or something similar
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@262
PS3, Line 262: num_filters_
> Is this needed as a member variable? Doesn't seem to be used anywhere.
do you think I should make it static ? and have better distribution across
queries ?
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@690
PS3, Line 690: if (params.__isset.filter_updates_received) {
> Add a comment above this line:
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@691
PS3, Line 691: if (backend_state->GetNumReceivedFilters() <
params.filter_updates_received) {
> Add another comment above this line:
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@884
PS3, Line 884: LOG(INFO)
: << "DebuggingPublishFilter Coordinator sending filter
to fragment with id "
: << fragment_idx;
> remove
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h
File be/src/runtime/filter-state.h:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@141
PS3, Line 141: FilterTarget
> const ref
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@150
PS3, Line 150: /// Need to cast the int type of this class to int32_t of thrift
> Still needed?
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@151
PS3, Line 151: set
> Why not unordered?
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@152
PS3, Line 152: int i
> int32_t here and remove the cast in the next line. Also, rename to 'idx'.
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@167
PS3, Line 167: TFilterTarget
> const ref
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@171
PS3, Line 171: boost
> std
Done
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@130
PS3, Line 130: bool disabled() const {
: if (is_bloom_filter()) {
: return bloom_filter_.always_true;
: } else {
: DCHECK(is_min_max_filter());
: return min_max_filter_.always_true;
: }
: }
:
: void ToThrift(TFilterState* f) const {
: std::vector t_targets;
: for (FilterTarget filter_target : targets_) {
: TFilterTarget thrift_filter_target;
: filter_target.ToThrift(&thrift_filter_target);
: t_targets.push_back(thrift_filter_target);
: }
: f->__set_targets(t_targets);
: f->__set_desc(desc_);
: f->__set_src(src_);
: f->__set_pending_count(pending_count_);
: /// Need to cast the int type of this class to int32_t of
thrift
: std::set src_fragment_instance_idxs;
: for (int i : src_fragment_instance_idxs_) {
: src_fragment_instance_idxs.insert((int32_t)i);
: }
:
f->__set_src_fragment_instance_idxs(src_fragment_instance_idxs);
: f->__set_bloom_filter(bloom_filter_);
: f->__set_min_max_filter(min_max_filter_);
: f->__set_first_arrival_time(first_arrival_time_);
: f->__set_completion_time(com
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Hello Michael Ho, Sailesh Mukil, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11055 to look at the new patch set (#4). Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A global flag has been added to switch back to legacy aggregation * The coordinator will now pick an aggregator for every filter and send the information via ExecQueryFInstances RPC. The corresponding QueryState will set up the filter routing table locally and send filter updates to the filter consumers. Failure Modes: 1. Aggregator node runs out of memory : There is a way to detect this and is currently employed by the coordinator, same behavior here i.e logs error. 2. Aggregator node crashes : Coordinator will detect this and cancel the query via cancel rpc, nothing to worry here. 3. Filter Producer/Producers crash before sending the filter to the Aggregator : Coordinator will detect the crash and inform other nodes. Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c --- M be/src/common/global-flags.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h A be/src/runtime/filter-state.cc R be/src/runtime/filter-state.h M be/src/runtime/query-exec-mgr.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-filter-bank.cc M be/src/service/impala-server.cc M common/thrift/ImpalaInternalService.thrift 14 files changed, 591 insertions(+), 151 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/11055/4 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 4 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Sailesh Mukil has posted comments on this change. (
http://gerrit.cloudera.org:8080/11055 )
Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation
..
Patch Set 3:
(40 comments)
http://gerrit.cloudera.org:8080/#/c/11055/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11055/3//COMMIT_MSG@14
PS3, Line 14:
Add comments about the following, so that it's easier for reviewers:
- Explain in a para or two how your patch achieves the distribution; i.e.
explain your approach in plain english.
- What kind of new failure modes can happen because of this change, as opposed
to before? (Talk about the Exec() RPC race with the UpdateFilter() RPC)
- How long do we expect the aggregators to be accessible? i.e. what is the
lifetime of an aggregator tied to?
- How are you updating the runtime profile?
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc
File be/src/runtime/coordinator-backend-state.cc:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@95
PS3, Line 95: x
naming
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@98
PS3, Line 98: LOG(INFO) << "DebuggingPublishFilter AggregatorAddress " <<
tfs.aggregator_address;
remove
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator-backend-state.cc@95
PS3, Line 95: for (auto const& x : filter_routing_table) {
: TFilterState tfs;
: x.second.ToThrift(&tfs);
: LOG(INFO) << "DebuggingPublishFilter AggregatorAddress " <<
tfs.aggregator_address;
: rpc_params->filter_routing_table.insert(
: std::pair(x.first, tfs));
: }
Add a comment about what you're doing here.
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc
File be/src/runtime/coordinator.cc:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@260
PS3, Line 260: x
naming. Call it 'filter' or something similar
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@262
PS3, Line 262: num_filters_
Is this needed as a member variable? Doesn't seem to be used anywhere.
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@690
PS3, Line 690: if (params.__isset.filter_updates_received) {
Add a comment above this line:
"Update aggregator's filter metrics in the runtime profile"
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@691
PS3, Line 691: if (backend_state->GetNumReceivedFilters() <
params.filter_updates_received) {
Add another comment above this line:
"Make sure not to double count filter updates from the same aggregator."
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@690
PS3, Line 690: if (params.__isset.filter_updates_received) {
: if (backend_state->GetNumReceivedFilters() <
params.filter_updates_received) {
: filter_updates_received_->Add(
: params.filter_updates_received -
backend_state->GetNumReceivedFilters());
:
backend_state->SetNumReceivedFilters(params.filter_updates_received);
: }
: }
There's a race here. If 2 updates for the same Backend execute in parallel,
you'll end up having an incorrect number of updated filters.
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/coordinator.cc@884
PS3, Line 884: LOG(INFO)
: << "DebuggingPublishFilter Coordinator sending filter
to fragment with id "
: << fragment_idx;
remove
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h
File be/src/runtime/filter-state.h:
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@141
PS3, Line 141: FilterTarget
const ref
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@150
PS3, Line 150: /// Need to cast the int type of this class to int32_t of thrift
Still needed?
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@151
PS3, Line 151: set
Why not unordered?
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@152
PS3, Line 152: int i
int32_t here and remove the cast in the next line. Also, rename to 'idx'.
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@167
PS3, Line 167: TFilterTarget
const ref
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@171
PS3, Line 171: boost
std
http://gerrit.cloudera.org:8080/#/c/11055/3/be/src/runtime/filter-state.h@130
PS3, Line 130: bool disabled() const {
: if (is_bloom_filter()) {
: return bloom_filter_.always_true;
: } else {
: DCHECK(is_min_max_filter());
: return min
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/81/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 3 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 27 Jul 2018 00:34:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/81/ Running initial code review checks. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 3 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 27 Jul 2018 00:00:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Hello Michael Ho, Sailesh Mukil, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11055 to look at the new patch set (#3). Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A global flag has been added to switch back to legacy aggregation Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c --- M be/src/common/global-flags.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h D be/src/runtime/coordinator-filter-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h A be/src/runtime/filter-state.cc A be/src/runtime/filter-state.h M be/src/runtime/query-exec-mgr.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-filter-bank.cc M be/src/service/impala-server.cc M common/thrift/ImpalaInternalService.thrift 15 files changed, 690 insertions(+), 265 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/11055/3 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 3 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/72/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 1 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 26 Jul 2018 08:14:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation
Hello Michael Ho, Sailesh Mukil, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/11055 to look at the new patch set (#2). Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation .. IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A global flag has been added to switch back to legacy aggregation Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c --- M be/src/common/global-flags.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h D be/src/runtime/coordinator-filter-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h A be/src/runtime/filter-state.cc A be/src/runtime/filter-state.h M be/src/runtime/query-exec-mgr.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-filter-bank.cc M be/src/service/impala-server.cc M common/thrift/ImpalaInternalService.thrift 15 files changed, 689 insertions(+), 265 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/11055/2 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 2 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated f
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11055 ) Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A glob .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-code-review-checks/72/ Running initial code review checks. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 1 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Thu, 26 Jul 2018 07:38:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated f
Rahul Shivu Mahadev has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11055 Change subject: IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A glob .. IMPALA-3825: Distribute Runtime Filtering Aggregation This patch moves runtime filter aggregation from the coordinator to other exec nodes. * QueryState will now house the Aggregated filters. * FilterState has been decoupled from coordinator code * A global flag has been added to switch back to legacy aggregation Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c --- M be/src/common/global-flags.cc M be/src/runtime/CMakeLists.txt M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h D be/src/runtime/coordinator-filter-state.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h A be/src/runtime/filter-state.cc A be/src/runtime/filter-state.h M be/src/runtime/query-exec-mgr.cc M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M be/src/runtime/runtime-filter-bank.cc M be/src/service/impala-server.cc M common/thrift/ImpalaInternalService.thrift 15 files changed, 689 insertions(+), 265 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/11055/1 -- To view, visit http://gerrit.cloudera.org:8080/11055 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I94e183a0353fc46f8d3eccae029d2d52c5cdc40c Gerrit-Change-Number: 11055 Gerrit-PatchSet: 1 Gerrit-Owner: Rahul Shivu Mahadev Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
