[
https://issues.apache.org/jira/browse/IMPALA-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483442#comment-16483442
]
ASF subversion and git services commented on IMPALA-7011:
---------------------------------------------------------
Commit 482ea3914093064da1f4f176b6c616150100768c in impala's branch
refs/heads/master from [~dhecht]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=482ea39 ]
IMPALA-7011: Simplify PlanRootSink control logic
1) The eos_ and sender_done_ bits really encode three possible states
that the sender can be in. Make this explicit using an enum with
three values.
2) The purpose of CloseConsumer() has changed over time and we can clean
this up now:
a) Originally, it looks like it was used to unblock the sender when the
consumer finishes before eos, but also keep the sink alive long
enough for the coordinator. This is no longer necessary now that
control structures are owned by the QueryState whose lifetime is
controlled by a reference count taken by the coordinator. So, we don't
need the coordinator to tell the sink it's done calling it and we
don't need the consumer_done_ state.
b) Later on, CloseConsumer() was used as a cancellation mechinism.
We need to keep this around (or use timeouts on the condvars) to kick
both the consumer and producer on cancellation. But let's make the
cancellation logic similar to the exec nodes and other sinks by
driving the cancellation using the RuntimeState's cancellation
flag. Now that CloseConsumer() is only about cancellation, rename it
to Cancel() (later we may promote it to DataSink and implement in the
data stream sender as well).
Testing:
- Exhaustive
- Minicluster concurrent_select.py stress
Change-Id: Ifc75617a253fd43a6122baa4b4dc7aeb1dbe633f
Reviewed-on: http://gerrit.cloudera.org:8080/10449
Reviewed-by: Dan Hecht <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Cleanups around PlanRootSink::CloseConsumer()
> ---------------------------------------------
>
> Key: IMPALA-7011
> URL: https://issues.apache.org/jira/browse/IMPALA-7011
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Dan Hecht
> Assignee: Dan Hecht
> Priority: Minor
>
> We may not need some CloseConsumer() calls. Also, this is more about
> cancellation than closing, so I think we should rename it (and perhaps
> integrate more directly with the normal cancellation mechanisms).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
