[
https://issues.apache.org/jira/browse/IMPALA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar resolved IMPALA-8780.
----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.3.0
Closing as fixed. We ended up referring the complete re-factoring of
IMPALA-8779 to a later patch.
> Implementation of BufferedPlanRootSink where FlushFinal blocks until all rows
> are fetched
> -----------------------------------------------------------------------------------------
>
> Key: IMPALA-8780
> URL: https://issues.apache.org/jira/browse/IMPALA-8780
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Fix For: Impala 3.3.0
>
>
> Implement {{BufferedPlanRootSink}} so that {{FlushFinal}} blocks until all
> rows are fetched. The implementation should use the {{RowBatchQueue}}
> introduced by IMPALA-8779. By blocking in {{FlushFinal}} all non-coordinator
> fragments will be closed if all results fit in the {{RowBatchQueue}}.
> {{BufferedPlanRootSink::Send}} should enqueue each given {{RowBatch}} onto
> the queue and then return. If the queue is full, it should block until there
> is more space left in the queue. {{BufferedPlanRootSink::GetNext}} reads from
> the queue and then fills in the given {{QueryResultSet}} by using the
> {{DataSink}} {{ScalarExprEvaluator}}-s. Since the producer thread can call
> {{BufferedPlanRootSink::Close}} while the consumer is calling
> {{BufferedPlanRootSink::GetNext}} the two methods need to be synchronized so
> that the {{DataSink}} {{MemTracker}}-s are not closed while {{GetNext}} is
> running.
> The implementation of {{BufferedPlanRootSink}} should remain the same
> regardless of whether a {{std::queue}} backed {{RowBatchQueue}} or a
> {{BufferedTupleStream}} backed {{RowBatchQueue}} is used.
> {{BufferedPlanRootSink}} and {{BlockingPlanRootSink}} are similar in the
> sense that {{BlockingPlanRootSink}} buffers one {{RowBatch}}, so for queries
> that return under 1024 rows, all non-coordinator fragments are closed
> immediately as well. The advantage of {{BufferedPlanRootSink}} is that allows
> buffering for 1+ {{RowBatch}}-es.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
