Sahil Takiar created IMPALA-8786:
------------------------------------
Summary: BufferedPlanRootSink should directly write to a
QueryResultSet if one is available
Key: IMPALA-8786
URL: https://issues.apache.org/jira/browse/IMPALA-8786
Project: IMPALA
Issue Type: Sub-task
Components: Backend
Reporter: Sahil Takiar
Assignee: Sahil Takiar
{{BufferedPlanRootSink}} uses a {{RowBatchQueue}} to buffer {{RowBatch}}-es and
then the consumer thread reads them and writes them to a given
{{QueryResultSet}}. Implementations of {{RowBatchQueue}} might end up copying
the buffered {{RowBatch}}-es (e.g. if the queue is backed by a
{{BufferedTupleStream}}). An optimization would be for the producer thread to
directly write to the consumer {{QueryResultSet}}. This optimization would only
be triggered if (1) the queue is empty, and (2) the consumer thread has a
{{QueryResultSet}} available for writing.
This "fast path" is useful in a few different scenarios:
* If the consumer is faster than at reading rows than the producer is at
sending them; in this case, the overhead of buffering rows in a
{{RowBatchQueue}} can be completely avoided
* For queries that return under 1024 its likely that the consumer will produce
a {{QueryResultSet}} before the first {{RowBatch}} is returned (except perhaps
for very trivial queries)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
