[
https://issues.apache.org/jira/browse/IMPALA-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898319#comment-16898319
]
Sahil Takiar commented on IMPALA-8818:
--------------------------------------
Makes sense. I'm considering adding two query options then:
* {{MAX_PINNED_RESULT_SPOOLING_MEMORY}} - limits the max amount of pinned
memory used for result spooling by setting a max reservation for the
{{PlanRootSink}}
** In terms of the actual code, this will be used to set
{{TBackendResourceProfile.max_reservation}}
** A value of 0 means the memory is unbounded, so no max reservation is set
(which means {{Long.MAX_VALUE}} is used for the max reservation value), but as
you said, the query-wide limit still applies
** Considering a default of 100 MB
* {{MAX_UNPINNED_RESULT_SPOOLING_MEMORY}} - limits the max amount of unpinned
memory used for result spooling
** I think this requires some changes to {{BufferedTupleStream}} to track how
much of its memory is unpinned (e.g. add an unpinned version of
{{BufferedTupleStream::BytesPinned}})
** Based on my understanding of {{BufferedTupleStream}}, a call to
{{UnpinStream}} unpins all the pages in the stream; this means that
{{MAX_UNPINNED_RESULT_SPOOLING_MEMORY}} must be >=
{{MAX_PINNED_RESULT_SPOOLING_MEMORY}} so that when {{UnpinStream}} is called,
we don't exceed the value of {{MAX_UNPINNED_RESULT_SPOOLING_MEMORY}}
** I don't see a straightforward way to make this a hard limit because
unpinned pages are not reserved (maybe I'm missing something), but I think for
now it is sufficient to make this a soft limit (e.g. adding a {{RowBatch}} to
the stream may push the amount of unpinned memory over the limit, but attempts
to add additional batches will block)
** Considering a default of 1 GB
A few things I'm still trying to understand in BTS:
* When a stream is unpinned, are new pages pinned or unpinned?
* When do unpinned pages get spilled to disk / what decides if unpinned pages
are spilled?
> Replace deque queue with spillable queue in BufferedPlanRootSink
> ----------------------------------------------------------------
>
> Key: IMPALA-8818
> URL: https://issues.apache.org/jira/browse/IMPALA-8818
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> Add a {{SpillableRowBatchQueue}} to replace the {{DequeRowBatchQueue}} in
> {{BufferedPlanRootSink}}. The {{SpillableRowBatchQueue}} will wrap a
> {{BufferedTupleStream}} and take in a {{TBackendResourceProfile}} created by
> {{PlanRootSink#computeResourceProfile}}.
> *BufferedTupleStream Usage*:
> The wrapped {{BufferedTupleStream}} should be created in 'attach_on_read'
> mode so that pages are attached to the output {{RowBatch}} in
> {{BufferedTupleStream::GetNext}}. The BTS should start off as pinned (e.g.
> all pages are pinned). If a call to {{BufferedTupleStream::AddRow}} returns
> false (it returns false if "the unused reservation was not sufficient to add
> a new page to the stream large enough to fit 'row' and the stream could not
> increase the reservation to get enough unused reservation"), it should unpin
> the stream ({{BufferedTupleStream::UnpinStream}}) and then add the row (if
> the row still could not be added, then an error must have occurred, perhaps
> an IO error, in which case return the error and fail the query).
> *Constraining Resources*:
> When result spooling is disabled, a user can run a {{select * from
> [massive-fact-table]}} and scroll through the results without affecting the
> health of the Impala cluster (assuming they close they query promptly).
> Impala will stream the results one batch at a time to the user.
> With result spooling, a naive implementation might try and buffer the enter
> fact table, and end up spilling all the contents to disk, which can
> potentially take up a large amount of space. So there needs to be
> restrictions on the memory and disk space used by the {{BufferedTupleStream}}
> in order to ensure a scan of a massive table does not consume all the memory
> or disk space of the Impala coordinator.
> This problem can be solved by placing a max size on the amount of unpinned
> memory (perhaps through a new config option
> {{MAX_PINNED_RESULT_SPOOLING_MEMORY}} (maybe set to a few GBs by default).
> The max amount of pinned memory should already be constrained by the
> reservation (see next paragraph). NUM_ROWS_PRODUCED_LIMIT already limits the
> number of rows returned by a query, and so it should limit the number of rows
> buffered by the BTS as well (although it is set to 0 by default).
> SCRATCH_LIMIT already limits the amount of disk space used for spilling
> (although it is set to -1 by default).
> The {{PlanRootSink}} should attempt to accurately estimate how much memory it
> needs to buffer all results in memory. This requires setting an accurate
> value of {{ResourceProfile#memEstimateBytes_}} in
> {{PlanRootSink#computeResourceProfile}}. If statistics are available, the
> estimate can be based on the number of estimated rows returned multiplied by
> the size of the rows returned. The min reservation should account for a read
> and write page for the {{BufferedTupleStream}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
