[
https://issues.apache.org/jira/browse/IMPALA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432814#comment-17432814
]
Xinyi Zou edited comment on IMPALA-3201 at 10/22/21, 7:35 AM:
--------------------------------------------------------------
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace
MemTracker in the future?
In the current code, query/process memory restrictions are enforced and
AdmissionController collects the memory status of the impalad process through
MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical
meaning. In other words, what reservationTracker can do that MemTracker can’t
do. What is the purpose of introducing ReservationTracker.
Thanks : ) [~tarmstrong]
was (Author: xinyi zou):
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace
MemTracker in the future?
In the current code, query/process memory restrictions are enforced and
AdmissionController collects the memory status of the impalad process through
MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical
meaning. In other words, what reservationTracker can do that MemTracker can’t
do.
Thanks : ) [~tarmstrong]
> Implement basic in-memory buffer pool
> -------------------------------------
>
> Key: IMPALA-3201
> URL: https://issues.apache.org/jira/browse/IMPALA-3201
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
> Fix For: Impala 2.8.0
>
>
> This is a subtask of IMPALA-3200. The first milestone we want to hit is a
> usable buffer pool that does not support spilling. This would include the
> actual buffer pool, minus the logic for flushing unpinned blocks to disk (if
> it runs out of buffers, we can just terminate queries).
> Initially we should just allocate memory from TCMalloc.
> It would also include the initial work to port over exec nodes to the new
> buffer pool: enough to run some queries using them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
