[
https://issues.apache.org/jira/browse/IMPALA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432814#comment-17432814
]
Xinyi Zou edited comment on IMPALA-3201 at 10/22/21, 9:15 AM:
--------------------------------------------------------------
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace
MemTracker in the future?
In the current code, query/process memory restrictions are enforced and
AdmissionController collects the memory status of the impalad process through
MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical
meaning. In other words, what reservationTracker can do that MemTracker can’t
do. What is the purpose of introducing ReservationTracker. (guessing can be
used to discover memory limit is exceeded in advance)
Thanks : ) [~tarmstrong]
was (Author: xinyi zou):
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace
MemTracker in the future?
In the current code, query/process memory restrictions are enforced and
AdmissionController collects the memory status of the impalad process through
MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical
meaning. In other words, what reservationTracker can do that MemTracker can’t
do. What is the purpose of introducing ReservationTracker.
Thanks : ) [~tarmstrong]
> Implement basic in-memory buffer pool
> -------------------------------------
>
> Key: IMPALA-3201
> URL: https://issues.apache.org/jira/browse/IMPALA-3201
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
> Fix For: Impala 2.8.0
>
>
> This is a subtask of IMPALA-3200. The first milestone we want to hit is a
> usable buffer pool that does not support spilling. This would include the
> actual buffer pool, minus the logic for flushing unpinned blocks to disk (if
> it runs out of buffers, we can just terminate queries).
> Initially we should just allocate memory from TCMalloc.
> It would also include the initial work to port over exec nodes to the new
> buffer pool: enough to run some queries using them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
