[
https://issues.apache.org/jira/browse/FLINK-31293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weijie Guo updated FLINK-31293:
-------------------------------
Description:
In our TPC-DS test, we found that in the case of fierce competition in network
memory, some tasks may hanging forever.
>From the thread dump information, we can see that the task is waiting for the
>{{LocalBufferPool}} to become available. It is strange that other tasks have
>finished and released network memory already. Undoubtedly, this is an
>unexpected behavior, which implies that there must be a bug in the
>{{LocalBufferPool}} or {{{}NetworkBufferPool{}}}.
!image-2023-03-02-12-23-50-572.png|width=650,height=153!
By dumping the heap memory, we can find a strange phenomenon that there are
available buffers in the {{{}LocalBufferPool{}}}, but it was considered to be
un-available. Another thing to note is that it now holds an overdraft buffer.
!image-2023-03-02-12-28-48-437.png|width=520,height=200!
!image-2023-03-02-12-29-03-003.png|width=438,height=84!
was:
>From the thread dump information, we can see that the task is waiting for the
>\{{LocalBufferPool}} to become available. It is strange that other tasks have
>finished and released network memory already. Undoubtedly, this is an
>unexpected behavior, which implies that there must be a bug in the
>\{{LocalBufferPool}} or \{{NetworkBufferPool}}.
!image-2023-03-02-12-23-50-572.png|width=650,height=153!
We can find a strange phenomenon that there are available buffers in the
LocalBufferPool, but they are considered to be non-available. Another thing to
note is that it now holds an overdraft buffer
> Request memory segment from LocalBufferPool may hanging forever.
> ----------------------------------------------------------------
>
> Key: FLINK-31293
> URL: https://issues.apache.org/jira/browse/FLINK-31293
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.17.0
> Reporter: Weijie Guo
> Priority: Major
> Attachments: image-2023-03-02-12-23-50-572.png,
> image-2023-03-02-12-28-48-437.png, image-2023-03-02-12-29-03-003.png
>
>
> In our TPC-DS test, we found that in the case of fierce competition in
> network memory, some tasks may hanging forever.
> From the thread dump information, we can see that the task is waiting for the
> {{LocalBufferPool}} to become available. It is strange that other tasks have
> finished and released network memory already. Undoubtedly, this is an
> unexpected behavior, which implies that there must be a bug in the
> {{LocalBufferPool}} or {{{}NetworkBufferPool{}}}.
> !image-2023-03-02-12-23-50-572.png|width=650,height=153!
> By dumping the heap memory, we can find a strange phenomenon that there are
> available buffers in the {{{}LocalBufferPool{}}}, but it was considered to be
> un-available. Another thing to note is that it now holds an overdraft buffer.
> !image-2023-03-02-12-28-48-437.png|width=520,height=200!
> !image-2023-03-02-12-29-03-003.png|width=438,height=84!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
