[jira] [Commented] (BAHIR-324) JobManager created a kuduReader that was not closed, resulting in an increase in server handles
[
https://issues.apache.org/jira/browse/BAHIR-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727616#comment-17727616
]
ASF subversion and git services commented on BAHIR-324:
---
Commit 4a8f60f03f7caeb5421c7c88fa34a3ba419b61ee in bahir-flink's branch
refs/heads/master from Shimin Huang
[ https://gitbox.apache.org/repos/asf?p=bahir-flink.git;h=4a8f60f ]
[BAHIR-324] Closing KuduReader at JobManager
> JobManager created a kuduReader that was not closed, resulting in an increase
> in server handles
>
>
> Key: BAHIR-324
> URL: https://issues.apache.org/jira/browse/BAHIR-324
> Project: Bahir
> Issue Type: Bug
> Components: Flink Streaming Connectors
>Affects Versions: Flink-1.1.0
>Reporter: ShiminHuang
>Assignee: ShiminHuang
>Priority: Critical
> Fix For: Flink-1.2.0
>
> Attachments: image-2023-05-30-15-03-38-388.png,
> image-2023-05-30-15-04-47-387.png, image-2023-05-30-15-05-33-472.png,
> image-2023-05-30-15-06-16-063.png, image-2023-05-30-15-25-01-115.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> h1. Problem
> {quote}When JobManager starts, the ExecutionJobVertex constructor calls the
> AbstractKuduInputFormat#createInputSplits method, Causes the KuduReader
> object to create the KuduClient and KuduSession objects, but
> ExecutionJobVertex does not call the Kudureader #close() method with the
> display, causing the Kudu-related handle to remain open after the Flink task
> is closed. As a result, the number of server handles increases with the
> number of Flink tasks. As a result, the entire server becomes
> unavailable{quote}
> h1. Code
> *ExecutionJobVertex#constructor*
> !image-2023-05-30-15-03-38-388.png!
> {*}AbstractKuduInputFormat#createInputSplits{*}{*}{*}
> *!image-2023-05-30-15-04-47-387.png!*
> {*}AbstractKuduInputFormat{*}{*}#startKuduReader{*}
> {*}!image-2023-05-30-15-06-16-063.png!{*}{*}{*}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (BAHIR-324) JobManager created a kuduReader that was not closed, resulting in an increase in server handles
[
https://issues.apache.org/jira/browse/BAHIR-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727366#comment-17727366
]
zhang junbo commented on BAHIR-324:
---
Yes, i also found this issue. and in jobmanager log ,id print the callstrack
log as below: (just open ,not closed kudureader):
!image-2023-05-30-15-25-01-115.png!
> JobManager created a kuduReader that was not closed, resulting in an increase
> in server handles
>
>
> Key: BAHIR-324
> URL: https://issues.apache.org/jira/browse/BAHIR-324
> Project: Bahir
> Issue Type: Bug
> Components: Flink Streaming Connectors
>Affects Versions: Flink-1.1.0
>Reporter: ShiminHuang
>Priority: Critical
> Fix For: Flink-1.1.0
>
> Attachments: image-2023-05-30-15-03-38-388.png,
> image-2023-05-30-15-04-47-387.png, image-2023-05-30-15-05-33-472.png,
> image-2023-05-30-15-06-16-063.png, image-2023-05-30-15-25-01-115.png
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> h1. Problem
> {quote}When JobManager starts, the ExecutionJobVertex constructor calls the
> AbstractKuduInputFormat#createInputSplits method, Causes the KuduReader
> object to create the KuduClient and KuduSession objects, but
> ExecutionJobVertex does not call the Kudureader #close() method with the
> display, causing the Kudu-related handle to remain open after the Flink task
> is closed. As a result, the number of server handles increases with the
> number of Flink tasks. As a result, the entire server becomes
> unavailable{quote}
> h1. Code
> *ExecutionJobVertex#constructor*
> !image-2023-05-30-15-03-38-388.png!
> {*}AbstractKuduInputFormat#createInputSplits{*}{*}{*}
> *!image-2023-05-30-15-04-47-387.png!*
> {*}AbstractKuduInputFormat{*}{*}#startKuduReader{*}
> {*}!image-2023-05-30-15-06-16-063.png!{*}{*}{*}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
