[jira] [Commented] (HDFS-17434) Selector.select in SocketIOWithTimeout.java has significant overhead
[ https://issues.apache.org/jira/browse/HDFS-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835950#comment-17835950 ] ZanderXu commented on HDFS-17434: - [~qinyuren] Thanks for involving me. Can you share the screenshot about the sendbuffer of the connection on DN side? We need to confirm that if the connection is writable. > Selector.select in SocketIOWithTimeout.java has significant overhead > > > Key: HDFS-17434 > URL: https://issues.apache.org/jira/browse/HDFS-17434 > Project: Hadoop HDFS > Issue Type: Test >Reporter: qinyuren >Priority: Major > Attachments: image-2024-03-20-19-10-13-016.png, > image-2024-03-20-19-22-29-829.png, image-2024-03-20-19-24-02-233.png, > image-2024-03-20-19-55-18-378.png > > > In our cluster, the SendDataPacketBlockedOnNetworkNanosAvgTime metric ranges > from 5ms to 10ms, exceeding the usual disk reading overhead. Our machine > network card bandwidth is 2Mb/s. > !image-2024-03-20-19-10-13-016.png|width=662,height=135! > !image-2024-03-20-19-55-18-378.png! > By adding log printing, it turns out that the Selector.select function has > significant overhead. > !image-2024-03-20-19-22-29-829.png|width=474,height=262! > !image-2024-03-20-19-24-02-233.png|width=445,height=181! > I would like to know if this falls within the normal range or how we can > improve it. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17434) Selector.select in SocketIOWithTimeout.java has significant overhead
[ https://issues.apache.org/jira/browse/HDFS-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829366#comment-17829366 ] farmmamba commented on HDFS-17434: -- [~qinyuren] Hi, could you please show your createRbw avgTime? > Selector.select in SocketIOWithTimeout.java has significant overhead > > > Key: HDFS-17434 > URL: https://issues.apache.org/jira/browse/HDFS-17434 > Project: Hadoop HDFS > Issue Type: Test >Reporter: qinyuren >Priority: Major > Attachments: image-2024-03-20-19-10-13-016.png, > image-2024-03-20-19-22-29-829.png, image-2024-03-20-19-24-02-233.png, > image-2024-03-20-19-55-18-378.png > > > In our cluster, the SendDataPacketBlockedOnNetworkNanosAvgTime metric ranges > from 5ms to 10ms, exceeding the usual disk reading overhead. Our machine > network card bandwidth is 2Mb/s. > !image-2024-03-20-19-10-13-016.png|width=662,height=135! > !image-2024-03-20-19-55-18-378.png! > By adding log printing, it turns out that the Selector.select function has > significant overhead. > !image-2024-03-20-19-22-29-829.png|width=474,height=262! > !image-2024-03-20-19-24-02-233.png|width=445,height=181! > I would like to know if this falls within the normal range or how we can > improve it. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17434) Selector.select in SocketIOWithTimeout.java has significant overhead
[ https://issues.apache.org/jira/browse/HDFS-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828925#comment-17828925 ] qinyuren commented on HDFS-17434: - [~hexiaoqiao] [~tasanuma] [~zanderxu] Please take a look. > Selector.select in SocketIOWithTimeout.java has significant overhead > > > Key: HDFS-17434 > URL: https://issues.apache.org/jira/browse/HDFS-17434 > Project: Hadoop HDFS > Issue Type: Task >Reporter: qinyuren >Priority: Major > Attachments: image-2024-03-20-19-10-13-016.png, > image-2024-03-20-19-22-29-829.png, image-2024-03-20-19-24-02-233.png > > > In our cluster, the SendDataPacketBlockedOnNetworkNanosAvgTime metric ranges > from 5ms to 10ms, exceeding the usual disk reading overhead. Our machine > network card bandwidth is 10Gb/s. > !image-2024-03-20-19-10-13-016.png|width=662,height=135! > By adding log printing, it turns out that the Selector.select function has > significant overhead. > !image-2024-03-20-19-22-29-829.png|width=474,height=262! > !image-2024-03-20-19-24-02-233.png|width=445,height=181! > I would like to know if this falls within the normal range or how we can > improve it. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
