[jira] [Commented] (HDFS-15794) IBR and FBR use different queues to load data.
[
https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273253#comment-17273253
]
JiangHua Zhu commented on HDFS-15794:
-
[~kihwal] , thank you very much.
I think your suggestion is very meaningful.
> IBR and FBR use different queues to load data.
> --
>
> Key: HDFS-15794
> URL: https://issues.apache.org/jira/browse/HDFS-15794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Major
>
> When DataNode reports data to NameNode, IBR and FBR are included here.
> After the NameNode receives the DataNode request, it temporarily stores the
> data in a queue, here it refers to
> BlockManager#BlockReportProcessingThread#queue.
> NameNodeRpcServer#blockReport()
> for (int r = 0; r final BlockListAsLongs blocks = reports[r].getBlocks();
> final int index = r;
> noStaleStorages = bm.runBlockOp(() ->
> bm.processReport(nodeReg, reports[index].getStorage(),
> blocks, context));
> }
> NameNodeRpcServer#blockReport()
> for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) {
> bm.enqueueBlockOp(new Runnable() {
> @Override
> public void run() {
> try {
> namesystem.processIncrementalBlockReport(nodeReg, r);
> } catch (Exception ex) {
> // usually because the node is unregistered/dead. next heartbeat
> // will correct the problem
> blockStateChangeLog.error(
> "*BLOCK* NameNode.blockReceivedAndDeleted: "
> + "failed from "+ nodeReg + ":" + ex.getMessage());
> }
> }
> });
> }
> The problem here is that when the NameNode is blocked in processing the IBR,
> the FBR requested by the DN from the NameNode will be affected. Similarly,
> when the NameNode processing FBR is blocked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15794) IBR and FBR use different queues to load data.
[
https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272999#comment-17272999
]
Kihwal Lee commented on HDFS-15794:
---
{quote}The problem here is that when the NameNode is blocked in processing the
IBR, the FBR requested by the DN from the NameNode will be affected. Similarly,
when the NameNode processing FBR is blocked.
{quote}
The serial processing of IBR and FBR is not a side-effect of way a data
structure is used (single queue). In current namespace and block manager
design, each report is processed with the fsn write lock held. The queue made
it possible to process multiple IBRs under one lock, thus increasing
throughput. Having multiple queues for IBRs and FBRs won't help with
concurrency. In fact, it will complicate things, as it needs to maintain
certain processing order across multiple queues.
In order to make a meaningful performance improvement, we have to make NN
perform concurrent block report processing.
> IBR and FBR use different queues to load data.
> --
>
> Key: HDFS-15794
> URL: https://issues.apache.org/jira/browse/HDFS-15794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Major
>
> When DataNode reports data to NameNode, IBR and FBR are included here.
> After the NameNode receives the DataNode request, it temporarily stores the
> data in a queue, here it refers to
> BlockManager#BlockReportProcessingThread#queue.
> NameNodeRpcServer#blockReport()
> for (int r = 0; r final BlockListAsLongs blocks = reports[r].getBlocks();
> final int index = r;
> noStaleStorages = bm.runBlockOp(() ->
> bm.processReport(nodeReg, reports[index].getStorage(),
> blocks, context));
> }
> NameNodeRpcServer#blockReport()
> for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) {
> bm.enqueueBlockOp(new Runnable() {
> @Override
> public void run() {
> try {
> namesystem.processIncrementalBlockReport(nodeReg, r);
> } catch (Exception ex) {
> // usually because the node is unregistered/dead. next heartbeat
> // will correct the problem
> blockStateChangeLog.error(
> "*BLOCK* NameNode.blockReceivedAndDeleted: "
> + "failed from "+ nodeReg + ":" + ex.getMessage());
> }
> }
> });
> }
> The problem here is that when the NameNode is blocked in processing the IBR,
> the FBR requested by the DN from the NameNode will be affected. Similarly,
> when the NameNode processing FBR is blocked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15794) IBR and FBR use different queues to load data.
[
https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272840#comment-17272840
]
JiangHua Zhu commented on HDFS-15794:
-
[~weichiu] , thank you for your reply.
I noticed @14997 . The improvements made here are very meaningful.
But what I want to explain here is that @14997 is about the improvement of
DataNode.
What I want to express is that something meaningful can be done on the NameNode
side.
When NN processes IBR and FBR data, it can use different queues for processing,
instead of sharing one queue (BlockManager#BlockReportProcessingThread#queue).
This will benefit the NN's capabilities.
> IBR and FBR use different queues to load data.
> --
>
> Key: HDFS-15794
> URL: https://issues.apache.org/jira/browse/HDFS-15794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Major
>
> When DataNode reports data to NameNode, IBR and FBR are included here.
> After the NameNode receives the DataNode request, it temporarily stores the
> data in a queue, here it refers to
> BlockManager#BlockReportProcessingThread#queue.
> NameNodeRpcServer#blockReport()
> for (int r = 0; r final BlockListAsLongs blocks = reports[r].getBlocks();
> final int index = r;
> noStaleStorages = bm.runBlockOp(() ->
> bm.processReport(nodeReg, reports[index].getStorage(),
> blocks, context));
> }
> NameNodeRpcServer#blockReport()
> for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) {
> bm.enqueueBlockOp(new Runnable() {
> @Override
> public void run() {
> try {
> namesystem.processIncrementalBlockReport(nodeReg, r);
> } catch (Exception ex) {
> // usually because the node is unregistered/dead. next heartbeat
> // will correct the problem
> blockStateChangeLog.error(
> "*BLOCK* NameNode.blockReceivedAndDeleted: "
> + "failed from "+ nodeReg + ":" + ex.getMessage());
> }
> }
> });
> }
> The problem here is that when the NameNode is blocked in processing the IBR,
> the FBR requested by the DN from the NameNode will be affected. Similarly,
> when the NameNode processing FBR is blocked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15794) IBR and FBR use different queues to load data.
[
https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272795#comment-17272795
]
Wei-Chiu Chuang commented on HDFS-15794:
I think we made quite some improvements recently in here.
One of which is HDFS-14997.
> IBR and FBR use different queues to load data.
> --
>
> Key: HDFS-15794
> URL: https://issues.apache.org/jira/browse/HDFS-15794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Major
>
> When DataNode reports data to NameNode, IBR and FBR are included here.
> After the NameNode receives the DataNode request, it temporarily stores the
> data in a queue, here it refers to
> BlockManager#BlockReportProcessingThread#queue.
> NameNodeRpcServer#blockReport()
> for (int r = 0; r final BlockListAsLongs blocks = reports[r].getBlocks();
> final int index = r;
> noStaleStorages = bm.runBlockOp(() ->
> bm.processReport(nodeReg, reports[index].getStorage(),
> blocks, context));
> }
> NameNodeRpcServer#blockReport()
> for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) {
> bm.enqueueBlockOp(new Runnable() {
> @Override
> public void run() {
> try {
> namesystem.processIncrementalBlockReport(nodeReg, r);
> } catch (Exception ex) {
> // usually because the node is unregistered/dead. next heartbeat
> // will correct the problem
> blockStateChangeLog.error(
> "*BLOCK* NameNode.blockReceivedAndDeleted: "
> + "failed from "+ nodeReg + ":" + ex.getMessage());
> }
> }
> });
> }
> The problem here is that when the NameNode is blocked in processing the IBR,
> the FBR requested by the DN from the NameNode will be affected. Similarly,
> when the NameNode processing FBR is blocked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15794) IBR and FBR use different queues to load data.
[
https://issues.apache.org/jira/browse/HDFS-15794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272716#comment-17272716
]
JiangHua Zhu commented on HDFS-15794:
-
When DataNode requests IBR and FBR from NameNode, NameNode uses different queue
processing (BlockManager). This can improve the performance of NameNode
processing these two types of requests.
[~weichiu] [~elgoiri] Do you have different opinions?
> IBR and FBR use different queues to load data.
> --
>
> Key: HDFS-15794
> URL: https://issues.apache.org/jira/browse/HDFS-15794
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Major
>
> When DataNode reports data to NameNode, IBR and FBR are included here.
> After the NameNode receives the DataNode request, it temporarily stores the
> data in a queue, here it refers to
> BlockManager#BlockReportProcessingThread#queue.
> NameNodeRpcServer#blockReport()
> for (int r = 0; r final BlockListAsLongs blocks = reports[r].getBlocks();
> final int index = r;
> noStaleStorages = bm.runBlockOp(() ->
> bm.processReport(nodeReg, reports[index].getStorage(),
> blocks, context));
> }
> NameNodeRpcServer#blockReport()
> for (final StorageReceivedDeletedBlocks r: receivedAndDeletedBlocks) {
> bm.enqueueBlockOp(new Runnable() {
> @Override
> public void run() {
> try {
> namesystem.processIncrementalBlockReport(nodeReg, r);
> } catch (Exception ex) {
> // usually because the node is unregistered/dead. next heartbeat
> // will correct the problem
> blockStateChangeLog.error(
> "*BLOCK* NameNode.blockReceivedAndDeleted: "
> + "failed from "+ nodeReg + ":" + ex.getMessage());
> }
> }
> });
> }
> The problem here is that when the NameNode is blocked in processing the IBR,
> the FBR requested by the DN from the NameNode will be affected. Similarly,
> when the NameNode processing FBR is blocked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
