[jira] [Comment Edited] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-06 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783200#comment-17783200 ] Yuanbo Liu edited comment on HDFS-16016 at 11/6/23 11:58 AM: - [~hexiaoqiao] 

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-06 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783200#comment-17783200 ] Yuanbo Liu commented on HDFS-16016: --- [~hexiaoqiao]  Thanks for your reply. I find that my trunk is a

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-05 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783080#comment-17783080 ] Yuanbo Liu commented on HDFS-16016: --- [~hexiaoqiao]  >  I just wonder if meet some performance issue?

[jira] [Updated] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-05 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16016: -- Attachment: image-2023-11-06-10-55-50-939.png > BPServiceActor add a new thread to handle IBR >

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-05 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783075#comment-17783075 ] Yuanbo Liu commented on HDFS-16016: --- [~vjasani]  Unfortunately, there is no direct log that can prove

[jira] [Updated] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-05 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16016: -- Attachment: image-2023-11-06-10-53-13-584.png > BPServiceActor add a new thread to handle IBR >

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-05 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783070#comment-17783070 ] Yuanbo Liu commented on HDFS-16016: --- [~vjasani]  Thanks for your reply. I think HDFS-17129 is the one

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-03 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782505#comment-17782505 ] Yuanbo Liu commented on HDFS-16016: --- this scenario is rare but still happening > BPServiceActor add a

[jira] [Commented] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-03 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782501#comment-17782501 ] Yuanbo Liu commented on HDFS-16016: --- [~hexiaoqiao] [~vjasani] [~tasanuma]  Sorry to interrrupt. We've

[jira] [Updated] (HDFS-16016) BPServiceActor add a new thread to handle IBR

2023-11-03 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16016: -- Attachment: image-2023-11-03-18-11-54-502.png > BPServiceActor add a new thread to handle IBR >

[jira] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17147 ] Yuanbo Liu deleted comment on HDFS-17147: --- was (Author: yuanbo): cc [~inigoiri]  [~zhangshuyan] [~ayushsaxena] , > RBF: RouterRpcServer getListing become extremely slow when the children of >

[jira] [Commented] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751980#comment-17751980 ] Yuanbo Liu commented on HDFS-17147: --- cc [~inigoiri]  [~zhangshuyan] [~ayushsaxena] , > RBF:

[jira] [Commented] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751979#comment-17751979 ] Yuanbo Liu commented on HDFS-17147: --- cc [~inigoiri]  [~zhangshuyan] [~ayushsaxena] , > RBF:

[jira] [Updated] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-17147: -- Component/s: rbf > RBF: RouterRpcServer getListing become extremely slow when the children of > the

[jira] [Updated] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-17147: -- Description: Suppose we mount table as below:   {code:java} /dir -> ns0 ->  /target/dir /dir/child1

[jira] [Created] (HDFS-17147) RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns.

2023-08-08 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-17147: - Summary: RBF: RouterRpcServer getListing become extremely slow when the children of the dir are mounted in the same ns. Key: HDFS-17147 URL:

[jira] [Comment Edited] (HDFS-11461) DataNode Disk Outlier Detection

2023-07-28 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748590#comment-17748590 ] Yuanbo Liu edited comment on HDFS-11461 at 7/28/23 12:26 PM: - This patch

[jira] [Commented] (HDFS-11461) DataNode Disk Outlier Detection

2023-07-28 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-11461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748590#comment-17748590 ] Yuanbo Liu commented on HDFS-11461: --- This patch seems to need amend as the mean value of read/write io

[jira] [Commented] (HDFS-16657) Changing pool-level lock to volume-level lock for invalidation of blocks

2022-07-12 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566143#comment-17566143 ] Yuanbo Liu commented on HDFS-16657: --- Thanks for your comment. Sounds reasonable, I'll try it. >

[jira] [Updated] (HDFS-16657) Changing pool-level lock to volume-level lock for invalidation of blocks

2022-07-12 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16657: -- Description: Recently we see that the heartbeating of dn become slow in a very busy cluster, here is

[jira] [Created] (HDFS-16657) Changing pool-level lock to volume-level lock for invalidation of blocks

2022-07-12 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16657: - Summary: Changing pool-level lock to volume-level lock for invalidation of blocks Key: HDFS-16657 URL: https://issues.apache.org/jira/browse/HDFS-16657 Project: Hadoop

[jira] [Updated] (HDFS-16559) Seperate IBR thread waiting time from heart beat waiting time.

2022-05-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16559: -- Status: Patch Available (was: Open) > Seperate IBR thread waiting time from heart beat waiting time.

[jira] [Commented] (HDFS-16569) Consider attaching block location info from client when closing a completed file

2022-05-06 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533186#comment-17533186 ] Yuanbo Liu commented on HDFS-16569: --- Regarding Tectonic, seperating deamon threads from NN and

[jira] [Commented] (HDFS-16569) Consider attaching block location info from client when closing a completed file

2022-05-06 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532772#comment-17532772 ] Yuanbo Liu commented on HDFS-16569: --- [~ayushtkn] Thanks for your comments. > You mean to say, since the

[jira] [Commented] (HDFS-16569) Consider attaching block location info from client when closing a completed file

2022-05-06 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532765#comment-17532765 ] Yuanbo Liu commented on HDFS-16569: --- [~hexiaoqiao] Thanks for your comment. The process is here: 1.

[jira] [Created] (HDFS-16569) Consider attaching block location info from client when closing a completed file

2022-05-05 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16569: - Summary: Consider attaching block location info from client when closing a completed file Key: HDFS-16569 URL: https://issues.apache.org/jira/browse/HDFS-16569 Project:

[jira] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16477 ] Yuanbo Liu deleted comment on HDFS-16477: --- was (Author: yuanbo): [~ayushtkn] [~taoli91]  testExternalSPSMetrics is a duplicated function name which is already defined in

[jira] [Commented] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17527133#comment-17527133 ] Yuanbo Liu commented on HDFS-16477: --- [~ayushtkn] [~taoli91]  testExternalSPSMetrics is a duplicated

[jira] [Updated] (HDFS-16559) Seperate IBR thread waiting time from heart beat waiting time.

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16559: -- Summary: Seperate IBR thread waiting time from heart beat waiting time. (was: Adding new

[jira] [Updated] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16558: -- Attachment: image-2022-04-24-14-58-25-294.png > Consider changing the lock of delegation token from

[jira] [Updated] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16558: -- Description: In a very busy authed cluster, renewing/caneling/getting delegation token get slow and

[jira] [Updated] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16558: -- Attachment: image-2022-04-24-14-57-18-740.png > Consider changing the lock of delegation token from

[jira] [Created] (HDFS-16559) Adding new configuration for IBR thread

2022-04-24 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16559: - Summary: Adding new configuration for IBR thread Key: HDFS-16559 URL: https://issues.apache.org/jira/browse/HDFS-16559 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526986#comment-17526986 ] Yuanbo Liu commented on HDFS-16558: --- cc: [~hexiaoqiao] [~ste...@apache.org]   [~daryn] [~kihwal]  to

[jira] [Updated] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16558: -- Description: In a very busy authed cluster, renewing/caneling/getting delegation token get slow and

[jira] [Created] (HDFS-16558) Consider changing the lock of delegation token from write lock to read lock

2022-04-24 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16558: - Summary: Consider changing the lock of delegation token from write lock to read lock Key: HDFS-16558 URL: https://issues.apache.org/jira/browse/HDFS-16558 Project: Hadoop

[jira] [Created] (HDFS-16549) Consider using volume level lock for deleting blocks

2022-04-20 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16549: - Summary: Consider using volume level lock for deleting blocks Key: HDFS-16549 URL: https://issues.apache.org/jira/browse/HDFS-16549 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-16549) Consider using volume level lock for deleting blocks

2022-04-20 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16549: -- Description: It's great to see the implement of fine-grain lock for DN has been committed into trunk. 

[jira] [Deleted] (HDFS-16492) Add metrics and operation names for dataset lock

2022-03-16 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu deleted HDFS-16492: -- > Add metrics and operation names for dataset lock > > >

[jira] [Created] (HDFS-16492) Add metrics and operation names for dataset lock

2022-03-03 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16492: - Summary: Add metrics and operation names for dataset lock Key: HDFS-16492 URL: https://issues.apache.org/jira/browse/HDFS-16492 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-16479) EC: When reconstruting ec block index, liveBusyBlockIndicies is not enclude, then reconstructing will fail

2022-03-01 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499870#comment-17499870 ] Yuanbo Liu commented on HDFS-16479: --- [~tasanuma]  basically we use 

[jira] [Commented] (HDFS-16479) EC: When reconstruting ec block index, liveBusyBlockIndicies is not enclude, then reconstructing will fail

2022-02-28 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498867#comment-17498867 ] Yuanbo Liu commented on HDFS-16479: --- The simplest way to fix this issue is to add busy indices back to

[jira] [Updated] (HDFS-16479) EC: When reconstruting ec block index, liveBusyBlockIndicies is not enclude, then reconstructing will fail

2022-02-22 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16479: -- Description: We got this exception from DataNodes {color:#707070}java.lang.IllegalArgumentException:

[jira] [Created] (HDFS-16479) EC: When reconstruting ec block index, liveBusyBlockIndicies is not enclude, then reconstructing will fail

2022-02-22 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16479: - Summary: EC: When reconstruting ec block index, liveBusyBlockIndicies is not enclude, then reconstructing will fail Key: HDFS-16479 URL:

[jira] [Commented] (HDFS-16429) Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl

2022-02-20 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495364#comment-17495364 ] Yuanbo Liu commented on HDFS-16429: --- create pr for branch-3.2, thanks for your patch. > Add

[jira] [Commented] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

2022-02-20 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495308#comment-17495308 ] Yuanbo Liu commented on HDFS-15382: --- It seems not a  compatible feature, how about merging all sub-task

[jira] [Commented] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2022-02-20 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17495304#comment-17495304 ] Yuanbo Liu commented on HDFS-15180: --- Since it's suggested to split sub-tasks for this proposal, I'll

[jira] [Updated] (HDFS-16420) Avoid deleting unique data blocks when deleting redundancy striped blocks

2022-02-10 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16420: -- Component/s: ec > Avoid deleting unique data blocks when deleting redundancy striped blocks >

[jira] [Resolved] (HDFS-16433) The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing/opening block

2022-01-28 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu resolved HDFS-16433. --- Resolution: Not A Problem > The synchronized lock of IncrementalBlockReportManager will slow down

[jira] [Updated] (HDFS-16433) The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing/opening block

2022-01-21 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16433: -- Description: The code in IncrementalBlockReportManager.java {code:java} synchronized void

[jira] [Updated] (HDFS-16433) The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing/opening block

2022-01-21 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16433: -- Summary: The synchronized lock of IncrementalBlockReportManager will slow down the speed of

[jira] [Updated] (HDFS-16433) The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing block

2022-01-21 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16433: -- Description: The code in IncrementalBlockReportManager.java {code:java} synchronized void

[jira] [Created] (HDFS-16433) The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing block

2022-01-21 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16433: - Summary: The synchronized lock of IncrementalBlockReportManager will slow down the speed of creating/deleting/finalizing block Key: HDFS-16433 URL:

[jira] [Commented] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479090#comment-17479090 ] Yuanbo Liu commented on HDFS-15180: --- [~sodonnell]  Thanks for your comments. There's a background that

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14768: -- Attachment: HDFS-14768-branch-3.1.patch > EC : Busy DN replica should be consider in live replica

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479070#comment-17479070 ] Yuanbo Liu commented on HDFS-14768: --- Attached for branch-3.2, and branch-3.1, once it's merged, I'll

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14768: -- Attachment: HDFS-14768-branch-3.2.patch > EC : Busy DN replica should be consider in live replica

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14768: -- Attachment: (was: HDFS-14768-branch-3.2.patch) > EC : Busy DN replica should be consider in live

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-19 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14768: -- Attachment: HDFS-14768-branch-3.2.patch > EC : Busy DN replica should be consider in live replica

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2022-01-18 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478324#comment-17478324 ] Yuanbo Liu commented on HDFS-14768: --- For those whos is interesting in this patch, I've bachported

[jira] [Commented] (HDFS-15382) Split FsDatasetImpl from blockpool lock to blockpool volume lock

2021-11-16 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444371#comment-17444371 ] Yuanbo Liu commented on HDFS-15382: --- The design of splitting data lock by volume seems promising as the

[jira] [Commented] (HDFS-16201) Select datanode based on storage type

2021-09-26 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420439#comment-17420439 ] Yuanbo Liu commented on HDFS-16201: --- [~ayushtkn] Thanks for your comments, I'll update my patch later

[jira] [Commented] (HDFS-16201) Select datanode based on storage type

2021-09-02 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409233#comment-17409233 ] Yuanbo Liu commented on HDFS-16201: --- [~ayushtkn] Sorry to interrupt, would you mind taking a look at

[jira] [Commented] (HDFS-16201) Select datanode based on storage type

2021-09-01 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408495#comment-17408495 ] Yuanbo Liu commented on HDFS-16201: --- Those test failures are not related to this patch. > Select

[jira] [Updated] (HDFS-16201) Select datanode based on storage type

2021-09-01 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16201: -- Status: Patch Available (was: Open) > Select datanode based on storage type >

[jira] [Updated] (HDFS-16201) Select datanode based on storage type

2021-09-01 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-16201: -- Attachment: HDFS-16201-001.patch > Select datanode based on storage type >

[jira] [Created] (HDFS-16201) Select datanode based on storage type

2021-08-31 Thread Yuanbo Liu (Jira)
Yuanbo Liu created HDFS-16201: - Summary: Select datanode based on storage type Key: HDFS-16201 URL: https://issues.apache.org/jira/browse/HDFS-16201 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-15521) Remove INode.dumpTreeRecursively()

2020-08-10 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174269#comment-17174269 ] Yuanbo Liu commented on HDFS-15521: --- Thanks for opening this issue, not sure whether you're working on

[jira] [Commented] (HDFS-15486) Costly sendResponse operation slows down async editlog handling

2020-07-22 Thread Yuanbo Liu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17162780#comment-17162780 ] Yuanbo Liu commented on HDFS-15486: --- I happendly found a similiar issue discribing performance

[jira] [Commented] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-22 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823640#comment-16823640 ] Yuanbo Liu commented on HDFS-14421: --- Sorry, quite busy this week. Will comment as soon as I figure out.

[jira] [Commented] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-15 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16817614#comment-16817614 ] Yuanbo Liu commented on HDFS-14421: --- [~hexiaoqiao] I think I might find the reason. There are 12 disk

[jira] [Commented] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816178#comment-16816178 ] Yuanbo Liu commented on HDFS-14421: --- [~hexiaoqiao] Sorry. this is production environment. it doesn't

[jira] [Commented] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815188#comment-16815188 ] Yuanbo Liu commented on HDFS-14421: --- [~hexiaoqiao] Thanks for your comments. I've attached datanode

[jira] [Updated] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14421: -- Attachment: 326942161.log > HDFS block two replicas exist in one DataNode >

[jira] [Updated] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14421: -- Description: We're using Hadoop-2.7.0. There is a file in the cluster and it's replication factor is

[jira] [Updated] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-14421: -- Issue Type: Bug (was: Improvement) > HDFS block two replicas exist in one DataNode >

[jira] [Created] (HDFS-14421) HDFS block two replicas exist in one DataNode

2019-04-10 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-14421: - Summary: HDFS block two replicas exist in one DataNode Key: HDFS-14421 URL: https://issues.apache.org/jira/browse/HDFS-14421 Project: Hadoop HDFS Issue Type:

[jira] [Assigned] (HDFS-13724) Storage Tiering Show Paths with Policies applied

2018-07-15 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-13724: - Assignee: (was: Yuanbo Liu) > Storage Tiering Show Paths with Policies applied >

[jira] [Commented] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-15 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544450#comment-16544450 ] Yuanbo Liu commented on HDFS-13730: --- Good idea. I'll update my patch according to your comments. >

[jira] [Updated] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-15 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-13730: -- Status: Patch Available (was: Open) > BlockReaderRemote.sendReadResult throws NPE >

[jira] [Comment Edited] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541624#comment-16541624 ] Yuanbo Liu edited comment on HDFS-13730 at 7/12/18 1:21 PM: [~jojochuang] I'd

[jira] [Commented] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541624#comment-16541624 ] Yuanbo Liu commented on HDFS-13730: --- [~jojochuang] I'd suggested to return null instead of "" to keep

[jira] [Updated] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-13730: -- Attachment: HDFS-13730.001.patch > BlockReaderRemote.sendReadResult throws NPE >

[jira] [Assigned] (HDFS-13730) BlockReaderRemote.sendReadResult throws NPE

2018-07-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-13730: - Assignee: Yuanbo Liu > BlockReaderRemote.sendReadResult throws NPE >

[jira] [Assigned] (HDFS-13724) Storage Tiering Show Paths with Policies applied

2018-07-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-13724: - Assignee: Yuanbo Liu > Storage Tiering Show Paths with Policies applied >

[jira] [Commented] (HDFS-13724) Storage Tiering Show Paths with Policies applied

2018-07-07 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535988#comment-16535988 ] Yuanbo Liu commented on HDFS-13724: --- how about implementing a new command called "-find" to get the

[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade

2018-06-30 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528984#comment-16528984 ] Yuanbo Liu commented on HDFS-10664: --- [~djp] I find it's a bit complicated to reproduce it in unit test

[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade

2018-06-30 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528949#comment-16528949 ] Yuanbo Liu commented on HDFS-10664: --- [~djp] Thanks for your comments, I will test it again and if this

[jira] [Issue Comment Deleted] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade

2018-06-30 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10664: -- Comment: was deleted (was: [~djp] Thanks for your comments, will rebase my patch shortly) >

[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade

2018-06-30 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528948#comment-16528948 ] Yuanbo Liu commented on HDFS-10664: --- [~djp] Thanks for your comments, will rebase my patch shortly >

[jira] [Commented] (HDFS-7765) FSOutputSummer throwing ArrayIndexOutOfBoundsException

2018-06-27 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524777#comment-16524777 ] Yuanbo Liu commented on HDFS-7765: -- Any update about this issue? We've met the same issue in our env

[jira] [Resolved] (HDFS-9477) namenode starts failed:FSEditLogLoader: Encountered exception on operation TimesOp

2017-11-28 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu resolved HDFS-9477. -- Resolution: Fixed I believe that HDFS-8269 has solved this issue, please check it. > namenode starts

[jira] [Updated] (HDFS-12504) Ozone: Improve SQLCLI performance

2017-10-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-12504: -- Attachment: HDFS-12504-HDFS-7240.001.patch upload v1 patch for this JIRA. I've refactored the code in

[jira] [Updated] (HDFS-12504) Ozone: Improve SQLCLI performance

2017-10-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-12504: -- Status: Patch Available (was: In Progress) > Ozone: Improve SQLCLI performance >

[jira] [Work started] (HDFS-12504) Ozone: Improve SQLCLI performance

2017-10-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12504 started by Yuanbo Liu. - > Ozone: Improve SQLCLI performance > - > >

[jira] [Updated] (HDFS-12609) Ozone: SCM: Refactoring of chill mode logic in NodeManager

2017-10-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-12609: -- Resolution: Fixed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) >

[jira] [Commented] (HDFS-12609) Ozone: SCM: Refactoring of chill mode logic in NodeManager

2017-10-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196901#comment-16196901 ] Yuanbo Liu commented on HDFS-12609: --- [~nandakumar131] Committed it for you, thanks for your

[jira] [Commented] (HDFS-12609) Ozone: SCM: Refactoring of chill mode logic in NodeManager

2017-10-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196108#comment-16196108 ] Yuanbo Liu commented on HDFS-12609: --- [~nandakumar131] Thanks for your patch, those test failures are not

[jira] [Commented] (HDFS-12411) Ozone: Add container usage information to DN container report

2017-10-06 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194548#comment-16194548 ] Yuanbo Liu commented on HDFS-12411: --- [~xyao] Thanks for filing this JIRA, I have some comments about

[jira] [Updated] (HDFS-12539) Ozone: refactor some functions in KSMMetadataManagerImpl to be more readable and reusable

2017-10-03 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-12539: -- Attachment: HDFS-12539-HDFS-7240.003.patch Thanks Weiwei for your kindly review, upload v3 patch >

  1   2   3   4   5   6   7   >