[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-06-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359651#comment-17359651 ] Xing Lin commented on HDFS-14703: - Hi [~prasad-acit], that is awesome! Konstantin is on vacation this

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-16 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345870#comment-17345870 ] Xing Lin commented on HDFS-14703: - I did some performance benchmarks using a physical server (a d430

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-16 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345870#comment-17345870 ] Xing Lin edited comment on HDFS-14703 at 5/17/21, 4:41 AM: --- I did some

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-16 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345870#comment-17345870 ] Xing Lin edited comment on HDFS-14703 at 5/17/21, 4:42 AM: --- I did some

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-15 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345069#comment-17345069 ] Xing Lin edited comment on HDFS-14703 at 5/15/21, 3:55 PM: --- [~prasad-acit] try

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-15 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345069#comment-17345069 ] Xing Lin commented on HDFS-14703: - [~prasad-acit] try this command: use -fs file:///, instead of

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-15 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345069#comment-17345069 ] Xing Lin edited comment on HDFS-14703 at 5/15/21, 3:53 PM: --- [~prasad-acit] try

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-15 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344970#comment-17344970 ] Xing Lin edited comment on HDFS-14703 at 5/15/21, 6:06 AM: --- [~prasad-acit] how

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-15 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344970#comment-17344970 ] Xing Lin commented on HDFS-14703: - [~prasad-acit] how many CPU cores does your server have? The OPS per

[jira] [Created] (HDFS-16125) iterator for PartitionedGSet would visit the first partition twice

2021-07-12 Thread Xing Lin (Jira)
Xing Lin created HDFS-16125: --- Summary: iterator for PartitionedGSet would visit the first partition twice Key: HDFS-16125 URL: https://issues.apache.org/jira/browse/HDFS-16125 Project: Hadoop HDFS

[jira] [Updated] (HDFS-16125) iterator for PartitionedGSet would visit the first partition twice

2021-07-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16125: Description: Iterator in PartitionedGSet would visit the first partition twice, since we did not set the

[jira] [Updated] (HDFS-16125) fix the iterator for PartitionedGSet

2021-07-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16125: Summary: fix the iterator for PartitionedGSet (was: iterator for PartitionedGSet would visit the first

[jira] [Commented] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-09-02 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408949#comment-17408949 ] Xing Lin commented on HDFS-16128: - Hi [~prasad-acit],  The issue is given a inode as a long value, the

[jira] [Comment Edited] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-09-02 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408949#comment-17408949 ] Xing Lin edited comment on HDFS-16128 at 9/2/21, 4:14 PM: -- Hi [~prasad-acit], 

[jira] [Commented] (HDFS-16191) [FGL] Fix FSImage loading issues on dynamic partitions

2021-09-13 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414719#comment-17414719 ] Xing Lin commented on HDFS-16191: - Yeah, that does not sound right: when there are 256 partitions, we

[jira] [Commented] (HDFS-16191) [FGL] Fix FSImage loading issues on dynamic partitions

2021-09-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412107#comment-17412107 ] Xing Lin commented on HDFS-16191: - Hi [~prasad-acit], Thanks for working on this! As you asked in the

[jira] [Updated] (HDFS-16128) Add support for saving/loading an FS Image for PartitionedGSet

2021-07-14 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Parent: HDFS-14703 Issue Type: Sub-task (was: Improvement) > Add support for saving/loading an FS

[jira] [Updated] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-14 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Summary: [FGL] Add support for saving/loading an FS Image for PartitionedGSet (was: Add support for

[jira] [Assigned] (HDFS-16128) Add support for saving/loading an FS Image for PartitionedGSet

2021-07-13 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-16128: --- Assignee: Xing Lin > Add support for saving/loading an FS Image for PartitionedGSet >

[jira] [Updated] (HDFS-16128) Add support for saving/loading an FS Image for PartitionedGSet

2021-07-13 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Summary: Add support for saving/loading an FS Image for PartitionedGSet (was: Add support for

[jira] [Updated] (HDFS-16128) Add support for saving/loading an FS Image

2021-07-13 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Description: Add support to save Inodes stored in PartitionedGSet when saving an FS image and load Inodes

[jira] [Created] (HDFS-16128) Add support for saving/loading an FS Image

2021-07-13 Thread Xing Lin (Jira)
Xing Lin created HDFS-16128: --- Summary: Add support for saving/loading an FS Image Key: HDFS-16128 URL: https://issues.apache.org/jira/browse/HDFS-16128 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-21 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384675#comment-17384675 ] Xing Lin commented on HDFS-16128: - Instead of loading inodes directly into the final inodeMap, we now

[jira] [Work started] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-16128 started by Xing Lin. --- > [FGL] Add support for saving/loading an FS Image for PartitionedGSet >

[jira] [Updated] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-14703: Fix Version/s: Fine-Grained Locking > NameNode Fine-Grained Locking via Metadata Partitioning >

[jira] [Updated] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-14703: Fix Version/s: (was: Fine-Grained Locking) > NameNode Fine-Grained Locking via Metadata Partitioning >

[jira] [Commented] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382941#comment-17382941 ] Xing Lin commented on HDFS-16128: - [~prasad-acit] Thanks for your comments. One of them is a bug in my

[jira] [Updated] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-16 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Labels: pull-request-available (was: ) > [FGL] Add support for saving/loading an FS Image for

[jira] [Updated] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-17 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16128: Description: Add support to save Inodes stored in PartitionedGSet when saving an FS image and load Inodes

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-27 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387797#comment-17387797 ] Xing Lin commented on HDFS-14703: - [~daryn] Thanks for your comments. I will address your last question

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-27 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387797#comment-17387797 ] Xing Lin edited comment on HDFS-14703 at 7/27/21, 6:07 AM: --- [~daryn] Thanks for

[jira] [Commented] (HDFS-17231) HA: Safemode should exit when resources are from low to available

2023-10-21 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778222#comment-17778222 ] Xing Lin commented on HDFS-17231: - Hi [~kuper], Please assign this Jira to yourself since you have

[jira] [Created] (HDFS-16818) RBF TestRouterRPCMultipleDestinationMountTableResolver non-deterministic unit tests failures

2022-10-24 Thread Xing Lin (Jira)
Xing Lin created HDFS-16818: --- Summary: RBF TestRouterRPCMultipleDestinationMountTableResolver non-deterministic unit tests failures Key: HDFS-16818 URL: https://issues.apache.org/jira/browse/HDFS-16818

[jira] [Commented] (HDFS-16818) RBF TestRouterRPCMultipleDestinationMountTableResolver non-deterministic unit tests failures

2022-10-25 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624081#comment-17624081 ] Xing Lin commented on HDFS-16818: - This unit test failure is non-deterministic. May be related with

[jira] [Created] (HDFS-16816) RBF: auto-create user home dir for trash paths by router

2022-10-24 Thread Xing Lin (Jira)
Xing Lin created HDFS-16816: --- Summary: RBF: auto-create user home dir for trash paths by router Key: HDFS-16816 URL: https://issues.apache.org/jira/browse/HDFS-16816 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-16818) RBF TestRouterRPCMultipleDestinationMountTableResolver non-deterministic unit tests failures

2022-10-30 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626178#comment-17626178 ] Xing Lin commented on HDFS-16818: - Even when we unset storagepolicy, we still get HOT policy. 

[jira] [Created] (HDFS-16790) rbf wrong path when destination dir is not created

2022-10-02 Thread Xing Lin (Jira)
Xing Lin created HDFS-16790: --- Summary: rbf wrong path when destination dir is not created Key: HDFS-16790 URL: https://issues.apache.org/jira/browse/HDFS-16790 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-15901) Solve the problem of DN repeated block reports occupying too many RPCs during Safemode

2023-01-07 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655752#comment-17655752 ] Xing Lin commented on HDFS-15901: - Do we have any followup on this issue?  We are seeing a similar issue

[jira] [Comment Edited] (HDFS-15901) Solve the problem of DN repeated block reports occupying too many RPCs during Safemode

2023-01-07 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655752#comment-17655752 ] Xing Lin edited comment on HDFS-15901 at 1/8/23 1:42 AM: - Do we have any followup

[jira] [Updated] (HDFS-16852) Register the shutdown hook only when not in shutdown for KeyProviderCache constructor

2022-12-02 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16852: Summary: Register the shutdown hook only when not in shutdown for KeyProviderCache constructor (was:

[jira] [Updated] (HDFS-16852) HDFS-16852 Register the shutdown hook only when not in shutdown for KeyProviderCache constructor

2022-12-02 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16852: Summary: HDFS-16852 Register the shutdown hook only when not in shutdown for KeyProviderCache constructor

[jira] [Assigned] (HDFS-16852) swallow IllegalStateException in KeyProviderCache

2022-11-22 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-16852: --- Assignee: Xing Lin > swallow IllegalStateException in KeyProviderCache >

[jira] [Created] (HDFS-16852) swallow IllegalStateException in KeyProviderCache

2022-11-22 Thread Xing Lin (Jira)
Xing Lin created HDFS-16852: --- Summary: swallow IllegalStateException in KeyProviderCache Key: HDFS-16852 URL: https://issues.apache.org/jira/browse/HDFS-16852 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-16852) Swallow IllegalStateException in KeyProviderCache

2022-11-22 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16852: Summary: Swallow IllegalStateException in KeyProviderCache (was: swallow IllegalStateException in

[jira] [Updated] (HDFS-16689) Standby NameNode crashes when transitioning to Active with in-progress tailer

2023-01-17 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16689: Description: Standby NameNode crashes when transitioning to Active with a in-progress tailer. And the

[jira] [Updated] (HDFS-16838) Fix NPE in testAddRplicaProcessorForAddingReplicaInMap

2022-11-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-16838: Description: There is a NPE in

[jira] [Commented] (HDFS-15505) Fix NullPointerException when call getAdditionalDatanode method with null extendedBlock parameter

2022-11-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632720#comment-17632720 ] Xing Lin commented on HDFS-15505: - There is no update from [~hangc] on the PR. I am not sure whether he

[jira] [Commented] (HDFS-16816) RBF: auto-create user home dir for trash paths by router

2022-11-02 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627993#comment-17627993 ] Xing Lin commented on HDFS-16816: - /user dir may not be writable for any regular user. In that case,

[jira] [Assigned] (HDFS-16816) RBF: auto-create user home dir for trash paths by router

2023-02-24 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-16816: --- Assignee: (was: Xing Lin) > RBF: auto-create user home dir for trash paths by router >

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

2023-07-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744437#comment-17744437 ] Xing Lin commented on HDFS-17093: - Hi [~yuyanlei], Thanks for sharing! I don't fully understand how your

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

2023-07-19 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744749#comment-17744749 ] Xing Lin commented on HDFS-17093: - {quote}[~xinglin] ,I think you modify some more reasonable, datanode

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

2023-07-19 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744751#comment-17744751 ] Xing Lin commented on HDFS-17093: - FYI, we set dfs.namenode.max.full.block.report.leases = 6, even though

[jira] [Updated] (HDFS-17067) Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-03 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17067: Description: In HDFS-17030, we introduced an ExecutorService, to submit getHAServiceState() requests. We

[jira] [Updated] (HDFS-17067) Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-03 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17067: Description: In HDFS-17030, we introduced an ExecutorService, to submit getHAServiceState() requests. We

[jira] [Updated] (HDFS-17067) Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-03 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17067: Summary: Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy (was:

[jira] [Created] (HDFS-17067) allowCoreThreadTimeOut should be set to true for nnProbingThreadPool in ObserverReadProxy

2023-07-03 Thread Xing Lin (Jira)
Xing Lin created HDFS-17067: --- Summary: allowCoreThreadTimeOut should be set to true for nnProbingThreadPool in ObserverReadProxy Key: HDFS-17067 URL: https://issues.apache.org/jira/browse/HDFS-17067

[jira] [Updated] (HDFS-17067) Use BlockingThreadPoolExecutorService for nnProbingThreadPool in ObserverReadProxy

2023-07-03 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17067: Description: In HDFS-17030, we introduced an ExecutorService, to submit getHAServiceState() requests. We

[jira] [Created] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-29 Thread Xing Lin (Jira)
Xing Lin created HDFS-17030: --- Summary: Limit wait time for getHAServiceState in ObserverReaderProxy Key: HDFS-17030 URL: https://issues.apache.org/jira/browse/HDFS-17030 Project: Hadoop HDFS

[jira] [Assigned] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-29 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-17030: --- Assignee: Xing Lin > Limit wait time for getHAServiceState in ObserverReaderProxy >

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-29 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-29 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-29 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17042) Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17042: Description: We'd like to add two new types of metrics to the existing NN RpcMetrics/RpcDetailedMetrics.

[jira] [Created] (HDFS-17042) Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread Xing Lin (Jira)
Xing Lin created HDFS-17042: --- Summary: Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode Key: HDFS-17042 URL: https://issues.apache.org/jira/browse/HDFS-17042 Project: Hadoop

[jira] [Updated] (HDFS-17042) Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17042: Description: We'd like to add two new types of metrics to the existing NN RpcMetrics/RpcDetailedMetrics. 

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-30 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-30 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-30 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17030) Limit wait time for getHAServiceState in ObserverReaderProxy

2023-05-30 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17030: Description: When namenode HA is enabled and a standby NN is not responsible, we have observed it would

[jira] [Updated] (HDFS-17118) Fix minor checkstyle warnings in TestObserverReadProxyProvider

2023-07-24 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17118: Description: We noticed a few checkstyle warnings when backporting HDFS-17030 from trunk to branch-3.3.

[jira] [Updated] (HDFS-17118) Fix minor checkstyle warnings in TestObserverReadProxyProvider

2023-07-24 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17118: Description: We noticed a few checkstyle warnings when backporting HDFS-17030 from trunk to branch-3.3.

[jira] [Updated] (HDFS-17118) Fix minor checkstyle warnings in TestObserverReadProxyProvider

2023-07-23 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17118: Description: We noticed a few checkstyle warnings when backporting HDFS-17030 from trunk to branch-3.3.

[jira] [Assigned] (HDFS-17118) Fix minor checkstyle warnings in TestObserverReadProxyProvider

2023-07-23 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-17118: --- Assignee: Xing Lin > Fix minor checkstyle warnings in TestObserverReadProxyProvider >

[jira] [Created] (HDFS-17118) Fix minor checkstyle warnings in TestObserverReadProxyProvider

2023-07-23 Thread Xing Lin (Jira)
Xing Lin created HDFS-17118: --- Summary: Fix minor checkstyle warnings in TestObserverReadProxyProvider Key: HDFS-17118 URL: https://issues.apache.org/jira/browse/HDFS-17118 Project: Hadoop HDFS

[jira] [Created] (HDFS-17055) Export HAState as a metric from Namenode for monitoring

2023-06-21 Thread Xing Lin (Jira)
Xing Lin created HDFS-17055: --- Summary: Export HAState as a metric from Namenode for monitoring Key: HDFS-17055 URL: https://issues.apache.org/jira/browse/HDFS-17055 Project: Hadoop HDFS Issue

[jira] [Assigned] (HDFS-17055) Export HAState as a metric from Namenode for monitoring

2023-06-21 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-17055: --- Assignee: Xing Lin > Export HAState as a metric from Namenode for monitoring >

[jira] [Updated] (HDFS-17055) Export HAState as a metric from Namenode for monitoring

2023-06-21 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17055: Description: We'd like measure the uptime for Namenodes: percentage of time when we have the

[jira] [Updated] (HDFS-17055) Export HAState as a metric from Namenode for monitoring

2023-06-21 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17055: Description: We'd like measure the uptime for Namenodes: percentage of time when we have the

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: (was: active.png) > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: (was: Observer.png) > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-18 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: active.png observer.png > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: Observer.png > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: active.png > Add UDP as a transfer protocol for HDFS > ---

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: (was: Screenshot 2023-12-12 at 9.32.15 AM.png) > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: Screenshot 2023-12-12 at 9.31.58 AM.png > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: Screenshot 2023-12-12 at 9.32.15 AM.png > Add UDP as a transfer protocol for HDFS >

[jira] [Created] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
Xing Lin created HDFS-17286: --- Summary: Add UDP as a transfer protocol for HDFS Key: HDFS-17286 URL: https://issues.apache.org/jira/browse/HDFS-17286 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-17286) Add UDP as a transfer protocol for HDFS

2023-12-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17286: Attachment: (was: Screenshot 2023-12-12 at 9.31.58 AM.png) > Add UDP as a transfer protocol for HDFS >

[jira] [Updated] (HDFS-17281) Added support of reporting RPC round-trip time at NN.

2023-12-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17281: Description: We have come across a few cases where the hdfs clients are reporting very bad latencies, we

[jira] [Created] (HDFS-17281) Added support of reporting RPC round-trip time at NN.

2023-12-08 Thread Xing Lin (Jira)
Xing Lin created HDFS-17281: --- Summary: Added support of reporting RPC round-trip time at NN. Key: HDFS-17281 URL: https://issues.apache.org/jira/browse/HDFS-17281 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-17281) Added support of reporting RPC round-trip time at NN.

2023-12-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17281: Description: We have come across a few cases where the hdfs clients are reporting very bad latencies,

[jira] [Updated] (HDFS-17281) Added support of reporting RPC round-trip time at NN.

2023-12-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17281: Description: We have come across a few cases where the hdfs clients are reporting very bad latencies, we

[jira] [Updated] (HDFS-17281) Added support of reporting RPC round-trip time at NN.

2023-12-08 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17281: Attachment: Screenshot 2023-10-28 at 10.26.41 PM.png > Added support of reporting RPC round-trip time at

[jira] [Updated] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17332: Description: In DFSInputStream#actualGetFromOneDataNode(), it would send the exception stacktrace to the

[jira] [Assigned] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin reassigned HDFS-17332: --- Assignee: Xing Lin > DFSInputStream: avoid logging stacktrace until when we really need to fail a

[jira] [Updated] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17332: Environment: (was: In DFSInputStream#actualGetFromOneDataNode(), it would send the exception

[jira] [Updated] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17332: Description: In DFSInputStream#actualGetFromOneDataNode(), it would send the exception stacktrace to the

[jira] [Created] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
Xing Lin created HDFS-17332: --- Summary: DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException Key: HDFS-17332 URL:

[jira] [Updated] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-09 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xing Lin updated HDFS-17332: Description: In DFSInputStream#actualGetFromOneDataNode(), it would send the exception stacktrace to the

[jira] [Commented] (HDFS-17332) DFSInputStream: avoid logging stacktrace until when we really need to fail a read request with a MissingBlockException

2024-01-12 Thread Xing Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806170#comment-17806170 ] Xing Lin commented on HDFS-17332: - In this jira, we only addressed the above issue for DFSInputStream. We

  1   2   >