[jira] [Resolved] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2024-01-05 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-14305. Resolution: Fixed > Serial number in BlockTokenSecretManager could overlap between

[jira] [Commented] (HDFS-16517) In 2.10 the distance metric is wrong for non-DN machines

2022-03-22 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511011#comment-17511011 ] Konstantin Shvachko commented on HDFS-16517: Looks like the same issue as HADOOP-16161, as

[jira] [Updated] (HDFS-10650) DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission

2022-02-15 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-10650: --- Fix Version/s: 2.10.2 Merged this into branch-2.10. Updated fix version. >

[jira] [Commented] (HDFS-16322) The NameNode implementation of ClientProtocol.truncate(...) can cause data loss.

2021-11-15 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444033#comment-17444033 ] Konstantin Shvachko commented on HDFS-16322: Is it specific to truncate? Same thing should

[jira] [Resolved] (HDFS-7612) TestOfflineEditsViewer.testStored() uses incorrect default value for cacheDir

2021-10-21 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-7612. --- Fix Version/s: 3.2.4 3.3.2 2.10.2

[jira] [Assigned] (HDFS-7612) TestOfflineEditsViewer.testStored() uses incorrect default value for cacheDir

2021-10-21 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reassigned HDFS-7612: - Assignee: Michael Kuchenbecker > TestOfflineEditsViewer.testStored() uses

[jira] [Commented] (HDFS-13150) [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC

2021-09-30 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423048#comment-17423048 ] Konstantin Shvachko commented on HDFS-13150: We end up implementing quorum read from JNs for

[jira] [Comment Edited] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC

2021-09-29 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422417#comment-17422417 ] Konstantin Shvachko edited comment on HDFS-16220 at 9/29/21, 11:22 PM:

[jira] [Commented] (HDFS-16228) [FGL]Improve safer PartitionedGSet#size

2021-09-29 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422419#comment-17422419 ] Konstantin Shvachko commented on HDFS-16228: Atomic variables have the same semantics as

[jira] [Commented] (HDFS-16220) [FGL]Configurable INodeMap#NAMESPACE_KEY_DEPTH_RANGES_STATIC

2021-09-29 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422417#comment-17422417 ] Konstantin Shvachko commented on HDFS-16220: # I don't think we should make Key depth

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2021-09-10 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-14216: --- Fix Version/s: 2.10.2 Also saw this NPE on branch-2.10. Back-port is clean. Adding

[jira] [Commented] (HDFS-16211) Complete some descriptions related to AuthToken

2021-09-08 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412096#comment-17412096 ] Konstantin Shvachko commented on HDFS-16211: Hi [~jianghuazhu]. Thanks for contributing.

[jira] [Resolved] (HDFS-16141) [FGL] Address permission related issues with File / Directory

2021-08-13 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-16141. Fix Version/s: Fine-Grained Locking Hadoop Flags: Reviewed Resolution:

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-30 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390818#comment-17390818 ] Konstantin Shvachko commented on HDFS-14703: Some thoughts on [~daryn]'s comment: * For small

[jira] [Assigned] (HDFS-14540) Block deletion failure causes an infinite polling in TestDeleteBlockPool

2021-07-30 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reassigned HDFS-14540: -- Assignee: Anton Kutuzov > Block deletion failure causes an infinite polling

[jira] [Commented] (HDFS-14540) Block deletion failure causes an infinite polling in TestDeleteBlockPool

2021-07-30 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390757#comment-17390757 ] Konstantin Shvachko commented on HDFS-14540: I think we should just check the return value of

[jira] [Updated] (HDFS-16130) [FGL] Implement Create File with FGL

2021-07-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-16130: --- Fix Version/s: Fine-Grained Locking > [FGL] Implement Create File with FGL >

[jira] [Resolved] (HDFS-16130) [FGL] Implement Create File with FGL

2021-07-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-16130. Hadoop Flags: Reviewed Resolution: Fixed I just committed this. Fixed a few

[jira] [Resolved] (HDFS-16128) [FGL] Add support for saving/loading an FS Image for PartitionedGSet

2021-07-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-16128. Fix Version/s: Fine-Grained Locking Hadoop Flags: Reviewed Resolution:

[jira] [Resolved] (HDFS-16125) [FGL] Fix the iterator for PartitionedGSet

2021-07-16 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-16125. Fix Version/s: Fine-Grained Locking Hadoop Flags: Reviewed Resolution:

[jira] [Updated] (HDFS-16125) [FGL] Fix the iterator for PartitionedGSet

2021-07-14 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-16125: --- Summary: [FGL] Fix the iterator for PartitionedGSet (was: fix the iterator for

[jira] [Updated] (HDFS-16125) fix the iterator for PartitionedGSet

2021-07-14 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-16125: --- Parent: HDFS-14703 Issue Type: Sub-task (was: Bug) > fix the iterator for

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-14 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345870#comment-17345870 ] Konstantin Shvachko edited comment on HDFS-14703 at 7/14/21, 5:28 PM:

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-14 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380757#comment-17380757 ] Konstantin Shvachko commented on HDFS-14703: ??Shall I raise separate Jira for Create and

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-07-09 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17378314#comment-17378314 ] Konstantin Shvachko commented on HDFS-14703: Great progress [~prasad-acit]. It proves the

[jira] [Updated] (HDFS-16040) RpcQueueTime metric counts requeued calls as unique events.

2021-05-27 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-16040: --- Fix Version/s: 3.3.2 3.2.3 2.10.2

[jira] [Commented] (HDFS-16040) RpcQueueTime metric counts requeued calls as unique events.

2021-05-27 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352920#comment-17352920 ] Konstantin Shvachko commented on HDFS-16040: Wow - clean build. Adding my +1 to that. Will

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-27 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352653#comment-17352653 ] Konstantin Shvachko commented on HDFS-15915: [~daryn] would appreciate your review. > Race

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-26 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Fix Version/s: 3.3.2 3.2.3 2.10.2

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-26 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352023#comment-17352023 ] Konstantin Shvachko commented on HDFS-15915: Ran unit tests that failed on Jenkins. All

[jira] [Commented] (HDFS-16040) RpcQueueTime metric counts requeued calls as unique events.

2021-05-26 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351510#comment-17351510 ] Konstantin Shvachko commented on HDFS-16040: I checked that the test is fails without the fix

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-25 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351284#comment-17351284 ] Konstantin Shvachko commented on HDFS-15915: Thanks for thorough review [~virajith]. BTW,

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-25 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: HDFS-15915-05.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347336#comment-17347336 ] Konstantin Shvachko commented on HDFS-15915: Updated the patch per [~virajith]'s suggestions.

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-05-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: HDFS-15915-04.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-17 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346379#comment-17346379 ] Konstantin Shvachko commented on HDFS-14703: Thanks [~prasad-acit] and [~xinglin] for

[jira] [Commented] (HDFS-16004) BackupNode and QJournal lack Permission check.

2021-05-10 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342108#comment-17342108 ] Konstantin Shvachko commented on HDFS-16004: Hey guys. I wouldn't worry about {{BackupNode}}.

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-10 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341089#comment-17341089 ] Konstantin Shvachko edited comment on HDFS-14703 at 5/10/21, 7:30 PM:

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341089#comment-17341089 ] Konstantin Shvachko edited comment on HDFS-14703 at 5/8/21, 1:05 AM: -

[jira] [Comment Edited] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341089#comment-17341089 ] Konstantin Shvachko edited comment on HDFS-14703 at 5/8/21, 1:04 AM: -

[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341089#comment-17341089 ] Konstantin Shvachko commented on HDFS-14703: Updated the POC patches. There were indeed some

[jira] [Updated] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

2021-05-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-14703: --- Attachment: 003-partitioned-inodeMap-POC.tar.gz > NameNode Fine-Grained Locking via

[jira] [Commented] (HDFS-16001) TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes

2021-05-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341046#comment-17341046 ] Konstantin Shvachko commented on HDFS-16001: Checked the test. This fixes it. +1 thanks

[jira] [Commented] (HDFS-15912) Allow ProtobufRpcEngine to be extensible

2021-04-30 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337546#comment-17337546 ] Konstantin Shvachko commented on HDFS-15912: Since all changes are in {{hadoop-common}} this

[jira] [Updated] (HDFS-15652) Make block size from NNThroughputBenchmark configurable

2021-04-29 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15652: --- Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-29 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: HDFS-15915-03.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-15652) Make block size from NNThroughputBenchmark configurable

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335066#comment-17335066 ] Konstantin Shvachko commented on HDFS-15652: I would like to backport this to earlier

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335052#comment-17335052 ] Konstantin Shvachko commented on HDFS-15915: Patch v.02 fixes findbugs and white space

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: HDFS-15915-02.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-16001) TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335051#comment-17335051 ] Konstantin Shvachko commented on HDFS-16001: This fails consistently on trunk but not in

[jira] [Created] (HDFS-16001) TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes

2021-04-28 Thread Konstantin Shvachko (Jira)
Konstantin Shvachko created HDFS-16001: -- Summary: TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes Key: HDFS-16001 URL: https://issues.apache.org/jira/browse/HDFS-16001

[jira] [Updated] (HDFS-16001) TestOfflineEditsViewer.testStored() fails reading negative value of FSEditLogOpCodes

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-16001: --- Docs Text: (was: {{TestOfflineEditsViewer.testStored()}} fails consistently with

[jira] [Commented] (HDFS-7612) TestOfflineEditsViewer.testStored() uses incorrect default value for cacheDir

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335043#comment-17335043 ] Konstantin Shvachko commented on HDFS-7612: --- Came across this again. Still not fixed. We just

[jira] [Updated] (HDFS-7612) TestOfflineEditsViewer.testStored() uses incorrect default value for cacheDir

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-7612: -- Labels: newbie (was: ) > TestOfflineEditsViewer.testStored() uses incorrect default

[jira] [Assigned] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko reassigned HDFS-15915: -- Assignee: Konstantin Shvachko > Race condition with async edits logging due

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Status: Patch Available (was: Open) > Race condition with async edits logging due

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: HDFS-15915-01.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-04-28 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334483#comment-17334483 ] Konstantin Shvachko commented on HDFS-15915: Attaching a patch to fix the problem. The is a

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-03-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Target Version/s: 2.10.2 > Race condition with async edits logging due to updating

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-03-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307463#comment-17307463 ] Konstantin Shvachko commented on HDFS-15915: Attached the test reproducing the bug. Looks

[jira] [Updated] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-03-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15915: --- Attachment: testMkdirsRace.patch > Race condition with async edits logging due to

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-03-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307458#comment-17307458 ] Konstantin Shvachko commented on HDFS-15915: # Suppose two {{mkdirs}} for the same path are

[jira] [Created] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

2021-03-23 Thread Konstantin Shvachko (Jira)
Konstantin Shvachko created HDFS-15915: -- Summary: Race condition with async edits logging due to updating txId outside of the namesystem log Key: HDFS-15915 URL:

[jira] [Commented] (HDFS-14731) [FGL] Remove redundant locking on NameNode.

2021-03-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305081#comment-17305081 ] Konstantin Shvachko commented on HDFS-14731: Sure [~xilangyan] I was thinking about back

[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-03-07 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17296958#comment-17296958 ] Konstantin Shvachko commented on HDFS-15808: Hey there is no way to modify a commit unless we

[jira] [Updated] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-03-06 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15808: --- Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Updated] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-03-06 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15808: --- Status: Patch Available (was: Open) > Add metrics for FSNamesystem read/write lock

[jira] [Updated] (HDFS-15849) ExpiredHeartbeats metric should be of Type.COUNTER

2021-03-01 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15849: --- Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-03-01 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17293268#comment-17293268 ] Konstantin Shvachko commented on HDFS-15808: [~tomscut] I have problem merging this to

[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-22 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288566#comment-17288566 ] Konstantin Shvachko commented on HDFS-15808: +1 on pull request. Created HDFS-15849 to fix

[jira] [Created] (HDFS-15849) ExpiredHeartbeats metric should be of Type.COUNTER

2021-02-22 Thread Konstantin Shvachko (Jira)
Konstantin Shvachko created HDFS-15849: -- Summary: ExpiredHeartbeats metric should be of Type.COUNTER Key: HDFS-15849 URL: https://issues.apache.org/jira/browse/HDFS-15849 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287396#comment-17287396 ] Konstantin Shvachko commented on HDFS-15808: [~tomscut] sure we all use different systems for

[jira] [Updated] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15808: --- Attachment: ExpiredHeartbeat.png > Add metrics for FSNamesystem read/write lock hold

[jira] [Updated] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15808: --- Attachment: (was: ExpiredHeartbeat.png) > Add metrics for FSNamesystem

[jira] [Updated] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15808: --- Attachment: ExpiredHeartbeat.png > Add metrics for FSNamesystem read/write lock

[jira] [Commented] (HDFS-15808) Add metrics for FSNamesystem read/write lock hold long time

2021-02-17 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286133#comment-17286133 ] Konstantin Shvachko commented on HDFS-15808: Hey [~tomscut]. The patch looks fine, but I

[jira] [Commented] (HDFS-15792) ClasscastException while loading FSImage

2021-02-09 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282003#comment-17282003 ] Konstantin Shvachko commented on HDFS-15792: Hey guys, sorry for coming late to this. But we

[jira] [Resolved] (HDFS-15632) AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases.

2021-01-22 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-15632. Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Resolved] (HDFS-954) There are two security packages in hdfs, should be one

2021-01-21 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-954. -- Resolution: Won't Fix Hey [~antn.kutuzov] this is a rather old jira. I don't think it is

[jira] [Commented] (HDFS-15632) AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases.

2021-01-21 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17269631#comment-17269631 ] Konstantin Shvachko commented on HDFS-15632: +1 PR looks good. Will commit in a bit. >

[jira] [Commented] (HDFS-15751) Add documentation for msync() API to filesystem.md

2021-01-19 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17268314#comment-17268314 ] Konstantin Shvachko commented on HDFS-15751: Created HADOOP-17477 to track implementation of

[jira] [Commented] (HDFS-15751) Add documentation for msync() API to filesystem.md

2021-01-03 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257840#comment-17257840 ] Konstantin Shvachko commented on HDFS-15751: Thanks guys for taking care of that. > Add

[jira] [Updated] (HDFS-15751) Add documentation for msync() API to filesystem.md

2020-12-24 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15751: --- Status: Patch Available (was: Open) > Add documentation for msync() API to

[jira] [Commented] (HDFS-15751) Add documentation for msync() API to filesystem.md

2020-12-24 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254662#comment-17254662 ] Konstantin Shvachko commented on HDFS-15751: Added documentation for {{msync()}}. I put a

[jira] [Updated] (HDFS-15751) Add documentation for msync() API to filesystem.md

2020-12-24 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15751: --- Attachment: HDFS-15751-01.patch > Add documentation for msync() API to filesystem.md

[jira] [Created] (HDFS-15751) Add documentation for msync() API to filesystem.md

2020-12-24 Thread Konstantin Shvachko (Jira)
Konstantin Shvachko created HDFS-15751: -- Summary: Add documentation for msync() API to filesystem.md Key: HDFS-15751 URL: https://issues.apache.org/jira/browse/HDFS-15751 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15746) Standby NameNode crash when replay editlog

2020-12-24 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254641#comment-17254641 ] Konstantin Shvachko commented on HDFS-15746: Looks like your branch is either quite old or

[jira] [Commented] (HDFS-15746) Standby NameNode crash when replay editlog

2020-12-23 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254245#comment-17254245 ] Konstantin Shvachko commented on HDFS-15746: [~hexiaoqiao] thanks for reporting this. Which

[jira] [Commented] (HDFS-15567) [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly.

2020-12-13 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248677#comment-17248677 ] Konstantin Shvachko commented on HDFS-15567: Hey Steve, I am not sure I fully understand what

[jira] [Comment Edited] (HDFS-15567) [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly.

2020-12-13 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17210550#comment-17210550 ] Konstantin Shvachko edited comment on HDFS-15567 at 12/13/20, 9:11 PM:

[jira] [Commented] (HDFS-4452) getAdditionalBlock() can create multiple blocks if the client times out and retries.

2020-11-25 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17238894#comment-17238894 ] Konstantin Shvachko commented on HDFS-4452: --- This is an interesting observation [~honestman].

[jira] [Commented] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

2020-11-09 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228786#comment-17228786 ] Konstantin Shvachko commented on HDFS-15562: Hey guys, I think generally the checkpointer

[jira] [Resolved] (HDFS-15623) Respect configured values of rpc.engine

2020-11-06 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved HDFS-15623. Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Commented] (HDFS-15623) Respect configured values of rpc.engine

2020-11-06 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227602#comment-17227602 ] Konstantin Shvachko commented on HDFS-15623: Was looking into this a bit more. * So the

[jira] [Updated] (HDFS-15665) Balancer logging improvement

2020-11-03 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15665: --- Fix Version/s: 3.2.3 2.10.2 3.1.5

[jira] [Updated] (HDFS-15665) Balancer logging improvement

2020-11-02 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15665: --- Attachment: HDFS-15665.002.patch > Balancer logging improvement >

[jira] [Commented] (HDFS-15665) Balancer logging improvement

2020-11-02 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17225092#comment-17225092 ] Konstantin Shvachko commented on HDFS-15665: Thanks for the review [~vagarychen]. *

[jira] [Commented] (HDFS-15665) Balancer logging improvement

2020-11-02 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224959#comment-17224959 ] Konstantin Shvachko commented on HDFS-15665: Attached a patch * It logs additional config

[jira] [Updated] (HDFS-15665) Balancer logging improvement

2020-11-02 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15665: --- Status: Patch Available (was: Open) > Balancer logging improvement >

[jira] [Updated] (HDFS-15665) Balancer logging improvement

2020-11-02 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-15665: --- Attachment: HDFS-15665.001.patch > Balancer logging improvement >

  1   2   3   4   5   6   7   8   9   10   >