[jira] [Updated] (HDFS-11345) Document the configuration key for FSNamesystem lock fairness

[ https://issues.apache.org/jira/browse/HDFS-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11345: --- Attachment: HADOOP-11345.001.patch v000 patch carelessly only updated the {{hdfs-default.xml}}

[jira] [Updated] (HDFS-11290) TestFSNameSystemMBean should wait until the cache is cleared

[ https://issues.apache.org/jira/browse/HDFS-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11290: --- Attachment: HDFS-11290.001.patch Thanks for the pointer [~ajisakaa]! I missed that completely. I've

[jira] [Created] (HDFS-11352) Potential deadlock in NN when failing over

Erik Krogen created HDFS-11352: -- Summary: Potential deadlock in NN when failing over Key: HDFS-11352 URL: https://issues.apache.org/jira/browse/HDFS-11352 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-11352) Potential deadlock in NN when failing over

[ https://issues.apache.org/jira/browse/HDFS-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11352: --- Attachment: HDFS-11352-branch-2.7.000.patch Attaching patch with the (one-line) fix. > Potential

[jira] [Updated] (HDFS-11352) Potential deadlock in NN when failing over

[ https://issues.apache.org/jira/browse/HDFS-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11352: --- Status: Patch Available (was: Open) > Potential deadlock in NN when failing over >

[jira] [Commented] (HDFS-11352) Potential deadlock in NN when failing over

[ https://issues.apache.org/jira/browse/HDFS-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832057#comment-15832057 ] Erik Krogen commented on HDFS-11352: Whitespace issues and ASF license are clearly not related. All of

[jira] [Commented] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886094#comment-15886094 ] Erik Krogen commented on HDFS-10798: My understanding of the intuition for why they would be different

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.004.patch > Add Logging for Long-held NN Read Locks >

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453363#comment-15453363 ] Erik Krogen commented on HDFS-10817: It is true that you would see an NPE. I initially ignored this

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453654#comment-15453654 ] Erik Krogen commented on HDFS-10817: Great, thanks Zhe! > Add Logging for Long-held NN Read Locks >

[jira] [Comment Edited] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453863#comment-15453863 ] Erik Krogen edited comment on HDFS-10817 at 9/1/16 12:32 AM: - Discovered that

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.addendum.000.patch Discovered that the test fails on branch-2 because the logs

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459771#comment-15459771 ] Erik Krogen commented on HDFS-10713: In HDFS-10475 we hope to add more detailed metrics for which

[jira] [Created] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

Erik Krogen created HDFS-10843: -- Summary: Quota Feature Cached Size != Computed Size When Block Committed But Not Completed Key: HDFS-10843 URL: https://issues.apache.org/jira/browse/HDFS-10843 Project:

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468414#comment-15468414 ] Erik Krogen commented on HDFS-10817: Right, the {{getReadHoldCount}} vs. {{getReadLockCount}}

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468631#comment-15468631 ] Erik Krogen commented on HDFS-10843: I will attach a patch containing a unit test demonstrating the

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469153#comment-15469153 ] Erik Krogen commented on HDFS-10843: After further consideration it seems that it is probably most

[jira] [Comment Edited] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15469153#comment-15469153 ] Erik Krogen edited comment on HDFS-10843 at 9/7/16 1:11 AM: After further

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468294#comment-15468294 ] Erik Krogen commented on HDFS-10817: [~arpitagarwal], thank you for the concern but I do not believe

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468993#comment-15468993 ] Erik Krogen commented on HDFS-10843: I have realized that this bug is more severe than I previously

[jira] [Assigned] (HDFS-10829) Improve Testing for NN Lock Logging

[ https://issues.apache.org/jira/browse/HDFS-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-10829: -- Assignee: Erik Krogen > Improve Testing for NN Lock Logging >

[jira] [Created] (HDFS-10829) Improve Testing for NN Lock Logging

Erik Krogen created HDFS-10829: -- Summary: Improve Testing for NN Lock Logging Key: HDFS-10829 URL: https://issues.apache.org/jira/browse/HDFS-10829 Project: Hadoop HDFS Issue Type: Test

[jira] [Updated] (HDFS-10829) Improve Testing for NN Lock Logging

[ https://issues.apache.org/jira/browse/HDFS-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10829: --- Description: Currently the testing for the namenode logging when locks are held for a long time is a

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457055#comment-15457055 ] Erik Krogen commented on HDFS-10817: Created HDFS-10829 to address concerns in testing noted in Zhe's

[jira] [Created] (HDFS-10817) Add Logging for Long-held NN Read Locks

Erik Krogen created HDFS-10817: -- Summary: Add Logging for Long-held NN Read Locks Key: HDFS-10817 URL: https://issues.apache.org/jira/browse/HDFS-10817 Project: Hadoop HDFS Issue Type:

[jira] [Work started] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10817 started by Erik Krogen. -- > Add Logging for Long-held NN Read Locks > ---

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455994#comment-15455994 ] Erik Krogen commented on HDFS-10817: Thanks [~zhz]! But I disagree that spinning up multiple threads

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450414#comment-15450414 ] Erik Krogen commented on HDFS-10817: Added a patch to implement this with a configurable threshold.

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.000.patch > Add Logging for Long-held NN Read Locks >

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.001.patch Uploaded new patch that doesn't use semaphores; forgot about Java's

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Status: Patch Available (was: In Progress) > Add Logging for Long-held NN Read Locks >

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450738#comment-15450738 ] Erik Krogen commented on HDFS-10817: Thanks for the comments [~zhz]! Uploading a new patch addressing

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.002.patch > Add Logging for Long-held NN Read Locks >

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.003.patch Test failure does not seem related. Updated patch removing some

[jira] [Comment Edited] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485496#comment-15485496 ] Erik Krogen edited comment on HDFS-10843 at 9/13/16 5:08 PM: - v002 patch

[jira] [Commented] (HDFS-10475) Adding metrics for long FSNamesystem read and write locks

[ https://issues.apache.org/jira/browse/HDFS-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491251#comment-15491251 ] Erik Krogen commented on HDFS-10475: Thanks for the pointer to RpcDetailedActivity, [~andrew.wang].

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491505#comment-15491505 ] Erik Krogen commented on HDFS-10843: What advantage is there to wrapping those two specific calls in a

[jira] [Commented] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475060#comment-15475060 ] Erik Krogen commented on HDFS-10742: For v16 patch IIUC you actually still have the default set at 2s

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475013#comment-15475013 ] Erik Krogen commented on HDFS-10843: TestDFSStripedOutputStreamWithFailure appears unrelated and

[jira] [Comment Edited] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475060#comment-15475060 ] Erik Krogen edited comment on HDFS-10742 at 9/8/16 9:26 PM: For v16 patch IIUC

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Status: Patch Available (was: Open) I am attaching a patch which includes 6 new tests in

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.001.patch > Quota Feature Cached Size != Computed Size When Block Committed

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.000.patch > Quota Feature Cached Size != Computed Size When Block Committed

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475609#comment-15475609 ] Erik Krogen commented on HDFS-10843: 1. The problem is that most calls to completeBlock() originate

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488666#comment-15488666 ] Erik Krogen commented on HDFS-10843: Thanks for the comments. I am a little unsure about point 3. It

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.004.patch > Quota Feature Cached Size != Computed Size When Block Committed

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488521#comment-15488521 ] Erik Krogen commented on HDFS-10713: One comment on this is that it would be ideal to change it to use

[jira] [Comment Edited] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485496#comment-15485496 ] Erik Krogen edited comment on HDFS-10843 at 9/12/16 10:35 PM: -- v002 patch

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.002.patch v3 patch incorporates your suggestions, [~shv]. Tests now only start

[jira] [Comment Edited] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485496#comment-15485496 ] Erik Krogen edited comment on HDFS-10843 at 9/12/16 10:34 PM: -- v002 patch

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.003.patch Seems that I accidentally uploaded a slightly out of date patch file

[jira] [Comment Edited] (HDFS-10475) Adding metrics for long FSNamesystem read and write locks

[ https://issues.apache.org/jira/browse/HDFS-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485649#comment-15485649 ] Erik Krogen edited comment on HDFS-10475 at 9/12/16 11:47 PM: -- To get a

[jira] [Commented] (HDFS-10475) Adding metrics for long FSNamesystem read and write locks

[ https://issues.apache.org/jira/browse/HDFS-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485649#comment-15485649 ] Erik Krogen commented on HDFS-10475: To get a mapping of operation -> lock time metrics we propose the

[jira] [Comment Edited] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491505#comment-15491505 ] Erik Krogen edited comment on HDFS-10843 at 9/14/16 10:26 PM: -- What advantage

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491623#comment-15491623 ] Erik Krogen commented on HDFS-10843: But you shouldn't be calling {{BlockInfo.convertToCompleteBlock}}

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.000.patch Attaching v000 patch. Note that this will conflict heavily with

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Status: Patch Available (was: Open) > Move lock logging logic from FSNamesystem into

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.001.patch Rebased for patch v001. > Move lock logging logic from FSNamesystem

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.003.patch Attached v003 patch with checkstyle fix. Thanks for the review,

[jira] [Commented] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533396#comment-15533396 ] Erik Krogen commented on HDFS-10921: [~shahrs87], the concern about namespace pollution is valid, but

[jira] [Comment Edited] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533396#comment-15533396 ] Erik Krogen edited comment on HDFS-10921 at 9/29/16 5:22 PM: - [~shahrs87], the

[jira] [Commented] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533813#comment-15533813 ] Erik Krogen commented on HDFS-10921: This is certainly true and something I discussed with Konstantin

[jira] [Commented] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533959#comment-15533959 ] Erik Krogen commented on HDFS-10921: True, agreed that it is definitely easy to accidentally miss the

[jira] [Commented] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552636#comment-15552636 ] Erik Krogen commented on HDFS-10921: [~shahrs87], I tried that previously, but when running the entire

[jira] [Comment Edited] (HDFS-10921) TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode

[ https://issues.apache.org/jira/browse/HDFS-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552636#comment-15552636 ] Erik Krogen edited comment on HDFS-10921 at 10/6/16 5:50 PM: - [~shahrs87], I

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843.005.patch After offline discussion with Konstantin, attaching v005 patch with

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504800#comment-15504800 ] Erik Krogen commented on HDFS-10843: Test failure does not appear to be related and runs successfully

[jira] [Commented] (HDFS-10475) Adding metrics for long FSNamesystem read and write locks

[ https://issues.apache.org/jira/browse/HDFS-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504617#comment-15504617 ] Erik Krogen commented on HDFS-10475: After offline discussion with Zhe we have decided it would be

[jira] [Created] (HDFS-10873) Add histograms for FSNamesystemLock Metrics

Erik Krogen created HDFS-10873: -- Summary: Add histograms for FSNamesystemLock Metrics Key: HDFS-10873 URL: https://issues.apache.org/jira/browse/HDFS-10873 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10843: --- Attachment: HDFS-10843-branch-2.7.005.patch Attaching patch for branch-2.7 > Quota Feature Cached

[jira] [Commented] (HDFS-10843) Quota Feature Cached Size != Computed Size When Block Committed But Not Completed

[ https://issues.apache.org/jira/browse/HDFS-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516987#comment-15516987 ] Erik Krogen commented on HDFS-10843: findbugs - all unrelated to this patch whitespace - all unrelated

[jira] [Created] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

Erik Krogen created HDFS-10896: -- Summary: Move lock logging logic from FSNamesystem into FSNamesystemLock Key: HDFS-10896 URL: https://issues.apache.org/jira/browse/HDFS-10896 Project: Hadoop HDFS

[jira] [Work started] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10872 started by Erik Krogen. -- > Add MutableRate metrics for FSNamesystemLock operations >

[jira] [Created] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

Erik Krogen created HDFS-10872: -- Summary: Add MutableRate metrics for FSNamesystemLock operations Key: HDFS-10872 URL: https://issues.apache.org/jira/browse/HDFS-10872 Project: Hadoop HDFS

[jira] [Commented] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439434#comment-15439434 ] Erik Krogen commented on HDFS-10798: 1. Fair. It seems many keys in DFSConfigKeys follow the "replace

[jira] [Updated] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10798: --- Attachment: HDFS-10789.002.patch > Make the threshold of reporting FSNamesystem lock contention

[jira] [Commented] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439732#comment-15439732 ] Erik Krogen commented on HDFS-10798: Thanks [~zhz]! > Make the threshold of reporting FSNamesystem

[jira] [Work started] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10798 started by Erik Krogen. -- > Make the threshold of reporting FSNamesystem lock contention configurable >

[jira] [Updated] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10798: --- Status: Patch Available (was: In Progress) Add configuration

[jira] [Updated] (HDFS-10798) Make the threshold of reporting FSNamesystem lock contention configurable

[ https://issues.apache.org/jira/browse/HDFS-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10798: --- Attachment: HDFS-10789.001.patch > Make the threshold of reporting FSNamesystem lock contention

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.004.patch After looking at HDFS-5239 it turns out the lack of documentation

[jira] [Commented] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536508#comment-15536508 ] Erik Krogen commented on HDFS-10896: The constructor which accepts a Timer parameter is used for

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: (was: HDFS-10896.004.patch) > Move lock logging logic from FSNamesystem into

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.004.patch > Move lock logging logic from FSNamesystem into FSNamesystemLock >

[jira] [Updated] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10872: --- Attachment: FSLockPerf.java Some comments on performance considerations... I am attaching code

[jira] [Updated] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10872: --- Status: Patch Available (was: Open) > Add MutableRate metrics for FSNamesystemLock operations >

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896-branch-2.7.004.patch Attaching branch-2.7 patch. Backport wasn't as messy as

[jira] [Updated] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10872: --- Status: Patch Available (was: In Progress) > Add MutableRate metrics for FSNamesystemLock operations

[jira] [Updated] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10872: --- Attachment: HDFS-10872.000.patch > Add MutableRate metrics for FSNamesystemLock operations >

[jira] [Updated] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10872: --- Status: Open (was: Patch Available) > Add MutableRate metrics for FSNamesystemLock operations >

[jira] [Commented] (HDFS-10872) Add MutableRate metrics for FSNamesystemLock operations

[ https://issues.apache.org/jira/browse/HDFS-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537392#comment-15537392 ] Erik Krogen commented on HDFS-10872: Attaching v000 patch. This makes {{FSNamesystemLock}} into a

[jira] [Comment Edited] (HDFS-10713) Throttle FsNameSystem lock warnings

[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526786#comment-15526786 ] Erik Krogen edited comment on HDFS-10713 at 9/27/16 5:13 PM: - [~arpitagarwal],

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526786#comment-15526786 ] Erik Krogen commented on HDFS-10713: [~arpitagarwal], I'm confident that my HDFS-10817 patch would not

[jira] [Comment Edited] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527007#comment-15527007 ] Erik Krogen edited comment on HDFS-10896 at 9/27/16 6:18 PM: - Attaching v002

[jira] [Updated] (HDFS-10896) Move lock logging logic from FSNamesystem into FSNamesystemLock

[ https://issues.apache.org/jira/browse/HDFS-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10896: --- Attachment: HDFS-10896.002.patch Attaching v002 patch that rebases on top of HDFS-10713 and also

[jira] [Comment Edited] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608923#comment-15608923 ] Erik Krogen edited comment on HDFS-9500 at 10/26/16 4:25 PM: - [~shv], thanks

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-9500: -- Attachment: HDFS-9500.001.patch [~shv], thanks for the review. I have added Javadocs. For

[jira] [Commented] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606385#comment-15606385 ] Erik Krogen commented on HDFS-9500: --- The {{TestDiskspaceQuotaUpdate}} failure is unrelated and documented

[jira] [Updated] (HDFS-9500) datanodesSoftwareVersions map may counting wrong when rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-9500: -- Attachment: HDFS-9500.002.patch Good catch, [~raviprak]! Attaching v002 patch. Thanks! >

[jira] [Updated] (HDFS-11087) NamenodeFsck should check if the output writer is still writable.

[ https://issues.apache.org/jira/browse/HDFS-11087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-11087: --- Attachment: HDFS-11087.branch-2.000.patch Attaching v000 patch. Since {{checkError}} internally

  1   2   3   4   5   6   7   8   9   10   >