[jira] [Updated] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12684: --- Attachment: HDFS-12684-HDFS-7240.001.patch > Ozone: SCM metrics NodeCount is overlapping with node

[jira] [Created] (HDFS-12685) FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
Ewan Higgs created HDFS-12685: - Summary: FsVolumeImpl exception when scanning Provided storage volume Key: HDFS-12685 URL: https://issues.apache.org/jira/browse/HDFS-12685 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy

2017-10-19 Thread Takanobu Asanuma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HDFS-12677: Status: Patch Available (was: Open) > Extend TestReconstructStripedFile with a random EC

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210642#comment-16210642 ] Xiao Chen commented on HDFS-7337: - Got it, will look into those. Thanks for the prompt response! >

[jira] [Updated] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy

2017-10-19 Thread Takanobu Asanuma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HDFS-12677: Attachment: HDFS-12677.1.patch Uploaded the 1st patch. The new test class with a random ec

[jira] [Commented] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210685#comment-16210685 ] Hadoop QA commented on HDFS-12684: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210765#comment-16210765 ] Yiqun Lin edited comment on HDFS-11468 at 10/19/17 9:27 AM: Thanks for the

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-10-19 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210634#comment-16210634 ] Rakesh R commented on HDFS-7337: bq. Is there a jira for persisting EC policy to NN metadata? HDFS-7859,

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-10-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210640#comment-16210640 ] SammiChen commented on HDFS-7337: - Hi [~xiaochen], HDFS-7859 is for persist EC policy in protobuffer

[jira] [Created] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Weiwei Yang (JIRA)
Weiwei Yang created HDFS-12684: -- Summary: Ozone: SCM metrics NodeCount is overlapping with node manager metrics Key: HDFS-12684 URL: https://issues.apache.org/jira/browse/HDFS-12684 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12684: --- Status: Patch Available (was: Open) > Ozone: SCM metrics NodeCount is overlapping with node manager

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210790#comment-16210790 ] SammiChen commented on HDFS-12682: -- Hi [~xiaochen], thanks for reporting this issue. Inspired by your

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210630#comment-16210630 ] Xiao Chen commented on HDFS-12682: -- Familiarized myself with the code more, and investigated further.

[jira] [Commented] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210662#comment-16210662 ] Yiqun Lin commented on HDFS-12680: -- Hi [~nandakumar131], besides [~anu]'s review comments, some other

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-19 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210677#comment-16210677 ] Hudson commented on HDFS-12502: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13105 (See

[jira] [Updated] (HDFS-12685) [READ] FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12685: -- Description: I left a Datanode running overnight and found this in the logs in the morning: {code}

[jira] [Updated] (HDFS-12685) [READ] FsVolumeImpl exception when scanning Provided storage volume

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12685: -- Summary: [READ] FsVolumeImpl exception when scanning Provided storage volume (was: FsVolumeImpl

[jira] [Created] (HDFS-12686) Erasure coding system policy state is not correctly saved and loaded during real cluster restart

2017-10-19 Thread SammiChen (JIRA)
SammiChen created HDFS-12686: Summary: Erasure coding system policy state is not correctly saved and loaded during real cluster restart Key: HDFS-12686 URL: https://issues.apache.org/jira/browse/HDFS-12686

[jira] [Commented] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210765#comment-16210765 ] Yiqun Lin commented on HDFS-11468: -- Thanks for the response, [~cheersyang]. I am thinking for this again

[jira] [Assigned] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang reassigned HDFS-12684: -- Assignee: Weiwei Yang > Ozone: SCM metrics NodeCount is overlapping with node manager metrics

[jira] [Commented] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210802#comment-16210802 ] Hadoop QA commented on HDFS-12677: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210657#comment-16210657 ] Hadoop QA commented on HDFS-12502: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12521) Ozone: SCM should read all Container info into memory when booting up

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210674#comment-16210674 ] Yiqun Lin commented on HDFS-12521: -- Thanks [~ljain] for the updating patch. The change in v03 patch looks

[jira] [Updated] (HDFS-10984) Expose nntop output as metrics

2017-10-19 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10984: - Fix Version/s: 2.7.5 Thanks [~swagle] and [~xyao]. I backported this to branch-2.7 as well. > Expose

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-19 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0

[jira] [Assigned] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs reassigned HDFS-12665: - Assignee: Ewan Higgs > [AliasMap] Create a version of the AliasMap that runs in memory in the >

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Attachment: HDFS-12665-HDFS-9806.001.patch Attaching work from WDC implementing the In Memory AliasMap.

[jira] [Commented] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210870#comment-16210870 ] Weiwei Yang commented on HDFS-11468: Hi [~linyiqun] sounds good to me, please go ahead submitting a

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Status: Patch Available (was: Open) > [AliasMap] Create a version of the AliasMap that runs in memory

[jira] [Commented] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210890#comment-16210890 ] Hadoop QA commented on HDFS-12665: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-12688) HDFS File Not Removed Despite Successful "Moved to .Trash" Message

2017-10-19 Thread Shriya Gupta (JIRA)
Shriya Gupta created HDFS-12688: --- Summary: HDFS File Not Removed Despite Successful "Moved to .Trash" Message Key: HDFS-12688 URL: https://issues.apache.org/jira/browse/HDFS-12688 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12688) HDFS File Not Removed Despite Successful "Moved to .Trash" Message

2017-10-19 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211729#comment-16211729 ] Jason Lowe commented on HDFS-12688: --- Have you checked the HDFS audit logs? That should give you clues

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211473#comment-16211473 ] Xiao Chen commented on HDFS-12682: -- Thanks for the response Sammi, good find on HDFS-12686! I propose

[jira] [Updated] (HDFS-12663) Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12663: --- Attachment: HDFS-12663-HDFS-7240.002.patch > Ozone: OzoneClient: Remove protobuf classes exposed to

[jira] [Comment Edited] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211660#comment-16211660 ] Íñigo Goiri edited comment on HDFS-12620 at 10/19/17 8:15 PM: -- I got the

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211724#comment-16211724 ] Hadoop QA commented on HDFS-12620: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211784#comment-16211784 ] Hadoop QA commented on HDFS-12683: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread Subru Krishnan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211736#comment-16211736 ] Subru Krishnan commented on HDFS-12620: --- [~elgoiri], thanks for working through this. Please go

[jira] [Commented] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211564#comment-16211564 ] Nanda kumar commented on HDFS-12675: Thanks for the contribution [~linyiqun]. I have committed the

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12675: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Ozone:

[jira] [Resolved] (HDFS-9808) Combine READ_ONLY_SHARED DatanodeStorages with the same ID

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs resolved HDFS-9808. -- Resolution: Won't Fix This was part of HDFS-11190. > Combine READ_ONLY_SHARED DatanodeStorages with the

[jira] [Commented] (HDFS-12663) Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211630#comment-16211630 ] Hadoop QA commented on HDFS-12663: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Attachment: HDFS-12683.02.patch > DFSZKFailOverController re-order logic for logging

[jira] [Commented] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211436#comment-16211436 ] Eric Yang commented on HDFS-12684: -- [~cheersyang] Would it make more sense to remove node manager metrics

[jira] [Commented] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211551#comment-16211551 ] Nanda kumar commented on HDFS-12675: Thanks [~linyiqun] for working on this. +1 the patch looks good

[jira] [Comment Edited] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-19 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211463#comment-16211463 ] Rushabh S Shah edited comment on HDFS-12667 at 10/19/17 6:06 PM: - bq.

[jira] [Commented] (HDFS-12482) Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211538#comment-16211538 ] Hadoop QA commented on HDFS-12482: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211639#comment-16211639 ] Nanda kumar commented on HDFS-12680: HDFS-12689 is created to track the deletion of containers in

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-19 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Parent Issue: HDFS-9806 (was: HDFS-12090) > [AliasMap] Create a version of the AliasMap that runs in

[jira] [Updated] (HDFS-12482) Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery

2017-10-19 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-12482: - Attachment: HDFS-12482.01.patch Thanks for the reviews, [~andrew.wang] Updated the patch to add

[jira] [Comment Edited] (HDFS-12482) Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery

2017-10-19 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211500#comment-16211500 ] Lei (Eddy) Xu edited comment on HDFS-12482 at 10/19/17 6:25 PM: Thanks for

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Attachment: HDFS-12683.01.patch > DFSZKFailOverController re-order logic for logging

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Status: Patch Available (was: In Progress) > DFSZKFailOverController re-order logic

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Attachment: HDFS-12683.04.patch > DFSZKFailOverController re-order logic for logging

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211484#comment-16211484 ] Xiao Chen commented on HDFS-12667: -- bq. will lock individual queue. Sounds good to me. Thanks for the

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-12683: - Description: The ZKFC should log fatal exceptions before closing the connections and terminating

[jira] [Created] (HDFS-12689) Ozone: SCM: Clean up of container in DELETING state

2017-10-19 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-12689: -- Summary: Ozone: SCM: Clean up of container in DELETING state Key: HDFS-12689 URL: https://issues.apache.org/jira/browse/HDFS-12689 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211660#comment-16211660 ] Íñigo Goiri commented on HDFS-12620: I got the following: {code} Total Elapsed time: 925m 35s -1

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620-branch-2.fixes.patch HDFS-12620-branch-2.fixes.patch contains the fixes after

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Attachment: HDFS-12683.03.patch > DFSZKFailOverController re-order logic for logging

[jira] [Commented] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211747#comment-16211747 ] Hadoop QA commented on HDFS-12683: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-19 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211463#comment-16211463 ] Rushabh S Shah commented on HDFS-12667: --- bq. Sorry for breaking this, I'll investigate about the

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-19 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211464#comment-16211464 ] Daryn Sharp commented on HDFS-12667: I was originally going to comment on another key roll occurring

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211677#comment-16211677 ] Arun Suresh commented on HDFS-12620: bq. I'm trying one last time with jenkins with version 011 but if

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211004#comment-16211004 ] Hadoop QA commented on HDFS-12620: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12619) Do not catch and throw unchecked exceptions if IBRs fail to process

2017-10-19 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-12619: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Issue Type: Sub-task (was: Improvement) Parent: HDFS-10467 > Backporting HDFS-10467 to

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Resolution: Fixed Hadoop Flags: Reviewed Target Version/s: 2.9.0, 3.0.0

[jira] [Updated] (HDFS-12684) Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12684: --- Summary: Ozone: SCMMXBean NodeCount is overlapping with NodeManagerMXBean (was: Ozone: SCM metrics

[jira] [Commented] (HDFS-12688) HDFS File Not Removed Despite Successful "Moved to .Trash" Message

2017-10-19 Thread Shriya Gupta (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212035#comment-16212035 ] Shriya Gupta commented on HDFS-12688: - I suspected that same asynchronous behaviour when this issue

[jira] [Updated] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

2017-10-19 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-12448: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved

[jira] [Commented] (HDFS-12544) SnapshotDiff - support diff generation on any snapshot root descendant directory

2017-10-19 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211876#comment-16211876 ] Manoj Govindassamy commented on HDFS-12544: --- Thanks for the review comments [~yzhangal]. Good

[jira] [Comment Edited] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212000#comment-16212000 ] Íñigo Goiri edited comment on HDFS-12620 at 10/20/17 1:11 AM: -- Thanks

[jira] [Updated] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-12682: --- Priority: Blocker (was: Major) > ECAdmin -listPolicies will always show policy state as DISABLED >

[jira] [Updated] (HDFS-12671) [READ] Test NameNode restarts when PROVIDED is configured

2017-10-19 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12671: -- Summary: [READ] Test NameNode restarts when PROVIDED is configured (was: [READ] Test

[jira] [Updated] (HDFS-12544) SnapshotDiff - support diff generation on any snapshot root descendant directory

2017-10-19 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-12544: -- Attachment: HDFS-12544.04.patch Attached v04 patch to address the following: 1. Handled

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620.000.patch > Backporting HDFS-10467 to branch-2 >

[jira] [Updated] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11468: - Resolution: Fixed Status: Resolved (was: Patch Available) > Ozone: SCM: Add Node Metrics for SCM

[jira] [Reopened] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin reopened HDFS-11468: -- > Ozone: SCM: Add Node Metrics for SCM > > > Key:

[jira] [Updated] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11468: - Status: Patch Available (was: Reopened) Re-trigger Jenkins. > Ozone: SCM: Add Node Metrics for SCM >

[jira] [Updated] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12682: - Attachment: HDFS-12682.01.patch Uploading patch 1 to show the idea. It's not easy to split the metadata

[jira] [Created] (HDFS-12687) Client has recovered DN will not be removed from the “filed”

2017-10-19 Thread xuzq (JIRA)
xuzq created HDFS-12687: --- Summary: Client has recovered DN will not be removed from the “filed” Key: HDFS-12687 URL: https://issues.apache.org/jira/browse/HDFS-12687 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-19 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211232#comment-16211232 ] James Clampffer commented on HDFS-11807: This seems to hang forever in

[jira] [Commented] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

2017-10-19 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212085#comment-16212085 ] Hudson commented on HDFS-12448: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13114 (See

[jira] [Commented] (HDFS-12482) Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212087#comment-16212087 ] Xiao Chen commented on HDFS-12482: -- Thanks [~eddyxu] for revving. Looks good in general, a few comments:

[jira] [Updated] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11468: - Attachment: HDFS-11468-HDFS-7240.004.patch Attach new patch that removing node metrics related change.

[jira] [Commented] (HDFS-12544) SnapshotDiff - support diff generation on any snapshot root descendant directory

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212118#comment-16212118 ] Hadoop QA commented on HDFS-12544: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12544) SnapshotDiff - support diff generation on any snapshot root descendant directory

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212120#comment-16212120 ] Hadoop QA commented on HDFS-12544: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212125#comment-16212125 ] Hadoop QA commented on HDFS-11468: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12619) Do not catch and throw unchecked exceptions if IBRs fail to process

2017-10-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211355#comment-16211355 ] Xiao Chen commented on HDFS-12619: -- Thanks [~jojochuang], trunk maps to 3.1.0 now. I think you should

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12680: --- Attachment: HDFS-12680-HDFS-7240.001.patch > Ozone: SCM: Lease support for container creation >

[jira] [Commented] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-19 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211373#comment-16211373 ] Hadoop QA commented on HDFS-12680: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-19 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211372#comment-16211372 ] Nanda kumar commented on HDFS-12680: Thanks [~anu] & [~linyiqun] for the review. Please find my

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620-branch-2.011.patch > Backporting HDFS-10467 to branch-2 >

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-19 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211324#comment-16211324 ] Daryn Sharp commented on HDFS-12667: The requirement to ensure new edeks is fine. The problem is the

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: (was: HDFS-12620-branch-2.fixes.patch) > Backporting HDFS-10467 to branch-2 >

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-19 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620-branch-2.012.patch HDFS-10467-branch-2.004.patch Thanks

[jira] [Commented] (HDFS-12448) Make sure user defined erasure coding policy ID will not overflow

2017-10-19 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212021#comment-16212021 ] Kai Zheng commented on HDFS-12448: -- +1 on the latest patch. > Make sure user defined erasure coding

[jira] [Commented] (HDFS-12684) Ozone: SCM metrics NodeCount is overlapping with node manager metrics

2017-10-19 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212033#comment-16212033 ] Weiwei Yang commented on HDFS-12684: Hi [~eyang] Sorry I was not giving a clear title, this is

[jira] [Updated] (HDFS-12686) Erasure coding system policy state is not correctly saved and loaded during real cluster restart

2017-10-19 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-12686: - Priority: Critical (was: Major) > Erasure coding system policy state is not correctly saved and

  1   2   >