[jira] [Commented] (HDFS-13119) RBF: Manage unavailable clusters

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356614#comment-16356614 ] Yiqun Lin commented on HDFS-13119: -- Just looked into this, {quote}When a federated cluster has one of the

[jira] [Commented] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356603#comment-16356603 ] genericqa commented on HDFS-13120: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13117) Proposal to support writing replications to HDFS asynchronously

2018-02-07 Thread xuchuanyin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356592#comment-16356592 ] xuchuanyin commented on HDFS-13117: --- [~jojochuang] [~kihwal] Thanks for your response. I haven't

[jira] [Commented] (HDFS-13046) consider load of datanodes when read blocks of file

2018-02-07 Thread hu xiaodong (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356562#comment-16356562 ] hu xiaodong commented on HDFS-13046: hello, [~ajisakaa],      what do you think? > consider load of

[jira] [Commented] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica

2018-02-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356540#comment-16356540 ] Xiao Chen commented on HDFS-11187: -- Thanks for working on the branch-2 patch Gabor! I'm not entirely

[jira] [Commented] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356543#comment-16356543 ] Jianfei Jiang commented on HDFS-12935: --  Thanks [~brahmareddy]. Thank you for your guidance. I

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356542#comment-16356542 ] Aaron T. Myers commented on HDFS-12051: --- I've taken a quick look at the patch and have some 

[jira] [Updated] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-13110: Attachment: HDFS-13110-HDFS-10285-03.patch > [SPS]: Reduce the number of APIs in NamenodeProtocol used by

[jira] [Commented] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356528#comment-16356528 ] genericqa commented on HDFS-13099: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356529#comment-16356529 ] Rakesh R commented on HDFS-13110: - Attached another patch, where I've modified the low redundancy checks

[jira] [Commented] (HDFS-13123) RBF: Add a balancer tool to move data across subsluter

2018-02-07 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356499#comment-16356499 ] Wei Yan commented on HDFS-13123: I'll put a quick doc summarizing the solution by end of this week. >

[jira] [Created] (HDFS-13123) RBF: Add a balancer tool to move data across subsluter

2018-02-07 Thread Wei Yan (JIRA)
Wei Yan created HDFS-13123: -- Summary: RBF: Add a balancer tool to move data across subsluter Key: HDFS-13123 URL: https://issues.apache.org/jira/browse/HDFS-13123 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2

2018-02-07 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356497#comment-16356497 ] Wei Yan commented on HDFS-12615: [~elgoiri] [~linyiqun] just created HDFS-13123 for the balancer. >

[jira] [Commented] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356483#comment-16356483 ] Xiaoyu Yao commented on HDFS-13120: --- Fix checkstyle issue in the test added. > Snapshot diff could be

[jira] [Updated] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13120: -- Attachment: HDFS-13120.002.patch > Snapshot diff could be corrupted after concat >

[jira] [Commented] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356466#comment-16356466 ] genericqa commented on HDFS-13120: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12985) NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted

2018-02-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356452#comment-16356452 ] Xiao Chen commented on HDFS-12985: -- bq. I cherry-picked to 3.0.1.

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356446#comment-16356446 ] Yongjun Zhang commented on HDFS-13115: -- Thanks a lot [~xiaochen].     > In

[jira] [Commented] (HDFS-12985) NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356443#comment-16356443 ] Yongjun Zhang commented on HDFS-12985: -- Thanks [~jojochuang]. I cherry-picked to 3.0.1.   >

[jira] [Updated] (HDFS-12985) NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-12985: - Fix Version/s: 3.0.1 > NameNode crashes during restart after an OpenForWrite file present in the

[jira] [Assigned] (HDFS-13119) RBF: Manage unavailable clusters

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin reassigned HDFS-13119: Assignee: Yiqun Lin > RBF: Manage unavailable clusters > > >

[jira] [Commented] (HDFS-13119) RBF: Manage unavailable clusters

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356434#comment-16356434 ] Yiqun Lin commented on HDFS-13119: -- I'd like to make this improvement, :). Assign it to myself. > RBF:

[jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356427#comment-16356427 ] Yiqun Lin commented on HDFS-12615: -- {quote} For the rebalancer itself, I have some code working in my

[jira] [Updated] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-13099: - Attachment: HDFS-13099.006.patch > RBF: Use the ZooKeeper as the default State Store >

[jira] [Commented] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356421#comment-16356421 ] Yiqun Lin commented on HDFS-13099: -- New patch attached for addressing the comment. > RBF: Use the

[jira] [Created] (HDFS-13122) FSImage should not update quota counts on ObserverNode

2018-02-07 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-13122: -- Summary: FSImage should not update quota counts on ObserverNode Key: HDFS-13122 URL: https://issues.apache.org/jira/browse/HDFS-13122 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-13122) Tailing edits should not update quota counts on ObserverNode

2018-02-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13122: --- Summary: Tailing edits should not update quota counts on ObserverNode (was: FSImage should not

[jira] [Created] (HDFS-13121) NPE when request file descriptors when SC read

2018-02-07 Thread Gang Xie (JIRA)
Gang Xie created HDFS-13121: --- Summary: NPE when request file descriptors when SC read Key: HDFS-13121 URL: https://issues.apache.org/jira/browse/HDFS-13121 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356351#comment-16356351 ] Xiaoyu Yao commented on HDFS-13120: --- Thanks [~arpitagarwal],[~jnp] and [~szetszwo] for the offline

[jira] [Commented] (HDFS-12512) RBF: Add WebHDFS

2018-02-07 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356335#comment-16356335 ] Wei Yan commented on HDFS-12512: I'll try one more later night, maybe can help avoid other parallel jobs.

[jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2

2018-02-07 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356319#comment-16356319 ] Íñigo Goiri commented on HDFS-12615: [~ywskycn] I think it makes sense to create a separate umbrella

[jira] [Updated] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13120: -- Attachment: HDFS-13120.001.patch > Snapshot diff could be corrupted after concat >

[jira] [Updated] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13120: -- Status: Patch Available (was: Open) > Snapshot diff could be corrupted after concat >

[jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2

2018-02-07 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356309#comment-16356309 ] Wei Yan commented on HDFS-12615: [~elgoiri] For the #1 rebalancer, do we just create another separate

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356298#comment-16356298 ] Xiao Chen commented on HDFS-13115: -- cherry-picked to branch-3.0. thanks all > In

[jira] [Commented] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356279#comment-16356279 ] Xiaoyu Yao commented on HDFS-13120: --- *Error stack with SnapshotDiffReport* {code}

[jira] [Created] (HDFS-13120) Snapshot diff could be corrupted after concat

2018-02-07 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-13120: - Summary: Snapshot diff could be corrupted after concat Key: HDFS-13120 URL: https://issues.apache.org/jira/browse/HDFS-13120 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356249#comment-16356249 ] genericqa commented on HDFS-12051: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-13115: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.1

[jira] [Updated] (HDFS-13119) RBF: Manage unavailable clusters

2018-02-07 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-13119: --- Summary: RBF: Manage unavailable clusters (was: RBF: manage unavailable clusters) > RBF: Manage

[jira] [Commented] (HDFS-13119) RBF: Manage unavailable clusters

2018-02-07 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-13119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356211#comment-16356211 ] Íñigo Goiri commented on HDFS-13119: We had this happening the other day when we added a subcluster

[jira] [Created] (HDFS-13119) RBF: manage unavailable clusters

2018-02-07 Thread JIRA
Íñigo Goiri created HDFS-13119: -- Summary: RBF: manage unavailable clusters Key: HDFS-13119 URL: https://issues.apache.org/jira/browse/HDFS-13119 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Comment Edited] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356167#comment-16356167 ] Íñigo Goiri edited comment on HDFS-13099 at 2/7/18 10:24 PM: -

[jira] [Commented] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356167#comment-16356167 ] Íñigo Goiri commented on HDFS-13099: [^HDFS-13099.005.patch] seems to run the unit tests properly. I

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356159#comment-16356159 ] Hudson commented on HDFS-13115: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13632 (See

[jira] [Commented] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356160#comment-16356160 ] genericqa commented on HDFS-13116: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: Patch Available (was: In Progress) Addressed the latest commment by [~yzhangal]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Attachment: HDFS-12051.11.patch > Reimplement NameCache in NameNode: Intern duplicate byte[]

[jira] [Updated] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12051: -- Status: In Progress (was: Patch Available) > Reimplement NameCache in NameNode: Intern

[jira] [Comment Edited] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356082#comment-16356082 ] Anu Engineer edited comment on HDFS-13116 at 2/7/18 9:04 PM: - [~msingh] Thanks

[jira] [Commented] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356082#comment-16356082 ] Anu Engineer commented on HDFS-13116: - [~msingh] Thanks for working on this patch. It looks very good.

[jira] [Issue Comment Deleted] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-13115: - Comment: was deleted (was: Hi [~szetszwo], Sorry I posted my previous comment in wrong Jira

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356036#comment-16356036 ] Yongjun Zhang commented on HDFS-13115: -- Hi [~szetszwo], Sorry I posted my previous comment in wrong

[jira] [Issue Comment Deleted] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-13115: --- Comment: was deleted (was: And this seems not the correct JIRA?) > In

[jira] [Issue Comment Deleted] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-13115: --- Comment: was deleted (was: > ... and Tsz Wo Nicholas Sze for the review. I like to

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356031#comment-16356031 ] Tsz Wo Nicholas Sze commented on HDFS-12051: > ... and Tsz Wo Nicholas Sze for the review. I

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356028#comment-16356028 ] Tsz Wo Nicholas Sze commented on HDFS-13115: And this seems not the correct JIRA? > In

[jira] [Commented] (HDFS-12051) Reimplement NameCache in NameNode: Intern duplicate byte[] arrays (mainly those denoting file/directory names) to save memory

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356026#comment-16356026 ] Yongjun Zhang commented on HDFS-12051: -- Thanks [~mi...@cloudera.com] for the new revs and [~szetszwo] 

[jira] [Issue Comment Deleted] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-13115: - Comment: was deleted (was: Thanks [~mi...@cloudera.com] for the new revs and [~szetszwo] for the

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356025#comment-16356025 ] Tsz Wo Nicholas Sze commented on HDFS-13115: > ... and Tsz Wo Nicholas Sze for the review. I

[jira] [Commented] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356019#comment-16356019 ] genericqa commented on HDFS-12935: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13115) In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted

2018-02-07 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356011#comment-16356011 ] Yongjun Zhang commented on HDFS-13115: -- Thanks [~mi...@cloudera.com] for the new revs and [~szetszwo]

[jira] [Commented] (HDFS-13078) Ozone: Ratis read fail because stream is closed before the reply is received

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355968#comment-16355968 ] genericqa commented on HDFS-13078: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11701) NPE from Unresolved Host causes permanent DFSInputStream failures

2018-02-07 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355965#comment-16355965 ] Hudson commented on HDFS-11701: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13631 (See

[jira] [Commented] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355959#comment-16355959 ] genericqa commented on HDFS-13110: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11701) NPE from Unresolved Host causes permanent DFSInputStream failures

2018-02-07 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDFS-11701: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was:

[jira] [Commented] (HDFS-12990) Change default NameNode RPC port back to 8020

2018-02-07 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355941#comment-16355941 ] Anu Engineer commented on HDFS-12990: - Committed to branch-3.0 also. > Change default NameNode RPC

[jira] [Commented] (HDFS-11187) Optimize disk access for last partial chunk checksum of Finalized replica

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355937#comment-16355937 ] Brahma Reddy Battula commented on HDFS-11187: - Need to be in branch-3.0.1 also..? > Optimize

[jira] [Commented] (HDFS-13081) Datanode#checkSecureConfig should check HTTPS and SASL encryption

2018-02-07 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355938#comment-16355938 ] Jitendra Nath Pandey commented on HDFS-13081: - {quote}Delegation tokens send passwords in the

[jira] [Commented] (HDFS-12933) Improve logging when DFSStripedOutputStream failed to write some blocks

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355932#comment-16355932 ] Brahma Reddy Battula commented on HDFS-12933: - Need to be in branch-3.0.1 also..? > Improve

[jira] [Commented] (HDFS-11701) NPE from Unresolved Host causes permanent DFSInputStream failures

2018-02-07 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355918#comment-16355918 ] Jitendra Nath Pandey commented on HDFS-11701: - +1, I will commit shortly. > NPE from

[jira] [Commented] (HDFS-12990) Change default NameNode RPC port back to 8020

2018-02-07 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355903#comment-16355903 ] Anu Engineer commented on HDFS-12990: - [~brahmareddy] I will commit this to 3.0 now. Thanks for

[jira] [Commented] (HDFS-12990) Change default NameNode RPC port back to 8020

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355897#comment-16355897 ] Brahma Reddy Battula commented on HDFS-12990: - should be committed to branch-3.0 which is

[jira] [Updated] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-13116: - Attachment: HDFS-13116-HDFS-7240.006.patch > Ozone: Refactor Pipeline to have transport

[jira] [Commented] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355848#comment-16355848 ] Hudson commented on HDFS-12935: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13629 (See

[jira] [Updated] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-12935: Fix Version/s: 3.0.2 3.0.1 3.1.0 > Get

[jira] [Commented] (HDFS-13112) Token expiration edits may cause log corruption or deadlock

2018-02-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355844#comment-16355844 ] Kihwal Lee commented on HDFS-13112: --- The patch looks good. - The addition of read locks ensures these

[jira] [Comment Edited] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355839#comment-16355839 ] Brahma Reddy Battula edited comment on HDFS-12935 at 2/7/18 6:13 PM: -

[jira] [Commented] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355839#comment-16355839 ] Brahma Reddy Battula commented on HDFS-12935: - {{Committed to {{trunk}} and branch-3.0

[jira] [Commented] (HDFS-11701) NPE from Unresolved Host causes permanent DFSInputStream failures

2018-02-07 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355838#comment-16355838 ] Lokesh Jain commented on HDFS-11701: The unit test failures are not related. > NPE from Unresolved

[jira] [Updated] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-12935: Attachment: HDFS-12935.009-branch-2.patch > Get ambiguous result for DFSAdmin

[jira] [Commented] (HDFS-12935) Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up

2018-02-07 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355827#comment-16355827 ] Brahma Reddy Battula commented on HDFS-12935: - latest trunk patch lgtm, Committing shortly.

[jira] [Updated] (HDFS-13078) Ozone: Ratis read fail because stream is closed before the reply is received

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-13078: - Status: Patch Available (was: Open) > Ozone: Ratis read fail because stream is closed

[jira] [Updated] (HDFS-13078) Ozone: Ratis read fail because stream is closed before the reply is received

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-13078: - Attachment: HDFS-13078-HDFS-7240.001.patch > Ozone: Ratis read fail because stream is

[jira] [Commented] (HDFS-13078) Ozone: Ratis read fail because stream is closed before the reply is received

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355814#comment-16355814 ] Mukul Kumar Singh commented on HDFS-13078: -- Added some debug logging and determined that the

[jira] [Commented] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355759#comment-16355759 ] genericqa commented on HDFS-13116: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355747#comment-16355747 ] genericqa commented on HDFS-13099: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355685#comment-16355685 ] Rakesh R commented on HDFS-13110: - Attached new patch fixing test case failures and checkstyle warnings,

[jira] [Updated] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-13110: Attachment: HDFS-13110-HDFS-10285-02.patch > [SPS]: Reduce the number of APIs in NamenodeProtocol used by

[jira] [Updated] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-13110: Attachment: (was: HDFS-13110-HDFS-02.patch) > [SPS]: Reduce the number of APIs in NamenodeProtocol used

[jira] [Updated] (HDFS-13110) [SPS]: Reduce the number of APIs in NamenodeProtocol used by external satisfier

2018-02-07 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-13110: Attachment: HDFS-13110-HDFS-02.patch > [SPS]: Reduce the number of APIs in NamenodeProtocol used by

[jira] [Commented] (HDFS-13116) Ozone: Refactor Pipeline to have transport and container specific information

2018-02-07 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355659#comment-16355659 ] genericqa commented on HDFS-13116: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-10453) ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.

2018-02-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355650#comment-16355650 ] Erik Krogen edited comment on HDFS-10453 at 2/7/18 4:16 PM: Re: v008 patch,

[jira] [Commented] (HDFS-10453) ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.

2018-02-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355650#comment-16355650 ] Erik Krogen commented on HDFS-10453: Re: v008 patch, looks like you are using

[jira] [Updated] (HDFS-13104) Make hadoop proxy user changes reconfigurable in Namenode

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-13104: - Resolution: Not A Problem Status: Resolved (was: Patch Available) As pointed by

[jira] [Resolved] (HDFS-13105) Make hadoop proxy user changes reconfigurable in Datanode

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh resolved HDFS-13105. -- Resolution: Not A Problem As pointed by [~kihwal] & [~rajive],

[jira] [Commented] (HDFS-13117) Proposal to support writing replications to HDFS asynchronously

2018-02-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355520#comment-16355520 ] Kihwal Lee commented on HDFS-13117: --- During writes, the data is written to multiple nodes concurrently.

[jira] [Commented] (HDFS-13022) Block Storage: Kubernetes dynamic persistent volume provisioner

2018-02-07 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355486#comment-16355486 ] Mukul Kumar Singh commented on HDFS-13022: -- Thanks for the updated patch [~elek], The new patch

[jira] [Updated] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-13099: - Attachment: HDFS-13099.005.patch > RBF: Use the ZooKeeper as the default State Store >

[jira] [Commented] (HDFS-13099) RBF: Use the ZooKeeper as the default State Store

2018-02-07 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355476#comment-16355476 ] Yiqun Lin commented on HDFS-13099: -- Attach the patch to fix failed unit test. > RBF: Use the ZooKeeper

  1   2   >