[jira] [Commented] (HDFS-6583) Remove clientNode in FileUnderConstructionFeature

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039429#comment-14039429 ] Tsz Wo Nicholas Sze commented on HDFS-6583: --- This is a good idea. Please as well

[jira] [Updated] (HDFS-6583) Remove clientNode in FileUnderConstructionFeature

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6583: -- Component/s: namenode Priority: Minor (was: Major) Assignee: Haohui Mai

[jira] [Updated] (HDFS-4667) Capture renamed files/directories in snapshot diff report

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-4667: -- Component/s: (was: datanode) Assignee: Jing Zhao (was: Binglin Chang)

[jira] [Created] (HDFS-6584) Support archival storage

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6584: - Summary: Support archival storage Key: HDFS-6584 URL: https://issues.apache.org/jira/browse/HDFS-6584 Project: Hadoop HDFS Issue Type: New Feature

[jira] [Updated] (HDFS-5682) Heterogeneous Storage phase 2 - APIs to expose Storage Types

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-5682: -- Component/s: namenode datanode Issue Type: New Feature (was: Bug)

[jira] [Updated] (HDFS-6583) Remove clientNode in FileUnderConstructionFeature

2014-06-20 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6583: -- Hadoop Flags: Reviewed +1 the new patch looks good. Remove clientNode in

[jira] [Commented] (HDFS-6562) Refactor rename() in FSDirectory

2014-06-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041422#comment-14041422 ] Tsz Wo Nicholas Sze commented on HDFS-6562: --- Patch looks good. Some comments: -

[jira] [Updated] (HDFS-6584) Support archival storage

2014-06-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Attachment: HDFSArchivalStorageDesign20140623.pdf

[jira] [Commented] (HDFS-6562) Refactor rename() in FSDirectory

2014-06-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041507#comment-14041507 ] Tsz Wo Nicholas Sze commented on HDFS-6562: --- Thanks for the explanation.

[jira] [Updated] (HDFS-6562) Refactor rename() in FSDirectory

2014-06-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6562: -- Priority: Minor (was: Major) Hadoop Flags: Reviewed +1 the new patch looks

[jira] [Commented] (HDFS-3528) Use native CRC32 in DFS write path

2014-06-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041545#comment-14041545 ] Tsz Wo Nicholas Sze commented on HDFS-3528: --- Is the native library faster than

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-06-24 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042491#comment-14042491 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- By the Java implementation, do you

[jira] [Commented] (HDFS-6595) Configure the maximum threads allowed for balancing on datanodes

2014-06-24 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042570#comment-14042570 ] Tsz Wo Nicholas Sze commented on HDFS-6595: --- Benoy, thanks for working on this.

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-06-24 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042674#comment-14042674 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- We should be able to improve current

[jira] [Updated] (HDFS-6595) Configure the maximum threads allowed for balancing on datanodes

2014-06-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6595: -- Component/s: balancer Priority: Minor (was: Major) Hadoop Flags: Reviewed

[jira] [Updated] (HDFS-6595) Configure the maximum threads allowed for balancing on datanodes

2014-06-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6595: -- Resolution: Fixed Fix Version/s: 2.5.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-06-25 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044060#comment-14044060 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- For Java 7, NativeCrc32 with direct

[jira] [Updated] (HDFS-4685) Implementation of ACLs in HDFS

2014-06-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-4685: -- Assignee: Chris Nauroth (was: Andrew Wang) Implementation of ACLs in HDFS

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046571#comment-14046571 ] Tsz Wo Nicholas Sze commented on HDFS-6134: --- Correct me if I am wrong: The

[jira] [Updated] (HDFS-6418) Regression: DFS_NAMENODE_USER_NAME_KEY missing in trunk

2014-06-29 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6418: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-06-30 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048380#comment-14048380 ] Tsz Wo Nicholas Sze commented on HDFS-6134: --- ..., at rest. Someone who has root

[jira] [Resolved] (HDFS-1379) Multihoming brokenness in HDFS

2014-06-30 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-1379. --- Resolution: Not a Problem I believe this is not a problem anymore after other JIRAs

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-07-01 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049310#comment-14049310 ] Tsz Wo Nicholas Sze commented on HDFS-6134: --- [~andrew.purt...@gmail.com], thanks

[jira] [Updated] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-01 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6616: -- Component/s: webhdfs bestNode shouldn't always return the first DataNode

[jira] [Commented] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-01 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049420#comment-14049420 ] Tsz Wo Nicholas Sze commented on HDFS-6616: --- I think it is not a good idea to do

[jira] [Commented] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050618#comment-14050618 ] Tsz Wo Nicholas Sze commented on HDFS-6616: --- Correct me if I'm wrong: when using

[jira] [Commented] (HDFS-3528) Use native CRC32 in DFS write path

2014-07-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050901#comment-14050901 ] Tsz Wo Nicholas Sze commented on HDFS-3528: --- It turns out that

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-07-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14052104#comment-14052104 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- Sure. I just want to point out that

[jira] [Created] (HDFS-6633) Support reading new data in a being written file untill the file is closed

2014-07-07 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6633: - Summary: Support reading new data in a being written file untill the file is closed Key: HDFS-6633 URL: https://issues.apache.org/jira/browse/HDFS-6633

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Summary: Support reading new data in a being written file until the file is closed

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Attachment: h6633_20140707.patch h6633_20140707.patch: - The patch adds a new conf

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-07 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Status: Patch Available (was: Open) Support reading new data in a being written file

[jira] [Commented] (HDFS-6469) Coordinated replication of the namespace using ConsensusNode

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055002#comment-14055002 ] Tsz Wo Nicholas Sze commented on HDFS-6469: --- hflush() does not necessarily

[jira] [Created] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6643: - Summary: Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat Key: HDFS-6643 URL: https://issues.apache.org/jira/browse/HDFS-6643

[jira] [Updated] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6643: -- Attachment: h6643_20140708.patch h6643_20140708.patch: - adds LongBitFormat; - changes

[jira] [Updated] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6643: -- Status: Patch Available (was: Open) Refactor INodeFile.HeaderFormat and

[jira] [Commented] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055630#comment-14055630 ] Tsz Wo Nicholas Sze commented on HDFS-6633: --- Sounds good. The feature indeed is

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Attachment: h6643_20140708.patch h6643_20140708.patch: renames the new conf property.

[jira] [Updated] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6643: -- Attachment: h6643_20140708b.patch h6643_20140708b.patch: - Fix TestINodeFile -- it uses

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Attachment: (was: h6643_20140708.patch) Support reading new data in a being

[jira] [Updated] (HDFS-6633) Support reading new data in a being written file until the file is closed

2014-07-08 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6633: -- Attachment: h6633_20140708.patch Oops, uploaded a wrong file. Here you go.

[jira] [Resolved] (HDFS-172) Quota exceed exception creates file of size 0

2014-07-09 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-172. -- Resolution: Not a Problem Resolve this as not-a-problem. Please feel free to reopen if

[jira] [Updated] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-09 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6643: -- Attachment: h6643_20140709.patch Thanks Jing. It is a good point that the fields

[jira] [Updated] (HDFS-6643) Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat

2014-07-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6643: -- Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed

[jira] [Commented] (HDFS-6651) Deletion failure can leak inodes permanently.

2014-07-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058109#comment-14058109 ] Tsz Wo Nicholas Sze commented on HDFS-6651: --- Let's number the solutions: #

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-07-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058232#comment-14058232 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- [~james.thomas], I suspect our

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059035#comment-14059035 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- For CRC32C, I have no problem to use

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059392#comment-14059392 ] Tsz Wo Nicholas Sze commented on HDFS-6560: --- Hi Todd, for bytesPerChecksum ==

[jira] [Created] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6670: - Summary: Archival Storage: Add block storage policy Key: HDFS-6670 URL: https://issues.apache.org/jira/browse/HDFS-6670 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6671: - Summary: Archival Storage: Consider block storage policy in replicaiton Key: HDFS-6671 URL: https://issues.apache.org/jira/browse/HDFS-6671 Project: Hadoop

[jira] [Commented] (HDFS-6584) Support archival storage

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059448#comment-14059448 ] Tsz Wo Nicholas Sze commented on HDFS-6584: --- I just have created a branch

[jira] [Updated] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6670: -- Description: Support block storage policy such as the table below. ||ID||Policy Name ||

[jira] [Updated] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-11 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6670: -- Attachment: h6670_20140711.patch h6670_20140711.patch: 1st patch. Archival Storage:

[jira] [Updated] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6670: -- Description: Support block storage policy such as the table below. ||ID||Policy Name ||

[jira] [Created] (HDFS-6677) Archival Storage: Change INodeFile and FSImage to support storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6677: - Summary: Archival Storage: Change INodeFile and FSImage to support storage policy Key: HDFS-6677 URL: https://issues.apache.org/jira/browse/HDFS-6677

[jira] [Updated] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6670: -- Attachment: h6670_20140714.patch h6670_20140714.patch: reverts INodeFile. Archival

[jira] [Updated] (HDFS-6677) Archival Storage: Change INodeFile and FSImage to support storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6677: -- Attachment: h6677_20140714.patch h6677_20140714.patch: 1st patch. Archival Storage:

[jira] [Commented] (HDFS-6584) Support archival storage

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060924#comment-14060924 ] Tsz Wo Nicholas Sze commented on HDFS-6584: --- Hi Nathan, we already support

[jira] [Resolved] (HDFS-6677) Archival Storage: Change INodeFile and FSImage to support storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-6677. --- Resolution: Fixed Fix Version/s: Archival Storage (HDFS-6584) Hadoop

[jira] [Resolved] (HDFS-6670) Archival Storage: Add block storage policy

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-6670. --- Resolution: Fixed Fix Version/s: Archival Storage (HDFS-6584) Hadoop

[jira] [Updated] (HDFS-6679) Archival Storage: Bump NameNodeLayoutVersion and update editsStored test files

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6679: -- Component/s: (was: datanode) Archival Storage: Bump NameNodeLayoutVersion and

[jira] [Created] (HDFS-6679) Archival Storage: Bump NameNodeLayoutVersion and update editsStored test files

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6679: - Summary: Archival Storage: Bump NameNodeLayoutVersion and update editsStored test files Key: HDFS-6679 URL: https://issues.apache.org/jira/browse/HDFS-6679

[jira] [Commented] (HDFS-6584) Support archival storage

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061390#comment-14061390 ] Tsz Wo Nicholas Sze commented on HDFS-6584: --- Good question. Here is more

[jira] [Created] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6680: - Summary: BlockPlacementPolicyDefault does not choose favored nodes correctly Key: HDFS-6680 URL: https://issues.apache.org/jira/browse/HDFS-6680 Project:

[jira] [Updated] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6680: -- Component/s: namenode BlockPlacementPolicyDefault does not choose favored nodes

[jira] [Commented] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061504#comment-14061504 ] Tsz Wo Nicholas Sze commented on HDFS-6680: --- [~devaraj]/[~blackpearl], could you

[jira] [Assigned] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reassigned HDFS-6680: - Assignee: Tsz Wo Nicholas Sze BlockPlacementPolicyDefault does not choose

[jira] [Updated] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6680: -- Attachment: h6680_20140714.patch h6680_20140714.patch: passing boolean

[jira] [Updated] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6680: -- Status: Patch Available (was: Open) BlockPlacementPolicyDefault does not choose

[jira] [Updated] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6671: -- Attachment: h6671_20140714.patch h6671_20140714.patch: use storage policy to choose

[jira] [Commented] (HDFS-6010) Make balancer able to balance data among specified servers

2014-07-14 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061556#comment-14061556 ] Tsz Wo Nicholas Sze commented on HDFS-6010: --- [~carp84], have you looked at

[jira] [Updated] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6671: -- Attachment: h6671_20140715.patch Thanks Vinay for reveiwing the patch. # Good catch.

[jira] [Updated] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6671: -- Attachment: h6671_20140715b.patch h6671_20140715b.patch: adds more checks on parsing

[jira] [Updated] (HDFS-6584) Support archival storage

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Attachment: HDFSArchivalStorageDesign20140715.pdf

[jira] [Updated] (HDFS-6584) Support Archival Storage

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6584: -- Component/s: (was: datanode) balancer Summary: Support

[jira] [Assigned] (HDFS-6679) Archival Storage: Bump NameNodeLayoutVersion and update editsStored test files

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reassigned HDFS-6679: - Assignee: (was: Tsz Wo Nicholas Sze) Archival Storage: Bump

[jira] [Created] (HDFS-6685) Archival Storage: Update Balancer to preserve storage type of replicas

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6685: - Summary: Archival Storage: Update Balancer to preserve storage type of replicas Key: HDFS-6685 URL: https://issues.apache.org/jira/browse/HDFS-6685

[jira] [Created] (HDFS-6686) Archival Storage: Use fallback storage types

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6686: - Summary: Archival Storage: Use fallback storage types Key: HDFS-6686 URL: https://issues.apache.org/jira/browse/HDFS-6686 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062482#comment-14062482 ] Tsz Wo Nicholas Sze commented on HDFS-6616: --- @ zhaoyunjiong, are you going to

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062593#comment-14062593 ] Tsz Wo Nicholas Sze commented on HDFS-6658: --- Nice idea! For processing block

[jira] [Reopened] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reopened HDFS-5464: --- Simplify block report diff calculation --

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062648#comment-14062648 ] Tsz Wo Nicholas Sze commented on HDFS-6658: --- Let me try it in HDFS-5464.

[jira] [Updated] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-5464: -- Attachment: h5464_20140715.patch Some new idea inspired by HDFS-6658.

[jira] [Updated] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-5464: -- Status: Patch Available (was: Reopened) Simplify block report diff calculation

[jira] [Assigned] (HDFS-5284) Flatten INode hierarchy

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reassigned HDFS-5284: - Assignee: Jing Zhao (was: Tsz Wo Nicholas Sze) Flatten INode hierarchy

[jira] [Commented] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062720#comment-14062720 ] Tsz Wo Nicholas Sze commented on HDFS-5464: --- I think looping 500,000 is nothing

[jira] [Updated] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-5464: -- Attachment: h5464_20140715b.patch h5464_20140715b.patch: reverted other tests in

[jira] [Commented] (HDFS-5464) Simplify block report diff calculation

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062809#comment-14062809 ] Tsz Wo Nicholas Sze commented on HDFS-5464: --- [~andrew.wang], could you help doing

[jira] [Updated] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6671: -- Attachment: h6671_20140715c.patch h6671_20140715c.patch: renames getStorageTypes to

[jira] [Commented] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-15 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062937#comment-14062937 ] Tsz Wo Nicholas Sze commented on HDFS-6680: --- I discovered this bug when changing

[jira] [Updated] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-16 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6616: -- Status: Patch Available (was: Open) bestNode shouldn't always return the first

[jira] [Updated] (HDFS-6680) BlockPlacementPolicyDefault does not choose favored nodes correctly

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6680: -- Attachment: h6680_20140716.patch h6680_20140716.patch: fixes another bug in the loop

[jira] [Updated] (HDFS-6700) BlockPlacementPolicy shoud choose storage but not datanode for deletion

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6700: -- Attachment: h6700_20140717.patch h6700_20140717.patch: changes to choose storages.

[jira] [Updated] (HDFS-6700) BlockPlacementPolicy shoud choose storage but not datanode for deletion

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6700: -- Status: Patch Available (was: Open) BlockPlacementPolicy shoud choose storage but not

[jira] [Created] (HDFS-6700) BlockPlacementPolicy shoud choose storage but not datanode for deletion

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-6700: - Summary: BlockPlacementPolicy shoud choose storage but not datanode for deletion Key: HDFS-6700 URL: https://issues.apache.org/jira/browse/HDFS-6700

[jira] [Updated] (HDFS-6700) BlockPlacementPolicy shoud choose storage but not datanode for deletion

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6700: -- Component/s: namenode BlockPlacementPolicy shoud choose storage but not datanode for

[jira] [Updated] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6671: -- Attachment: h6671_20140717.patch h6671_20140717.patch: renames the chooseTarget methods

[jira] [Resolved] (HDFS-6671) Archival Storage: Consider block storage policy in replicaiton

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-6671. --- Resolution: Fixed Fix Version/s: Archival Storage (HDFS-6584) Hadoop

[jira] [Commented] (HDFS-6616) bestNode shouldn't always return the first DataNode

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065368#comment-14065368 ] Tsz Wo Nicholas Sze commented on HDFS-6616: --- Patch looks good. Some comments: -

[jira] [Updated] (HDFS-6700) BlockPlacementPolicy shoud choose storage but not datanode for deletion

2014-07-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6700: -- Attachment: h6700_20140717b.patch h6700_20140717b.patch: fixes TestReplicationPolicy

  1   2   3   4   5   6   7   8   9   10   >