[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553735#comment-14553735 ] Walter Su commented on HDFS-7621: - 006 patch changes: 1. rebase; 2. update {{blockManager}}

[jira] [Updated] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-7621: Attachment: HDFS-7621.006.patch Erasure Coding: update the Balancer/Mover data migration logic

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555454#comment-14555454 ] Walter Su commented on HDFS-8433: - Hi, [~szetszwo]! I'm wiling to work on this if you are

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Resolution: Won't Fix Status: Resolved (was: Patch Available) Erasure Coding: test skip in

[jira] [Commented] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555476#comment-14555476 ] Walter Su commented on HDFS-8441: - Thanks Kai. I commited this. Erasure Coding: make

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Resolution: Fixed Fix Version/s: HDFS-7285 Status: Resolved (was: Patch Available)

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: HDFS-8186-HDFS-7285.002.txt HDFS-8186.001.txt history patch. Erasure coding:

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Priority: Trivial (was: Major) Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Commented] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1472#comment-1472 ] Walter Su commented on HDFS-8342: - {{TestWriteReadStripedFile.verifySeek()}} has similar

[jira] [Commented] (HDFS-8220) Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize

2015-05-21 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555623#comment-14555623 ] Walter Su commented on HDFS-8220: - {code} public DatanodeStorageInfo[]

[jira] [Updated] (HDFS-8444) Erasure Coding: fix cannot rename a zone dir

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8444: Status: Patch Available (was: Open) Erasure Coding: fix cannot rename a zone dir

[jira] [Updated] (HDFS-8444) Erasure Coding: fix cannot rename a zone dir

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8444: Attachment: HDFS-8444-HDFS-7285.001.patch Erasure Coding: fix cannot rename a zone dir

[jira] [Created] (HDFS-8444) Erasure Coding: fix cannot rename a zone dir

2015-05-20 Thread Walter Su (JIRA)
Walter Su created HDFS-8444: --- Summary: Erasure Coding: fix cannot rename a zone dir Key: HDFS-8444 URL: https://issues.apache.org/jira/browse/HDFS-8444 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Description: Changes: 1. {{UnsupportedActionException}} is more user-firendly. 2. check condition before

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Description: Changes: 1. {{UnsupportedActionException}} is more user-firendly. 2. check condition before

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Attachment: HDFS-8441-HDFS-7285.002.patch Erasure Coding: make condition check earlier for setReplication

[jira] [Commented] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552069#comment-14552069 ] Walter Su commented on HDFS-8441: - 002 patch add support create EC file with replication=0

[jira] [Commented] (HDFS-8444) Erasure Coding: fix cannot rename a zone dir

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553474#comment-14553474 ] Walter Su commented on HDFS-8444: - jenkins not related. Please review. Erasure Coding:

[jira] [Commented] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553511#comment-14553511 ] Walter Su commented on HDFS-8441: - ... Maybe we could add a method in

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-20 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Attachment: HDFS-8441-HDFS-7285.003.patch Erasure Coding: make condition check earlier for setReplication

[jira] [Commented] (HDFS-8333) Create EC zone should not need superuser privilege

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550012#comment-14550012 ] Walter Su commented on HDFS-8333: - Patch looks good. I'm +1 for this idea. Hi, [~drankye],

[jira] [Commented] (HDFS-8378) Erasure Coding: Few improvements for the erasure coding worker

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549938#comment-14549938 ] Walter Su commented on HDFS-8378: - Thanks [~rakeshr] for the contribution! It's committed

[jira] [Updated] (HDFS-8378) Erasure Coding: Few improvements for the erasure coding worker

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8378: Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Status: Resolved

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Attachment: HDFS-8441-HDFS-7285.001.patch Erasure Coding: make condition check earlier for setReplication

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Attachment: (was: HDFS-8441-HDFS-7285.001.patch) Erasure Coding: make condition check earlier for

[jira] [Commented] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551617#comment-14551617 ] Walter Su commented on HDFS-8186: - Thanks Zhe Zhang. Jenkins didn't come out. I am trying

[jira] [Created] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-19 Thread Walter Su (JIRA)
Walter Su created HDFS-8441: --- Summary: Erasure Coding: make condition check earlier for setReplication Key: HDFS-8441 URL: https://issues.apache.org/jira/browse/HDFS-8441 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Attachment: HDFS-8441-HDFS-7285.001.patch Erasure Coding: make condition check earlier for setReplication

[jira] [Updated] (HDFS-8441) Erasure Coding: make condition check earlier for setReplication

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8441: Status: Patch Available (was: Open) Erasure Coding: make condition check earlier for setReplication

[jira] [Created] (HDFS-8438) Erasure Coding: support concat files in same EC zone

2015-05-19 Thread Walter Su (JIRA)
Walter Su created HDFS-8438: --- Summary: Erasure Coding: support concat files in same EC zone Key: HDFS-8438 URL: https://issues.apache.org/jira/browse/HDFS-8438 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: (was: HDFS-8186-HDFS-7285.003.patch) Erasure coding: Make block placement policy for EC

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Status: Open (was: Patch Available) Erasure coding: Make block placement policy for EC file configurable

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: HDFS-8186-HDFS-7285.003.patch Erasure coding: Make block placement policy for EC file

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Status: Patch Available (was: Open) Erasure coding: Make block placement policy for EC file configurable

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: (was: HDFS-8186.001.patch) Erasure coding: Make block placement policy for EC file

[jira] [Commented] (HDFS-8131) Implement a space balanced block placement policy

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14550433#comment-14550433 ] Walter Su commented on HDFS-8131: - Hi, [~kihwal]! Could you take a look at HDFS-8390? It's

[jira] [Created] (HDFS-8430) Erasure coding: update DFSClient.getFileChecksum() logic for stripe files

2015-05-19 Thread Walter Su (JIRA)
Walter Su created HDFS-8430: --- Summary: Erasure coding: update DFSClient.getFileChecksum() logic for stripe files Key: HDFS-8430 URL: https://issues.apache.org/jira/browse/HDFS-8430 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-19 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Labels: BlockPlacementPolicy (was: ) merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8417: Status: Patch Available (was: Open) Erasure Coding: Pread failed to read data starting from not-first

[jira] [Commented] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547998#comment-14547998 ] Walter Su commented on HDFS-8417: - {{TestDFSStripedInputStream}} doesn't depend on

[jira] [Updated] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8417: Attachment: HDFS-8417-HDFS-7285.002.patch Erasure Coding: Pread failed to read data starting from not-first

[jira] [Updated] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8417: Attachment: HDFS-8417-HDFS-7285.001.patch Erasure Coding: Pread failed to read data starting from not-first

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: HDFS-8352-HDFS-7285.001.patch Erasure Coding: test webhdfs read write stripe file

[jira] [Commented] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547688#comment-14547688 ] Walter Su commented on HDFS-8417: - tests included in HDFS-8352. 8352 tests fails for now.

[jira] [Updated] (HDFS-8417) Erasure Coding: Pread failed to read data starting from not-first stripe

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8417: Summary: Erasure Coding: Pread failed to read data starting from not-first stripe (was: Erasure Coding:

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: (was: HDFS-8352-HDFS-7285.001.patch) Erasure Coding: test webhdfs read write stripe file

[jira] [Commented] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549644#comment-14549644 ] Walter Su commented on HDFS-8186: - ...Looks like we should add a TODO that webhdfs on

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: HDFS-8186-HDFS-7285.003.patch Erasure coding: Make block placement policy for EC file

[jira] [Commented] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14549666#comment-14549666 ] Walter Su commented on HDFS-8352: - I'm very sorry. I'll keep it in mind. Erasure Coding:

[jira] [Resolved] (HDFS-8339) Erasure Coding: Badly treated when createBlockOutputStream failed in DataStreamer

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su resolved HDFS-8339. - Resolution: Duplicate Erasure Coding: Badly treated when createBlockOutputStream failed in

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-18 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Resolution: Fixed Target Version/s: HDFS-7285 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-8199) Erasure Coding: System test of creating ECZone and EC files.

2015-05-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14547560#comment-14547560 ] Walter Su commented on HDFS-8199: - Sorry. Mistake. normal files: 13m11s ec files: 45m33s

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: (was: HDFS-8352-HDFS-7285.001.patch) Erasure Coding: test webhdfs read write stripe file

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: HDFS-8352-HDFS-7285.001.patch Erasure Coding: test webhdfs read write stripe file

[jira] [Created] (HDFS-8417) Erasure Coding: Pread failed to read data starting from last incomplete stripe

2015-05-17 Thread Walter Su (JIRA)
Walter Su created HDFS-8417: --- Summary: Erasure Coding: Pread failed to read data starting from last incomplete stripe Key: HDFS-8417 URL: https://issues.apache.org/jira/browse/HDFS-8417 Project: Hadoop

[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545421#comment-14545421 ] Walter Su commented on HDFS-7621: - 005 patch totally rewrite, except tests. Please review.

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: HDFS-8390.004.patch merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Attachment: HDFS-8342-HDFS-7285.001.patch Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: (was: HDFS-8352-HDFS-7285.001.repost.patch) Erasure Coding: test webhdfs read write stripe

[jira] [Updated] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-7621: Attachment: HDFS-7621.005.patch Erasure Coding: update the Balancer/Mover data migration logic

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: HDFS-8352-HDFS-7285.001.patch Erasure Coding: test webhdfs read write stripe file

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: (was: HDFS-8352-HDFS-7285.001.patch) Erasure Coding: test webhdfs read write stripe file

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Attachment: (was: HDFS-8342-HDFS-7285.001.repost.patch) Erasure Coding: test skip in

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-15 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Attachment: (was: HDFS-8342-HDFS-7285.001.patch) Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Description: We saw requirements for adding new policies.( HDFS-7613, HDFS-7892, HDFS-8131, maybe

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Attachment: HDFS-8342-HDFS-7285.001.repost.patch Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Attachment: HDFS-8352-HDFS-7285.001.repost.patch Erasure Coding: test webhdfs read write stripe file

[jira] [Updated] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-7621: Attachment: HDFS-7621.004.patch Erasure Coding: update the Balancer/Mover data migration logic

[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543305#comment-14543305 ] Walter Su commented on HDFS-7621: - ...moveBlockAcrossStorage requires Mover send correct

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: HDFS-8390.003.patch merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Commented] (HDFS-8199) Erasure Coding: System test of creating ECZone and EC files.

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543488#comment-14543488 ] Walter Su commented on HDFS-8199: - Nodes: 1NN + 9DN + 1 Client Client node writes 100G (1G

[jira] [Commented] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543430#comment-14543430 ] Walter Su commented on HDFS-8186: - 1. ...Balancer and NamenodeFsck. Should we address these

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Status: Patch Available (was: Open) Erasure coding: Make block placement policy for EC file configurable

[jira] [Updated] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

2015-05-14 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8186: Attachment: HDFS-8186-HDFS-7285.002.patch Erasure coding: Make block placement policy for EC file

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: HDFS-8390.001.patch merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Description: We saw requirements for adding new policies.( HDFS-7613, HDFS-7892, maybe HDFS-4894,

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: (was: HDFS-8390.001.patch) merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: HDFS-8390.001.patch merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Status: Patch Available (was: Open) merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Created] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
Walter Su created HDFS-8390: --- Summary: merge BlockPlacementPolicyWithNodeGroup with default policy Key: HDFS-8390 URL: https://issues.apache.org/jira/browse/HDFS-8390 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14543136#comment-14543136 ] Walter Su commented on HDFS-7621: - Thanks [~zhz] for helpful comments.

[jira] [Updated] (HDFS-8390) merge BlockPlacementPolicyWithNodeGroup with default policy

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8390: Attachment: HDFS-8390.002.patch merge BlockPlacementPolicyWithNodeGroup with default policy

[jira] [Updated] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-05-13 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-7621: Attachment: HDFS-7621.003.patch Erasure Coding: update the Balancer/Mover data migration logic

[jira] [Updated] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8364: Description: issue 1: it's not user-friendly. We should tell user that `hdfs erasurecode` is the

[jira] [Updated] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8364: Attachment: HDFS-8364-HDFS-7285.001.patch Erasure coding: fix some minor bugs in EC CLI

[jira] [Updated] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8364: Status: Patch Available (was: Open) Erasure coding: fix some minor bugs in EC CLI

[jira] [Commented] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539444#comment-14539444 ] Walter Su commented on HDFS-8364: - after 001 patch: # bin/hdfs erasurecode -getZoneInfo /

[jira] [Updated] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8364: Attachment: HDFS-8364-HDFS-7285.002.patch Erasure coding: fix some minor bugs in EC CLI

[jira] [Commented] (HDFS-8323) Bump GenerationStamp for write faliure in DFSStripedOutputStream

2015-05-12 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539696#comment-14539696 ] Walter Su commented on HDFS-8323: - 1. {{DFSStripedOutputStream}} owns the streamer lock

[jira] [Created] (HDFS-8365) Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

2015-05-11 Thread Walter Su (JIRA)
Walter Su created HDFS-8365: --- Summary: Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer Key: HDFS-8365 URL: https://issues.apache.org/jira/browse/HDFS-8365 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8365) Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su resolved HDFS-8365. - Resolution: Duplicate Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

[jira] [Updated] (HDFS-8365) Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8365: Description: Currently, each innerBlock of blockGroup should put on different node. One node can't has 2

[jira] [Commented] (HDFS-8220) Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537729#comment-14537729 ] Walter Su commented on HDFS-8220: - You moved assert nodes == The location is good. I

[jira] [Commented] (HDFS-8365) Erasure Coding: Badly treated when short of Datanode in StripedDataStreamer

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537686#comment-14537686 ] Walter Su commented on HDFS-8365: - Sorry I didn't saw it. Of course. Erasure Coding:

[jira] [Commented] (HDFS-8220) Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537872#comment-14537872 ] Walter Su commented on HDFS-8220: - bq. Assertions should be used to check something that

[jira] [Commented] (HDFS-8220) Erasure Coding: StripedDataStreamer fails to handle the blocklocations which doesn't satisfy BlockGroupSize

2015-05-11 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539185#comment-14539185 ] Walter Su commented on HDFS-8220: - Minor tips. I don't get expected Exception running test.

[jira] [Created] (HDFS-8364) Erasure coding: fix some minor bugs in EC CLI

2015-05-10 Thread Walter Su (JIRA)
Walter Su created HDFS-8364: --- Summary: Erasure coding: fix some minor bugs in EC CLI Key: HDFS-8364 URL: https://issues.apache.org/jira/browse/HDFS-8364 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-08 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Status: Patch Available (was: Open) Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Updated] (HDFS-8342) Erasure Coding: test skip in TestDFSStripedInputStream

2015-05-08 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8342: Attachment: HDFS-8342-HDFS-7285.001.patch Erasure Coding: test skip in TestDFSStripedInputStream

[jira] [Updated] (HDFS-8352) Erasure Coding: test webhdfs read write stripe file

2015-05-08 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8352: Status: Patch Available (was: Open) Erasure Coding: test webhdfs read write stripe file

<    4   5   6   7   8   9   10   11   12   >