[jira] [Commented] (HDFS-5155) Depricate API of getStorageID() and setStorageID() in DatanodeID as multiple storages

[ https://issues.apache.org/jira/browse/HDFS-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771460#comment-13771460 ] Junping Du commented on HDFS-5155: -- Hi Arpit, looks like HDFS-4988 already cover the work

[jira] [Resolved] (HDFS-5155) Depricate API of getStorageID() and setStorageID() in DatanodeID as multiple storages

[ https://issues.apache.org/jira/browse/HDFS-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved HDFS-5155. -- Resolution: Duplicate Depricate API of getStorageID() and setStorageID() in DatanodeID as multiple

[jira] [Commented] (HDFS-5225) datanode keeps logging the same 'is no longer in the dataset' message over and over again

[ https://issues.apache.org/jira/browse/HDFS-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771520#comment-13771520 ] Junping Du commented on HDFS-5225: -- Hi Colin and Roman, I think the following code won't

[jira] [Commented] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens

[ https://issues.apache.org/jira/browse/HDFS-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771529#comment-13771529 ] Junping Du commented on HDFS-5208: -- Thanks for comments, Colin! bq. Unfortunately,

[jira] [Commented] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13768203#comment-13768203 ] Junping Du commented on HDFS-5207: -- Thanks for great comments, Nicholas! I addressed them

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Attachment: HDFS-5207-v3.patch In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Attachment: HDFS-5207-v4.patch Sorry. The cast issue should be addressed in v3 patch. Now update in v4

[jira] [Commented] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens

[ https://issues.apache.org/jira/browse/HDFS-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13769013#comment-13769013 ] Junping Du commented on HDFS-5208: -- Thanks for comments, Colin! I checked again on all

[jira] [Commented] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13769129#comment-13769129 ] Junping Du commented on HDFS-5207: -- Thanks Nicholas for review! :) In

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, the type of writer in parameters of chooseTarget() should be updated from DatanodeDescriptor to Node

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Attachment: HDFS-5207-v1.patch Upload the first patch. Do following cleanups in

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, the type of writer in parameters of chooseTarget() should be updated from DatanodeDescriptor to Node

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Status: Patch Available (was: Open) In BlockPlacementPolicy, the type of writer in parameters of

[jira] [Commented] (HDFS-5188) Clean up BlockPlacementPolicy and its implementations

[ https://issues.apache.org/jira/browse/HDFS-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13768007#comment-13768007 ] Junping Du commented on HDFS-5188: -- Hi Nicholas, one related JIRA of HDFS-5207 (to fix

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Summary: In BlockPlacementPolicy, update 2 parameters of chooseTarget() (was: In BlockPlacementPolicy,

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Description: We should change chooseTarget(..., DatanodeDescriptor writer, ...) to chooseTarget (...,

[jira] [Updated] (HDFS-5207) In BlockPlacementPolicy, update 2 parameters of chooseTarget()

[ https://issues.apache.org/jira/browse/HDFS-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5207: - Attachment: HDFS-5207-v2.patch Fix tiny issues on comments in v2 patch. In

[jira] [Created] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens

Junping Du created HDFS-5208: Summary: Only clear network location cache on specific nodes if invalid NetworkTopology happens Key: HDFS-5208 URL: https://issues.apache.org/jira/browse/HDFS-5208 Project:

[jira] [Updated] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens

[ https://issues.apache.org/jira/browse/HDFS-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5208: - Attachment: HDFS-5208-v1.patch Update the patch. Please note that the additional API added in

[jira] [Updated] (HDFS-5208) Only clear network location cache on specific nodes if invalid NetworkTopology happens

[ https://issues.apache.org/jira/browse/HDFS-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5208: - Target Version/s: 1.3.0, 2.1.1-beta Status: Patch Available (was: Open) Only clear

[jira] [Commented] (HDFS-5188) Clean up BlockPlacementPolicy and its implementations

[ https://issues.apache.org/jira/browse/HDFS-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767647#comment-13767647 ] Junping Du commented on HDFS-5188: -- Thanks for nice work, Nicholas! The patch looks good

[jira] [Commented] (HDFS-5188) Clean up BlockPlacementPolicy and its implementations

[ https://issues.apache.org/jira/browse/HDFS-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767673#comment-13767673 ] Junping Du commented on HDFS-5188: -- bq. For simple JIRAs, non-committer can review and +1

[jira] [Commented] (HDFS-4521) invalid network topologies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767677#comment-13767677 ] Junping Du commented on HDFS-4521: -- A minor optimization here is cleanup cache only on

[jira] [Commented] (HDFS-5188) Clean up BlockPlacementPolicy and its implementations

[ https://issues.apache.org/jira/browse/HDFS-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767221#comment-13767221 ] Junping Du commented on HDFS-5188: -- Thanks for the patch. Nicholas! I haven't finished my

[jira] [Created] (HDFS-5207) In BlockPlacementPolicy, the type of writer in parameters of chooseTarget() should be updated from DatanodeDescriptor to Node

Junping Du created HDFS-5207: Summary: In BlockPlacementPolicy, the type of writer in parameters of chooseTarget() should be updated from DatanodeDescriptor to Node Key: HDFS-5207 URL:

[jira] [Commented] (HDFS-5188) Clean up BlockPlacementPolicy and its implementations

[ https://issues.apache.org/jira/browse/HDFS-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767341#comment-13767341 ] Junping Du commented on HDFS-5188: -- Thanks for quickly response, Nicholas! 3. Agree. It

[jira] [Updated] (HDFS-5157) Add StorageType to FsVolume

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5157: - Description: To support heterogeneous storage, Datanode should aware (and manage later) different storage

[jira] [Commented] (HDFS-5157) Add StorageType to FsVolume

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764032#comment-13764032 ] Junping Du commented on HDFS-5157: -- Updated. Thanks for your review. Nicholas!

[jira] [Commented] (HDFS-5157) Add StorageType to FsVolume

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13764062#comment-13764062 ] Junping Du commented on HDFS-5157: -- Thanks Nicholas for review and comments!

[jira] [Created] (HDFS-5178) DataTransferProtocol changes for supporting mutiple storages per DN

Junping Du created HDFS-5178: Summary: DataTransferProtocol changes for supporting mutiple storages per DN Key: HDFS-5178 URL: https://issues.apache.org/jira/browse/HDFS-5178 Project: Hadoop HDFS

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762956#comment-13762956 ] Junping Du commented on HDFS-5157: -- Thanks for input. Nicholas! Filed HDFS-5178 to track

[jira] [Commented] (HDFS-4990) Block placement for storage types

[ https://issues.apache.org/jira/browse/HDFS-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763058#comment-13763058 ] Junping Du commented on HDFS-4990: -- Another question here is: here we specify StorageID

[jira] [Commented] (HDFS-4898) BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack

[ https://issues.apache.org/jira/browse/HDFS-4898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763101#comment-13763101 ] Junping Du commented on HDFS-4898: -- Thanks for fixing it. Eric and Nicholas! Link it to

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763163#comment-13763163 ] Junping Du commented on HDFS-5157: -- Hi Nicholas, I think both way have its strength. #2 is

[jira] [Updated] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5157: - Attachment: HDFS-5157-v2.patch Datanode should allow choosing the target storage

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763125#comment-13763125 ] Junping Du commented on HDFS-5157: -- Thanks for review and comments. Nicholas! I address

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763705#comment-13763705 ] Junping Du commented on HDFS-5157: -- Sure. Arpit. Let's wait their feedback first. However,

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763745#comment-13763745 ] Junping Du commented on HDFS-5157: -- bq. Since #2 has a global view of the entire cluster,

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763906#comment-13763906 ] Junping Du commented on HDFS-5157: -- bq. Since it is an incompatible change, let's leave

[jira] [Created] (HDFS-5183) Combine ReplicaPlacementPolicy with VolumeChoosingPolicy together to have a global view in choosing DN storage for replica.

Junping Du created HDFS-5183: Summary: Combine ReplicaPlacementPolicy with VolumeChoosingPolicy together to have a global view in choosing DN storage for replica. Key: HDFS-5183 URL:

[jira] [Updated] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5157: - Attachment: HDFS-5157-v3.patch Datanode should allow choosing the target storage

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763930#comment-13763930 ] Junping Du commented on HDFS-5157: -- Filed HDFS-5183 to combine BlockPlacementPolicy

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761677#comment-13761677 ] Junping Du commented on HDFS-5157: -- Oh. I see. Just notice the field of targets in

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761682#comment-13761682 ] Junping Du commented on HDFS-5157: -- BTW, can we address volume policy first (as no

[jira] [Updated] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5157: - Attachment: HDFS-5157-v1.patch Attach the first patch with addressing volumeChoosePolicy only. Test

[jira] [Commented] (HDFS-4990) Block placement for storage types

[ https://issues.apache.org/jira/browse/HDFS-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762509#comment-13762509 ] Junping Du commented on HDFS-4990: -- Hi Nicholas, thanks for the patch. Just some initial

[jira] [Commented] (HDFS-5168) BlockPlacementPolicy does not work for cross rack/node group dependencies

[ https://issues.apache.org/jira/browse/HDFS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762553#comment-13762553 ] Junping Du commented on HDFS-5168: -- IMO, this is a more generic scope effort as proposing

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13762760#comment-13762760 ] Junping Du commented on HDFS-5157: -- Ok.Thanks! Datanode should allow

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761223#comment-13761223 ] Junping Du commented on HDFS-5157: -- Also, DataTransferProtocol need to be updated with

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761568#comment-13761568 ] Junping Du commented on HDFS-5157: -- Hi Nicholas, Do you mean we should change parameter of

[jira] [Assigned] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned HDFS-5157: Assignee: Junping Du Datanode should allow choosing the target storage

[jira] [Commented] (HDFS-5157) Datanode should allow choosing the target storage

[ https://issues.apache.org/jira/browse/HDFS-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13761182#comment-13761182 ] Junping Du commented on HDFS-5157: -- I am thinking to add an interface of getStorageType()

[jira] [Commented] (HDFS-5168) BlockPlacementPolicy does not work for cross rack/node group dependencies

[ https://issues.apache.org/jira/browse/HDFS-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13760872#comment-13760872 ] Junping Du commented on HDFS-5168: -- That's a good idea. Nichola! I think you are

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755938#comment-13755938 ] Junping Du commented on HDFS-5154: -- Hi [~szetszwo] and [~arpitagarwal], are these fixes

[jira] [Commented] (HDFS-5009) NameNode should expose storage information in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756098#comment-13756098 ] Junping Du commented on HDFS-5009: -- Hi Nicholas, Thanks for the patch. I just finish my

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756258#comment-13756258 ] Junping Du commented on HDFS-5154: -- Thanks Nicholas for review and great comments. Will

[jira] [Updated] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5154: - Attachment: HDFS-5154-v2.patch Patch updated. Thx! Fix several tests failure after

[jira] [Commented] (HDFS-5009) NameNode should expose storage information in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756273#comment-13756273 ] Junping Du commented on HDFS-5009: -- Nicholas, Thanks for addressing my comments! On

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756337#comment-13756337 ] Junping Du commented on HDFS-5154: -- Ah~~ Sorry. I forget to include this change on v2

[jira] [Updated] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5154: - Attachment: HDFS-5154-v3.patch Fix several tests failure after HDFS-4987

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756337#comment-13756337 ] Junping Du commented on HDFS-5154: -- Ah~~ Sorry. I forget to include this change on v2

[jira] [Commented] (HDFS-5009) NameNode should expose storage information in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756365#comment-13756365 ] Junping Du commented on HDFS-5009: -- Thanks Nicholas for update. New patch looks good to

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756374#comment-13756374 ] Junping Du commented on HDFS-5154: -- Ok. Thanks for good suggestion. Suresh!

[jira] [Updated] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5154: - Attachment: HDFS-5154-v1.patch Fix several tests failure after HDFS-4987

[jira] [Commented] (HDFS-5154) Fix several tests failure after HDFS-4987

[ https://issues.apache.org/jira/browse/HDFS-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755732#comment-13755732 ] Junping Du commented on HDFS-5154: -- Fix TestBlockManager and TestDatanodeDescriptor that

[jira] [Created] (HDFS-5155) Depricate API of getStorageID() and setStorageID() in DatanodeID as multiple storages

Junping Du created HDFS-5155: Summary: Depricate API of getStorageID() and setStorageID() in DatanodeID as multiple storages Key: HDFS-5155 URL: https://issues.apache.org/jira/browse/HDFS-5155 Project:

[jira] [Commented] (HDFS-5134) NameNode improvements to per-storage block processing

[ https://issues.apache.org/jira/browse/HDFS-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755735#comment-13755735 ] Junping Du commented on HDFS-5134: -- And also bandwidth for balancer should be per storage.

[jira] [Created] (HDFS-5154) Fix several tests failure after HDFS-4987

Junping Du created HDFS-5154: Summary: Fix several tests failure after HDFS-4987 Key: HDFS-5154 URL: https://issues.apache.org/jira/browse/HDFS-5154 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-4987) Namenode changes to track multiple storages

[ https://issues.apache.org/jira/browse/HDFS-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755403#comment-13755403 ] Junping Du commented on HDFS-4987: -- File HDFS-5154 to track unit tests faiure.

[jira] [Commented] (HDFS-5009) NameNode should expose storage information in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13753543#comment-13753543 ] Junping Du commented on HDFS-5009: -- Sure. Nicholas. I am glad to help here. Yes. That

[jira] [Commented] (HDFS-2832) Enable support for heterogeneous storages in HDFS

[ https://issues.apache.org/jira/browse/HDFS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13751195#comment-13751195 ] Junping Du commented on HDFS-2832: -- Thanks for quickly reply. [~arpitagarwal]. bq. We have

[jira] [Commented] (HDFS-4987) Namenode changes to track multiple storages

[ https://issues.apache.org/jira/browse/HDFS-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13752128#comment-13752128 ] Junping Du commented on HDFS-4987: -- Thanks for addressing comments in new patch. Patch

[jira] [Commented] (HDFS-5009) NameNode should expose storage type in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749943#comment-13749943 ] Junping Du commented on HDFS-5009: -- I think this is already addressed in patch of

[jira] [Commented] (HDFS-4987) Namenode changes to track multiple storages

[ https://issues.apache.org/jira/browse/HDFS-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750845#comment-13750845 ] Junping Du commented on HDFS-4987: -- Nicholas, nice code refactoring work on

[jira] [Commented] (HDFS-4987) Namenode changes to track multiple storages

[ https://issues.apache.org/jira/browse/HDFS-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750874#comment-13750874 ] Junping Du commented on HDFS-4987: -- One additional comment is we can do iteration blocks

[jira] [Assigned] (HDFS-4989) Balancer needs to consider storage type in balancing

[ https://issues.apache.org/jira/browse/HDFS-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned HDFS-4989: Assignee: Junping Du Balancer needs to consider storage type in balancing

[jira] [Commented] (HDFS-5009) NameNode should expose storage information in the LocatedBlock

[ https://issues.apache.org/jira/browse/HDFS-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750888#comment-13750888 ] Junping Du commented on HDFS-5009: -- Thanks [~arpitagarwal]. Looks like Nicholas is working

[jira] [Commented] (HDFS-4987) Namenode changes to track multiple storages

[ https://issues.apache.org/jira/browse/HDFS-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13750964#comment-13750964 ] Junping Du commented on HDFS-4987: -- bq. For your second comment, I actually don't

[jira] [Commented] (HDFS-2832) Enable support for heterogeneous storages in HDFS

[ https://issues.apache.org/jira/browse/HDFS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13749510#comment-13749510 ] Junping Du commented on HDFS-2832: -- Nice doc. My additional comments is: we should

[jira] [Resolved] (HDFS-4853) Handle unresolved InetAddress for favored nodes in DFSClient#create()

[ https://issues.apache.org/jira/browse/HDFS-4853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved HDFS-4853. -- Resolution: Duplicate Handle unresolved InetAddress for favored nodes in DFSClient#create()

[jira] [Reopened] (HDFS-4853) Handle unresolved InetAddress for favored nodes in DFSClient#create()

[ https://issues.apache.org/jira/browse/HDFS-4853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reopened HDFS-4853: -- Handle unresolved InetAddress for favored nodes in DFSClient#create()

[jira] [Updated] (HDFS-3880) Use Builder to get RPC server in HDFS

[ https://issues.apache.org/jira/browse/HDFS-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-3880: - Attachment: HDFS-3880-branch2.patch Backport to branch-2 as blocking HADOOP-9756 to branch-2 which remove

[jira] [Updated] (HDFS-5019) Cleanup imports in HDFS project

[ https://issues.apache.org/jira/browse/HDFS-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5019: - Attachment: HDFS-5019-v2.patch Some necessary imports are wrongly deleted. Fix it in v2 patch.

[jira] [Updated] (HDFS-5019) Cleanup imports in HDFS project

[ https://issues.apache.org/jira/browse/HDFS-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5019: - Attachment: HDFS-5019.patch Attached a patch that fixed by eclipse automatically which reduces 88

[jira] [Created] (HDFS-5019) Cleanup imports in HDFS project

Junping Du created HDFS-5019: Summary: Cleanup imports in HDFS project Key: HDFS-5019 URL: https://issues.apache.org/jira/browse/HDFS-5019 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-5019) Cleanup imports in HDFS project

[ https://issues.apache.org/jira/browse/HDFS-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5019: - Status: Patch Available (was: Open) Cleanup imports in HDFS project

[jira] [Assigned] (HDFS-5019) Cleanup imports in HDFS project

[ https://issues.apache.org/jira/browse/HDFS-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned HDFS-5019: Assignee: Junping Du Cleanup imports in HDFS project ---

[jira] [Commented] (HDFS-5018) Misspelled DFSConfigKeys#DFS_NAMENODE_STALE_DATANODE_INTERVAL_DEFAULT in javadoc of DatanodeInfo#isStale()

[ https://issues.apache.org/jira/browse/HDFS-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714920#comment-13714920 ] Junping Du commented on HDFS-5018: -- Nice finding! Ted. +1 (non-binding)

[jira] [Updated] (HDFS-4521) invalid network topologies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4521: - Assignee: Colin Patrick McCabe (was: Junping Du) invalid network topologies should not be cached

[jira] [Commented] (HDFS-4521) invalid network topologies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709501#comment-13709501 ] Junping Du commented on HDFS-4521: -- Thanks Colin for review and commit this!

[jira] [Resolved] (HDFS-3499) Make NetworkTopology support user specified topology class

[ https://issues.apache.org/jira/browse/HDFS-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved HDFS-3499. -- Resolution: Fixed Fix Version/s: 1.2.0 2.1.0-beta HADOOP-8469 already address

[jira] [Commented] (HDFS-4521) invalid network toploogies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687767#comment-13687767 ] Junping Du commented on HDFS-4521: -- Yes. branch-1 is quite different from branch-2. I

[jira] [Updated] (HDFS-4521) invalid network toploogies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4521: - Attachment: HDFS-4521-branch1.patch invalid network toploogies should not be cached

[jira] [Updated] (HDFS-4521) invalid network toploogies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4521: - Target Version/s: 2.1.0-beta, 1.2.1 (was: 2.1.0-beta) Hadoop Flags: (was: Reviewed)

[jira] [Commented] (HDFS-4521) invalid network toploogies should not be cached

[ https://issues.apache.org/jira/browse/HDFS-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687885#comment-13687885 ] Junping Du commented on HDFS-4521: -- Attach a patch for branch-1 and the major differences

[jira] [Commented] (HDFS-4261) TestBalancerWithNodeGroup times out

[ https://issues.apache.org/jira/browse/HDFS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680094#comment-13680094 ] Junping Du commented on HDFS-4261: -- Ok. Thanks Suresh!

[jira] [Created] (HDFS-4895) Add a function of -fixmisplaced to fsck for fixing blocks in mistaken placed

Junping Du created HDFS-4895: Summary: Add a function of -fixmisplaced to fsck for fixing blocks in mistaken placed Key: HDFS-4895 URL: https://issues.apache.org/jira/browse/HDFS-4895 Project: Hadoop

[jira] [Commented] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy

[ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677916#comment-13677916 ] Junping Du commented on HDFS-4885: -- As replica number is per file/directory property, and

[jira] [Updated] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy

[ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4885: - Attachment: HDFS-4885.patch Update verifyBlockPlacement() API in BlockPlacementPolicy

[jira] [Updated] (HDFS-4885) Update verifyBlockPlacement() API in BlockPlacementPolicy

[ https://issues.apache.org/jira/browse/HDFS-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4885: - Labels: BlockPlacementPolicy (was: ) Target Version/s: 3.0.0 Status: Patch

[jira] [Commented] (HDFS-4895) Add a function of -fixmisplaced to fsck for fixing blocks in mistaken placed

[ https://issues.apache.org/jira/browse/HDFS-4895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678627#comment-13678627 ] Junping Du commented on HDFS-4895: -- Hi Colin, this is also a option. However, IMO,

[jira] [Created] (HDFS-4884) [Umbrella] Block Placement Policy Optimizer

Junping Du created HDFS-4884: Summary: [Umbrella] Block Placement Policy Optimizer Key: HDFS-4884 URL: https://issues.apache.org/jira/browse/HDFS-4884 Project: Hadoop HDFS Issue Type: Bug

<    4   5   6   7   8   9   10   11   12   >