[
https://issues.apache.org/jira/browse/HDFS-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561504#comment-13561504
]
nkeywal commented on HDFS-4350:
---
yep, I confirm there is no code change in HBase.
[
https://issues.apache.org/jira/browse/HDFS-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13560952#comment-13560952
]
nkeywal commented on HDFS-4350:
---
For HBase, it's not a big deal:
- in all case, both write
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13532488#comment-13532488
]
nkeywal commented on HDFS-3912:
---
Are you sure? It's committed in branch-1?
[
https://issues.apache.org/jira/browse/HDFS-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482458#comment-13482458
]
nkeywal commented on HDFS-3704:
---
The dead note list exists already, but is used only when
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473910#comment-13473910
]
nkeywal commented on HDFS-3912:
---
@Suresh, @Jing, Thanks a lot for doing the backport! I'm
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474272#comment-13474272
]
nkeywal commented on HDFS-3912:
---
Very good news: It works as expected :-). I don't have
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473246#comment-13473246
]
nkeywal commented on HDFS-3912:
---
Hi,
I'm ok with the new logic for the warning, 3 times the
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470412#comment-13470412
]
nkeywal commented on HDFS-3912:
---
It will try the patch on HBase 0.96 next week (hopefully).
I
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468535#comment-13468535
]
nkeywal commented on HDFS-3912:
---
I haven't kept the log files, but it was in the small test
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468542#comment-13468542
]
nkeywal commented on HDFS-3912:
---
Actually it seems it's HBASE-6928; so it's not related to
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal reassigned HDFS-3912:
-
Assignee: nkeywal (was: Jing Zhao)
Detecting and avoiding stale datanodes for writing
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467864#comment-13467864
]
nkeywal commented on HDFS-3912:
---
I like this approach, it's deterministic.
I had issues
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468131#comment-13468131
]
nkeywal commented on HDFS-3912:
---
@Sureshaaa
I was echoing my message from the 21th: I had
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13460811#comment-13460811
]
nkeywal commented on HDFS-3912:
---
Hi Jing,
Are you working on it currently? I would like to
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13460833#comment-13460833
]
nkeywal commented on HDFS-3912:
---
Great! So I will test it beginning of next week then. Thanks
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455165#comment-13455165
]
nkeywal commented on HDFS-3703:
---
Thanks a lot, all. I can't wait to have hardware issues on
[
https://issues.apache.org/jira/browse/HDFS-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455262#comment-13455262
]
nkeywal commented on HDFS-3912:
---
Some thinking, with an HBase bias:
- if the datanode is too
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13454076#comment-13454076
]
nkeywal commented on HDFS-3703:
---
Well, I think it would be quite useful on v1 as well, as
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452005#comment-13452005
]
nkeywal commented on HDFS-3703:
---
fyi, I've got an issue when trying with HBase on a real
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452303#comment-13452303
]
nkeywal commented on HDFS-3703:
---
Hi,
For the DFSInputStream#readBlockLength, I got it I
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452439#comment-13452439
]
nkeywal commented on HDFS-3703:
---
@Jing
Sure. The scenario is:
- open a file for writing,
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450777#comment-13450777
]
nkeywal commented on HDFS-3701:
---
Hi Uma and Nicholas,
I uploaded a merged and rebased patch,
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3701:
--
Attachment: HDFS-3701.branch-1.v2.merged.patch
HDFS may miss the final block when reading a file opened
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3701:
--
Attachment: (was: HDFS-3701.branch-1.v2.merged.patch)
HDFS may miss the final block when reading a
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3701:
--
Attachment: HDFS-3701.branch-1.v2.merged.patch
HDFS may miss the final block when reading a file opened
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450884#comment-13450884
]
nkeywal commented on HDFS-3705:
---
Yes, exactly, for all your points. Latency is key, so the
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3703:
--
Attachment: HDFS-3703-branch2.patch
Decrease the datanode failure detection time
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13448997#comment-13448997
]
nkeywal commented on HDFS-3703:
---
Hi,
I've done a test on HBase, with the minicluster.
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13447878#comment-13447878
]
nkeywal commented on HDFS-3703:
---
Hi Jing, Suresh,
I'm currently testing the patch with HBase
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442480#comment-13442480
]
nkeywal commented on HDFS-3701:
---
Hi Uma,
I've done a few changes, it seems to work. HBase
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3701:
--
Attachment: HDFS-3701.ontopof.v1.patch
HDFS may miss the final block when reading a file opened for
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442588#comment-13442588
]
nkeywal commented on HDFS-3705:
---
Hello Suresh,
For sure, HDFS-3703 is absolutely key for
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442182#comment-13442182
]
nkeywal commented on HDFS-3701:
---
Thanks Uma. Sure, I will try it early next week.
[
https://issues.apache.org/jira/browse/HDFS-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13434882#comment-13434882
]
nkeywal commented on HDFS-1075:
---
This could be very useful. The connect is not impacted by
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3705:
--
Affects Version/s: 3.0.0
Status: Open (was: Patch Available)
Add the possibility to mark a
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3705:
--
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
Add the possibility to mark a node as
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3705:
--
Attachment: HDFS-3705.v1.patch
Add the possibility to mark a node as 'low priority' for read in the
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13432693#comment-13432693
]
nkeywal commented on HDFS-3705:
---
v1. Complete patch on hdfs trunk.
Add the
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431821#comment-13431821
]
nkeywal commented on HDFS-3701:
---
Hi Uma,
Have you been able to generate the patch?
Thanks
[
https://issues.apache.org/jira/browse/HDFS-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431822#comment-13431822
]
nkeywal commented on HDFS-3704:
---
Yes, I agree. There is a dependency on HDFS-3701, as it
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431898#comment-13431898
]
nkeywal commented on HDFS-3701:
---
Thanks for the quick answer, Uma. It's not a matter of days
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13427896#comment-13427896
]
nkeywal commented on HDFS-3705:
---
It would be great to get some feedback from hdfs committers
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13428189#comment-13428189
]
nkeywal commented on HDFS-3705:
---
Hi Suresh,
Thanks a lot for having a look at this. I
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13425629#comment-13425629
]
nkeywal commented on HDFS-3703:
---
For HBase, it would be good to have an option to cap this
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13424910#comment-13424910
]
nkeywal commented on HDFS-3703:
---
The 3 approaches all have advantages. I don't think they are
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HDFS-3705:
--
Attachment: hdfs-3705.sample.patch
Add the possibility to mark a node as 'low priority' for read in the
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13424919#comment-13424919
]
nkeywal commented on HDFS-3705:
---
Attached a possible interface on branch 2.
- allow the
[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422372#comment-13422372
]
nkeywal commented on HDFS-3702:
---
For a file open in writing, we will start with a call to
[
https://issues.apache.org/jira/browse/HDFS-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422374#comment-13422374
]
nkeywal commented on HDFS-3705:
---
Most of the feedback is on the other jiras. For this
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421239#comment-13421239
]
nkeywal commented on HDFS-3701:
---
@Uma
I don't have the patch, so if you generate it it's
nkeywal created HDFS-3701:
-
Summary: HDFS may miss the final block when reading a file opened
for writing if one of the datanode is dead
Key: HDFS-3701
URL: https://issues.apache.org/jira/browse/HDFS-3701
nkeywal created HDFS-3702:
-
Summary: Add an option for NOT writing the blocks locally if there
is a datanode on the same box as the client
Key: HDFS-3702
URL: https://issues.apache.org/jira/browse/HDFS-3702
nkeywal created HDFS-3703:
-
Summary: Decrease the datanode failure detection time
Key: HDFS-3703
URL: https://issues.apache.org/jira/browse/HDFS-3703
Project: Hadoop HDFS
Issue Type: Improvement
nkeywal created HDFS-3704:
-
Summary: In the DFSClient, Add the node to the dead list when the
ipc.Client calls fails
Key: HDFS-3704
URL: https://issues.apache.org/jira/browse/HDFS-3704
Project: Hadoop HDFS
nkeywal created HDFS-3705:
-
Summary: Add the possibility to mark a node as 'low priority' for
read in the DFSClient
Key: HDFS-3705
URL: https://issues.apache.org/jira/browse/HDFS-3705
Project: Hadoop HDFS
nkeywal created HDFS-3706:
-
Summary: Add the possibility to mark a node as 'low priority' for
writes in the DFSClient
Key: HDFS-3706
URL: https://issues.apache.org/jira/browse/HDFS-3706
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420676#comment-13420676
]
nkeywal commented on HDFS-3702:
---
bq. When you say NOT write the block locally, do you mean
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420689#comment-13420689
]
nkeywal commented on HDFS-3701:
---
Hi Uma,
Thank you very much for the feedback. In 1.0.3
[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420829#comment-13420829
]
nkeywal commented on HDFS-3702:
---
bq. stack and nkeywal, when a node dies, there is a
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420851#comment-13420851
]
nkeywal commented on HDFS-3703:
---
bq. Can you describe this better?
If we see this in layers,
[
https://issues.apache.org/jira/browse/HDFS-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13420871#comment-13420871
]
nkeywal commented on HDFS-3701:
---
btw, I if you're ok with the approach I can propose a patch
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421005#comment-13421005
]
nkeywal commented on HDFS-3703:
---
bq. If the last heartbeat time for datanode is more than
[
https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13421009#comment-13421009
]
nkeywal commented on HDFS-3702:
---
It would work. But we're adding unnecessary local workload,
63 matches
Mail list logo
