[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16803199#comment-16803199
]
yunjiong zhao commented on HDFS-10477:
--
[~jojochuang], I don't mind, please go ahead.
Thank you.
>
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443349#comment-16443349
]
yunjiong zhao commented on HDFS-13441:
--
[~daryn] , you are right, it's not the best and reliable way
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Attachment: HDFS-13441.003.patch
> DataNode missed BlockKey update from NameNode due to
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441110#comment-16441110
]
yunjiong zhao commented on HDFS-13441:
--
[~hexiaoqiao], DataNode can't use NamenodeProtocol.
>
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440104#comment-16440104
]
yunjiong zhao commented on HDFS-13441:
--
Unit test failure is not related to this patch.
> DataNode
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439905#comment-16439905
]
yunjiong zhao commented on HDFS-13441:
--
Upload HDFS-13441.002.patch, which contain unit test.
>
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439897#comment-16439897
]
yunjiong zhao commented on HDFS-13441:
--
[~hexiaoqiao] , Let DataNode pull Block Key from NameNode is
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Attachment: HDFS-13441.002.patch
> DataNode missed BlockKey update from NameNode due to
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438531#comment-16438531
]
yunjiong zhao commented on HDFS-13441:
--
[~hexiaoqiao] , this issue is different, it is not about DN
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437740#comment-16437740
]
yunjiong zhao edited comment on HDFS-13441 at 4/13/18 9:50 PM:
---
{quote}
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Description:
After NameNode failover, lots of application failed due to some DataNodes can't
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Description:
After NameNode failover, lots of application failed due to some DataNodes can't
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437740#comment-16437740
]
yunjiong zhao commented on HDFS-13441:
--
{quote}
BlockKey is usually synchronized aggressively –
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Status: Patch Available (was: Open)
There are two ways to fix this bug: one is making sure
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Attachment: HDFS-13441.patch
> DataNode missed BlockKey update from NameNode due to
[
https://issues.apache.org/jira/browse/HDFS-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-13441:
-
Description:
After NameNode failover, lots of application failed due to some DataNodes can't
yunjiong zhao created HDFS-13441:
Summary: DataNode missed BlockKey update from NameNode due to
HeartbeatResponse was dropped
Key: HDFS-13441
URL: https://issues.apache.org/jira/browse/HDFS-13441
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995826#comment-15995826
]
yunjiong zhao commented on HDFS-11384:
--
[~shv] Thanks for the fix.
> Add option for balancer to
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933354#comment-15933354
]
yunjiong zhao commented on HDFS-11384:
--
Thanks [~shv] for review.
Only when you set
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933354#comment-15933354
]
yunjiong zhao edited comment on HDFS-11384 at 3/20/17 7:26 PM:
---
Thanks
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: HDFS-11384.002.patch
> Add option for balancer to disperse getBlocks calls to avoid
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: (was: HDFS-11384.002.patch)
> Add option for balancer to disperse getBlocks calls
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: HDFS-11384.002.patch
Use Semaphore instead lock to avoid findbug warning.
> Add
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: (was: HDFS-11384.001.patch)
> Add option for balancer to disperse getBlocks calls
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: HDFS-11384.001.patch
> Add option for balancer to disperse getBlocks calls to avoid
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891198#comment-15891198
]
yunjiong zhao commented on HDFS-11384:
--
Thank you [~benoyantony] for your time to review this patch.
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Attachment: balancer.day.png
balancer.week.png
[
https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11384:
-
Status: Patch Available (was: Open)
> Add option for balancer to disperse getBlocks calls to
yunjiong zhao created HDFS-11384:
Summary: Add option for balancer to disperse getBlocks calls to
avoid NameNode's rpc.CallQueueLength spike
Key: HDFS-11384
URL: https://issues.apache.org/jira/browse/HDFS-11384
[
https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848792#comment-15848792
]
yunjiong zhao edited comment on HDFS-11377 at 2/1/17 7:03 PM:
--
Removed unused
[
https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11377:
-
Attachment: HDFS-11377.002.patch
Remove unused variable MAX_NO_PENDING_MOVE_ITERATIONS.
Thanks
[
https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11377:
-
Status: Patch Available (was: Open)
> Balancer hung due to "No mover threads available"
>
[
https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-11377:
-
Attachment: HDFS-11377.001.patch
Remove PendingMove if after "No mover threads available" in this
yunjiong zhao created HDFS-11377:
Summary: Balancer hung due to "No mover threads available"
Key: HDFS-11377
URL: https://issues.apache.org/jira/browse/HDFS-11377
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-10831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10831:
-
Status: Patch Available (was: Open)
> Add log when URLConnectionFactory.openConnection failed
>
[
https://issues.apache.org/jira/browse/HDFS-10831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10831:
-
Attachment: HDFS-10831.001.patch
Since this patch only added on line code to log error
yunjiong zhao created HDFS-10831:
Summary: Add log when URLConnectionFactory.openConnection failed
Key: HDFS-10831
URL: https://issues.apache.org/jira/browse/HDFS-10831
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Attachment: HDFS-10477.005.patch
Update patch to fix unit test.
When called by tests like
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Attachment: HDFS-10477.004.patch
Update patch with below changes:
1. release lock after finish
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366966#comment-15366966
]
yunjiong zhao commented on HDFS-10477:
--
Those failed unit test is not related to this patch.
And
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Status: Open (was: Patch Available)
> Stop decommission a rack of DataNodes caused NameNode fail
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Status: Patch Available (was: Open)
> Stop decommission a rack of DataNodes caused NameNode fail
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Attachment: HDFS-10477.003.patch
Update patch according comments.
Thanks [~benoyantony]
> Stop
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314451#comment-15314451
]
yunjiong zhao commented on HDFS-10477:
--
[~kihwal],
What's your opinion on the second patch? Any
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Attachment: HDFS-10477.002.patch
[~kihwal] good idea, thanks.
We can release lock in
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Attachment: HDFS-10477.patch
This patch will release write lock after stopped decommission one
[
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-10477:
-
Status: Patch Available (was: Open)
> Stop decommission a rack of DataNodes caused NameNode fail
yunjiong zhao created HDFS-10477:
Summary: Stop decommission a rack of DataNodes caused NameNode
fail over to standby
Key: HDFS-10477
URL: https://issues.apache.org/jira/browse/HDFS-10477
Project:
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.5.patch
Thanks Arpit Agarwal.
Update the patch according to Arpit Agarwal's
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.4.patch
Thanks Tsz Wo Nicholas Sze for review the patch.
Update the patch to
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.3.withtest.patch
HDFS-9959.3.patch
Update patch according to
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.2.patch
How about this one?
In case of extreme case, for example, all
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204604#comment-15204604
]
yunjiong zhao commented on HDFS-9959:
-
+1 for this.
> add log when block removed from last live
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202137#comment-15202137
]
yunjiong zhao commented on HDFS-9959:
-
I understand, thanks.
> add log when block removed from last
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.1.patch
Update patch:
1. log after release the write lock
2. change error to
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194482#comment-15194482
]
yunjiong zhao commented on HDFS-9959:
-
It shouldn't, it will only print those blocks which was removed
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194351#comment-15194351
]
yunjiong zhao commented on HDFS-9959:
-
If removeNode(Block b, DatanodeDescriptor node) was invoked by
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194245#comment-15194245
]
yunjiong zhao commented on HDFS-9959:
-
For other logs, it is not that convenient. For example, if the
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Status: Patch Available (was: Open)
> add log when block removed from last live datanode
>
[
https://issues.apache.org/jira/browse/HDFS-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yunjiong zhao updated HDFS-9959:
Attachment: HDFS-9959.patch
> add log when block removed from last live datanode
>
yunjiong zhao created HDFS-9959:
---
Summary: add log when block removed from last live datanode
Key: HDFS-9959
URL: https://issues.apache.org/jira/browse/HDFS-9959
Project: Hadoop HDFS
Issue
61 matches
Mail list logo