[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096327#comment-16096327 ] Hudson commented on HDFS-8818: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12043 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12043/]) HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by (kihwal: rev 8e3a992eccff26a7344c3f0e719898fa97706b8c) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998564#comment-15998564 ] Kihwal Lee commented on HDFS-8818: -- Posted a patch to HDFS-11742. Hopefully, this will prevent users from experiencing performance degradation and take the benefit of the per-target thread pool. I have test it overnight yesterday and it seems good. Now I see throughput improvement even with the default config values. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997774#comment-15997774 ] Kihwal Lee commented on HDFS-8818: -- I will post a patch for improvement tomorrow. I am letting it run thought the night. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997429#comment-15997429 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- > Can you reveal more details on the nature of the testing? It is unrealistic > to expect 100x in our typical use case with the base line being 2.7. In our tests, we ran balancer over a 500-node cluster. We were only able to get ~5GB per minute before. Then, we were able to get ~500GB per minute after a serious of balancer improvement including this. This JIRA is the most critical since, without this, balancer schedules most the moves in the first few datanode pairs and the remaining datanodes are mostly idle. Below are the confs: - Datanode dfs.datanode.balance.max.concurrent.moves: 4 x #disks dfs.datanode.balance.bandwidthPerSec: 10737418240 (=10GB) - Balancer dfs.datanode.balance.max.concurrent.moves: 4 x #disks dfs.balancer.moverThreads: 20,000 dfs.balancer.max-size-to-move: 107374182400 (=100GB) dfs.balancer.getBlocks.min-block-size: 104857600 (=100MB) > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996803#comment-15996803
]
Daryn Sharp commented on HDFS-8818:
---
Rather than creating fixed thread pools which will be idle as cluster size
increases, perhaps cached thread pools that spawn dynamically would help.
The previous balancer was easy to configure. I don't fully understand the
previous design but a simpler approach that achieves the same improvement would
be returning to a single fixed thread pool - with intelligent queuing of work.
Ie. interleaving work for all targets, with a max queued limit, so replications
are distributed evenly across nodes. I'm assuming it didn't do that.
{quote}
Do you have HDFS-8824 in your runs? I suspect the first run has it but the
second one does not.
bq. over time older nodes will end up with only small blocks, if it is set
permanently? It will look good for quick balancing, but may not be good in long
term
{quote}
Exactly. We had to disable the feature because nodes become concentrated with
small blocks. getBlocks becomes increasing expensive as it searches for a
dwindling number of large blocks on unbalanced nodes. The client load
increases on those nodes due to block volume. Eventually the balancer just
plays a shell game moving the larger blocks.
The current balancer probably works great when adding nodes, but not as a
continuous service. If not reverted, something has to be done to restore
previous steady state performance.
> Allow Balancer to run faster
>
>
> Key: HDFS-8818
> URL: https://issues.apache.org/jira/browse/HDFS-8818
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer & mover
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: bal1.png, bal2.png, h8818_20150723.patch,
> h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch
>
>
> The original design of Balancer is intentionally to make it run slowly so
> that the balancing activities won't affect the normal cluster activities and
> the running jobs.
> There are new use case that cluster admin may choose to balance the cluster
> when the cluster load is low, or in a maintain window. So that we should
> have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996738#comment-15996738 ] Kihwal Lee commented on HDFS-8818: -- bq. Do you have HDFS-8824 in your runs? I suspect the first run has it but the second one does not. It is the up-to-date branch-2.8, so all runs had it. About HDFS-8824, you do realize that over time older nodes will end up with only small blocks, if it is set permanently? It will look good for quick balancing, but may not be good in long term. We run with the min block size set to 1. bq. The performance did have been improved around 100x. Can you reveal more details on the nature of the testing? It is unrealistic to expect 100x in our typical use case with the base line being 2.7. What was your config when you tested on the 500 node cluster? What was the nature of imbalance? Did the default values work? If not, how did you get there? Do you expect regular users to easily get there? At what point did you hit HDFS-11377? > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995548#comment-15995548 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- [~kihwal], the metric replaceblockoperationspersec may not directly reflect the actual performance. Do you have HDFS-8824 in your runs? I suspect the first run has it but the second one does not. What are the values shown in the Balancer output? > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995534#comment-15995534 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- > ... I'm sure the redesign works great on a couple dozen node cluster. ... We never design Hadoop to ONLY work great on a couple dozen node cluster. We did have tested this with 500-node cluster. The performance did have been improved around 100x. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995525#comment-15995525 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- 15*72 = 1080. I guess you were using default fs.balancer.moverThreads, which is 1,000. Have you tried to increased it, say 10,000 or 30,000? > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995477#comment-15995477 ] Kihwal Lee commented on HDFS-8818: -- I set the dfs.datanode.balance.max.concurrent.moves to 15, which is what we used to use with 2.7. 71 thread pools with 15 threads in each created. The 72th one is small and then started to fail. So I still see "skipping..." message in the log. The throughput is still visibly lower, after a minute of initial spike. The iteration lasts a bit longer, but the average throughput is still similarly low. Other config values are all set to default. I believe I have presented enough evidence. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995450#comment-15995450 ] Daryn Sharp commented on HDFS-8818: --- bq. The new design is more flexible than the previous one since we can control the number of thread per datanode pair. That sounds good on paper but flexibility does not negate the fact it's proven not to scale. I'm sure the redesign works great on a couple dozen node cluster. As illustrated by Kihwal, it limps along on a 280 node cluster running slower than before and is virtually unusable on multi-thousand node clusters even with HDFS-11377. This has to be fixed in a manner that restores previous performance or be reverted. A jira touting "run faster" can't make the balancer slower and unfit for production... > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995369#comment-15995369 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- > ... However, please do realize that it will be a regression for many other > users. ... I believe this statement is incorrect. The new design is more flexible than the previous one since we can control the number of thread per datanode pair. BTW, using the default configuration may not give us performance since we want the balancing activities not affecting the normal cluster activities and the running jobs. > ... which cannot be worked around with config tweaks. ... What are the config used exactly? > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994840#comment-15994840 ] Kihwal Lee commented on HDFS-8818: -- We see this problem in EVERY single cluster we tried this on. The first big three bumps (20 min each) is 2.8 with this change reverted. The following small ones are the result of non-reverted 2.8 with the hang fix (HDFS-11377). It is still no where near what it used to be. !https://issues.apache.org/jira/secure/attachment/12866172/bal1.png! This is the stock 2.8 balancer without HDFS-11377. !https://issues.apache.org/jira/secure/attachment/12866171/bal2.png! I don't doubt that it worked great for the case you designed and tested for. However, please do realize that it will be a regression for many other users. I am not denying the shortcomings of the existing design. But the new design has clear issues, which cannot be worked around with config tweaks. Even if it works better for some cases, it is a regression. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal1.png, bal2.png, h8818_20150723.patch, > h8818_20150727.patch, HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993855#comment-15993855 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- Hi [~kihwal], let's have a more detailed discussion before reverting. The patch here (a thread pool per datanode pair) is indeed an improvement for the previous design (a global thread pool) since it limits the number of threads assigned to a particular datanode pair. Previously, if the first datanode pair has a lot of pending moves, all the threads will be used to execute the moves for that pair so that it will be very slow since it cannot utilize the entire network. We also has tested the new code a lot and see significant performance improvement. Have you tested it with HDFS-11377? > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Kihwal Lee > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: h8818_20150723.patch, h8818_20150727.patch, > HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993840#comment-15993840 ] Kihwal Lee commented on HDFS-8818: -- I will open a new one and post the patch there. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Kihwal Lee > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: bal_revert.branch-2.8.patch, bal_revert.branch-2.patch, > bal_revert.trunk.patch, h8818_20150723.patch, h8818_20150727.patch, > HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993824#comment-15993824 ] Andrew Wang commented on HDFS-8818: --- Can we open a new JIRA to track the revert in the changelog? This one has already gone out in some releases. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Kihwal Lee > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: h8818_20150723.patch, h8818_20150727.patch, > HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993723#comment-15993723 ] Kihwal Lee commented on HDFS-8818: -- Thanks for the update, [~jojochuang]. That fixes the hang, but the design is still flawed. Move decision is made then thread pools are created, which likely only cover a subset of targets in the schedule. Also, the schedule will less likely make use of all 50 threads anyway. In any case, the moves will all pile up to the first 25 targets (in our case, 25 thread pools seemed to be the limit). I don't think it will be faster than the previous one. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: h8818_20150723.patch, h8818_20150727.patch, > HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993664#comment-15993664 ] Wei-Chiu Chuang commented on HDFS-8818: --- [~kihwal] FWIW, the hung you mentioned is fixed in HDFS-11377. > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: h8818_20150723.patch, h8818_20150727.patch, > HDFS-8818-branch-2.7.00.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993654#comment-15993654
]
Kihwal Lee commented on HDFS-8818:
--
-1. Proposing revert of this untested change.
We've run it on a small (to us) 280 node cluster. It moves a little for a
couple of iterations and hangs forever. So, contrary to the title, it becomes
extremely slow. Blindly creating a thread pool per datanode is a bad idea.
{code:java}
public void executePendingMove(final PendingMove p) {
// move the block
final DDatanode targetDn = p.target.getDDatanode();
ExecutorService moveExecutor = targetDn.getMoveExecutor();
if (moveExecutor == null) {
final int nThreads =
moverThreadAllocator.allocate(maxConcurrentMovesPerNode);
if (nThreads > 0) {
moveExecutor = targetDn.initMoveExecutor(nThreads);
}
}
if (moveExecutor == null) {
LOG.warn("No mover threads available: skip moving " + p);
return;
}
{code}
By default, this causes a pool of 50 threads to be created per target and at
some point no more can be created. If this happens, the pending move is simply
skipped. This causes eternal hang at {{waitForMoveCompletion()}}, because the
pending moves are not removed. I saw 25 thread pools with 50 threads in the
280-node cluster in this hang state.
IMO, creating more threads is not a scalable solution. Somehow allowing more
thread pool creation is definitely not a solution either.
> Allow Balancer to run faster
>
>
> Key: HDFS-8818
> URL: https://issues.apache.org/jira/browse/HDFS-8818
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer & mover
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: h8818_20150723.patch, h8818_20150727.patch,
> HDFS-8818-branch-2.7.00.patch
>
>
> The original design of Balancer is intentionally to make it run slowly so
> that the balancing activities won't affect the normal cluster activities and
> the running jobs.
> There are new use case that cluster admin may choose to balance the cluster
> when the cluster load is low, or in a maintain window. So that we should
> have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485387#comment-15485387
]
Hadoop QA commented on HDFS-8818:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 7 new + 574 unchanged - 1 fixed = 581 total (was 575) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1967 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
51s{color} | {color:red} The patch 161 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 0s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_111. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
19s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}128m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_101 Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.web.TestHttpsFileSystem |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| JDK v1.7.0_111 Failed junit tests | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c420dfe |
| JIRA Issue | HDFS-8818 |
| JIRA Patch URL |
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485402#comment-15485402
]
Hadoop QA commented on HDFS-8818:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
25s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
56s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 7 new + 574 unchanged - 1 fixed = 581 total (was 575) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1967 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
47s{color} | {color:red} The patch 161 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} the patch passed with JDK v1.7.0_111 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 46s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_111. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
18s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}131m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_101 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.namenode.TestFSImageWithSnapshot |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| JDK v1.7.0_111 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.balancer.TestBalancer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484907#comment-15484907 ] Zhe Zhang commented on HDFS-8818: - Quick note that 2 failures were found even without the patch. HDFS-10859 > Allow Balancer to run faster > > > Key: HDFS-8818 > URL: https://issues.apache.org/jira/browse/HDFS-8818 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HDFS-8818-branch-2.7.00.patch, h8818_20150723.patch, > h8818_20150727.patch > > > The original design of Balancer is intentionally to make it run slowly so > that the balancing activities won't affect the normal cluster activities and > the running jobs. > There are new use case that cluster admin may choose to balance the cluster > when the cluster load is low, or in a maintain window. So that we should > have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681648#comment-14681648 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1014 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1014/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681643#comment-14681643 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #284 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/284/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681918#comment-14681918 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #281 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/281/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681915#comment-14681915 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2230 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2230/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681962#comment-14681962 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2211 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2211/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14681996#comment-14681996 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #273 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/273/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680964#comment-14680964 ] Jitendra Nath Pandey commented on HDFS-8818: +1 for the latest patch. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680996#comment-14680996 ] Hudson commented on HDFS-8818: -- FAILURE: Integrated in Hadoop-trunk-Commit #8281 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8281/]) HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. (szetszwo: rev b56daff6a186599764b046248565918b894ec116) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/MovedBlocks.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Fix For: 2.8.0 Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644620#comment-14644620
]
Chang Li commented on HDFS-8818:
[~szetszwo] thanks for the patch. One thing I am concerned is the change
{code}
- return srcBlocks.size() SOURCE_BLOCKS_MIN_SIZE blocksToReceive 0;
+ return blocksToReceive 0;
{code}
now that dispatcher will keep fetching more blocks from namenode every
iteration, but namenode is likely to return very same list of blocks since the
block moving is not that fast and namenode can't know the blocks just moved
instantly. This could increase useless load on namenode.
Allow Balancer to run faster
Key: HDFS-8818
URL: https://issues.apache.org/jira/browse/HDFS-8818
Project: Hadoop HDFS
Issue Type: Sub-task
Components: balancer mover
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h8818_20150723.patch, h8818_20150727.patch
The original design of Balancer is intentionally to make it run slowly so
that the balancing activities won't affect the normal cluster activities and
the running jobs.
There are new use case that cluster admin may choose to balance the cluster
when the cluster load is low, or in a maintain window. So that we should
have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644790#comment-14644790 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- now that dispatcher will keep fetching more blocks from namenode every iteration, but namenode is likely to return very same list of blocks since the block moving is not that fast and namenode can't know the blocks just moved instantly. ... I think it is not the case by the following reasons. - blocksToReceive will become = 0. - Balancer waits until all block transfer are done each iteration and Datanodes send a block receipt immediately once they receive a block. - BlockManager.getBlocks(..) uses a random offset to get the blocks from the list. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644888#comment-14644888 ] Chang Li commented on HDFS-8818: bq. Balancer waits until all block transfer are done each iteration and Datanodes send a block receipt immediately once they receive a block. But inside dispatchBlocks(), the Source will not wait for block gets transferred, it will quickly iterate and ask namenode for more blocks, even random offset can not prevent namenode from returning many same blocks, which will be waste Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644906#comment-14644906 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- Datanode store blocks in TB scale. Balancer only gets GB blocks. It seems unlikely to get the same blocks. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644919#comment-14644919 ] Chang Li commented on HDFS-8818: Also right now Souce will fetch from namenode no more than 2GB blocks at a time. IMO it's better to increase MAX_BLOCKS_SIZE_TO_FETCH, say about 10G. It's not efficient to ask namenode for this little amount each time and ask it a lot of times. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644944#comment-14644944 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- I am going to make MAX_BLOCKS_SIZE_TO_FETCH configurable in HDFS-8824. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch, h8818_20150727.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643274#comment-14643274 ] Jitendra Nath Pandey commented on HDFS-8818: +1 conditional on addressing findbugs/style issues. Allow Balancer to run faster Key: HDFS-8818 URL: https://issues.apache.org/jira/browse/HDFS-8818 Project: Hadoop HDFS Issue Type: Improvement Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h8818_20150723.patch The original design of Balancer is intentionally to make it run slowly so that the balancing activities won't affect the normal cluster activities and the running jobs. There are new use case that cluster admin may choose to balance the cluster when the cluster load is low, or in a maintain window. So that we should have an option to allow Balancer to run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643788#comment-14643788
]
Hadoop QA commented on HDFS-8818:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 0s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 34s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 20s | The applied patch generated 7
new checkstyle issues (total was 525, now 531). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 20s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 32s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 8s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 159m 31s | Tests failed in hadoop-hdfs. |
| | | 203m 2s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12747448/h8818_20150727.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3e6fce9 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11849/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11849/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11849/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11849/console |
This message was automatically generated.
Allow Balancer to run faster
Key: HDFS-8818
URL: https://issues.apache.org/jira/browse/HDFS-8818
Project: Hadoop HDFS
Issue Type: Sub-task
Components: balancer mover
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h8818_20150723.patch, h8818_20150727.patch
The original design of Balancer is intentionally to make it run slowly so
that the balancing activities won't affect the normal cluster activities and
the running jobs.
There are new use case that cluster admin may choose to balance the cluster
when the cluster load is low, or in a maintain window. So that we should
have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8818) Allow Balancer to run faster
[
https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14640298#comment-14640298
]
Hadoop QA commented on HDFS-8818:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 21m 53s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 10m 6s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 43s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 27s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 43s | The applied patch generated 7
new checkstyle issues (total was 524, now 530). |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 6 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 40s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 8s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 43s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 174m 34s | Tests failed in hadoop-hdfs. |
| | | 229m 40s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
| | hadoop.hdfs.TestParallelShortCircuitReadUnCached |
| | hadoop.hdfs.TestDistributedFileSystem |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12746966/h8818_20150723.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 02c0181 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11824/console |
This message was automatically generated.
Allow Balancer to run faster
Key: HDFS-8818
URL: https://issues.apache.org/jira/browse/HDFS-8818
Project: Hadoop HDFS
Issue Type: Improvement
Components: balancer mover
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h8818_20150723.patch
The original design of Balancer is intentionally to make it run slowly so
that the balancing activities won't affect the normal cluster activities and
the running jobs.
There are new use case that cluster admin may choose to balance the cluster
when the cluster load is low, or in a maintain window. So that we should
have an option to allow Balancer to run faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
