[
https://issues.apache.org/jira/browse/CASSANDRA-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13857930#comment-13857930
]
Lars Bohl commented on CASSANDRA-6268:
--------------------------------------
I built from trunk today and restarted the cluster using the generated
build/dist folder in place of the 2.0.3 tarball. There were still over 50000
map tasks in hadoop, the same number as before. Maybe some setting in
cassandra.yaml needs to change, or some setting in cascading-cassandra source
tap?
> Poor performance of Hadoop if any DC is using VNodes
> ----------------------------------------------------
>
> Key: CASSANDRA-6268
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6268
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Reporter: Piotr Kołaczkowski
> Assignee: Piotr Kołaczkowski
> Fix For: 1.2.13, 2.0.4
>
> Attachments: 6268-src-1.2.txt, 6268-src-2.0.txt, 6268-thrift-1.2.txt,
> 6268-thrift-2.0.txt
>
>
> Some customers are complaining about huge number of splits in Hadoop caused
> by VNodes. Disabling vnodes only in Hadoop DC does not fix it. Splits are
> generated from the results of describe_ring, which returns a huge number of
> ranges anyways, and doesn't take into account that there will be huge number
> of consecutive ranges residing on the nodes we'd like the M/R job to be run.
> The proposed fix:
> 1. allows for specifying the DC(s) the Hadoop job should be run in (in DSE -
> defaults to all Hadoop DCs)
> 2. merges consecutive ranges before generating Hadoop splits, so we don't
> have artificial range splitting caused by vnodes in the other DCs
> For non-DSE users this feature is turned off by default and doesn't change
> the old behaviour.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
