[
https://issues.apache.org/jira/browse/CASSANDRA-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139728#comment-13139728
]
Mck SembWever commented on CASSANDRA-3150:
------------------------------------------
Back after an upgrade to cassandra-1.0.0
Example job start logs{noformat}[ INFO] 20:39:21 Restricting input range:
3589a548d20f80a7b41368b59973bcbc -- 36f0bedaf02a49a3b41368b59973bcbc [] at
no.finntech.countstats.reduce.rolled.internal.GenericCountAggregation.configureCountToAggregateMapper(GenericCountAggregation.java:222)
[ INFO] 20:39:21 Corresponding time range is Sun Oct 30 00:00:00 CEST 2011
(1319925600000) -- Sun Oct 30 23:00:00 CET 2011 (1320012000000) [] at
no.finntech.countstats.reduce.rolled.internal.GenericCountAggregation.configureCountToAggregateMapper(GenericCountAggregation.java:225)
[ INFO] 20:39:21 Starting AdIdCountAggregation-phase1-DAY_2011303 (
to=1320002999000)) [] at
no.finntech.countstats.reduce.rolled.internal.GenericCountAggregation.run(GenericCountAggregation.java:142)
[DEBUG] 20:39:21 adding
ColumnFamilySplit{startToken='3589a548d20f80a7b41368b59973bcbc',
endToken='36f0bedaf02a49a3b41368b59973bcbc', dataNodes=[0.0.0.0]} [] at
org.apache.cassandra.hadoop.ColumnFamilyInputFormat$SplitCallable.call(ColumnFamilyInputFormat.java:210)
{noformat}
In this split there is in fact ~40 million rows, and with a split size of
393216, it is expected ~100 splits to be returned.
I'm also very confused by the {{dataNodes=[0.0.0.0]}}. this looks to be wrong,
i know that data lies on the third node: cassandra03, where-else the job is
running on the first node: cassandra01.
> ColumnFormatRecordReader loops forever (StorageService.getSplits(..) out of
> whack)
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-3150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3150
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 0.8.4, 0.8.5
> Reporter: Mck SembWever
> Assignee: Mck SembWever
> Priority: Critical
> Fix For: 0.8.6
>
> Attachments: CASSANDRA-3150.patch, Screenshot-Counters for
> task_201109212019_1060_m_000029 - Mozilla Firefox.png, Screenshot-Hadoop map
> task list for job_201109212019_1060 on cassandra01 - Mozilla Firefox.png,
> attempt_201109071357_0044_m_003040_0.grep-get_range_slices.log,
> fullscan-example1.log
>
>
> From http://thread.gmane.org/gmane.comp.db.cassandra.user/20039
> {quote}
> bq. Cassandra-0.8.4 w/ ByteOrderedPartitioner
> bq. CFIF's inputSplitSize=196608
> bq. 3 map tasks (from 4013) is still running after read 25 million rows.
> bq. Can this be a bug in StorageService.getSplits(..) ?
> getSplits looks pretty foolproof to me but I guess we'd need to add
> more debug logging to rule out a bug there for sure.
> I guess the main alternative would be a bug in the recordreader paging.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
