[
https://issues.apache.org/jira/browse/CASSANDRA-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904549#action_12904549
]
Juho Mäkinen commented on CASSANDRA-1264:
-----------------------------------------
I've ran into the same problem when I'm trying to do mapreduce on a super
column family which contains too big rows that the MR tasks occasionally
timeout.
> Please add the ability to provide retry logic for the
> ColumnFamilyRecordWriter and ColumnFamilyRecordReader
> -----------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-1264
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1264
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
> Reporter: James Mello
>
> Looking at the code for the ColumnFamilyRecordWriter and
> ColumnFamilyRecordReader, if the service throws a backpressure exception, the
> map/reduce framework will end up having to do retries to continue on the job.
> This failure scenario is pretty expensive and doesn't address transient load
> exceptions. If the hadoop integration can add retry logic internally for
> these scenarios, then the overall system would be greatly improved (IMO).
> A current idea on this would be to use a custom thread pool executor. Then
> the afterExecution method could be overridden to do retry if a specific type
> of retry class is provided in the job conf. I'll be trying to get a code
> example out soon as we need this internally, but if it proves to be a
> reasonable model, we'd like to try to help out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
