Please add the ability to provide retry logic for the ColumnFamilyRecordWriter
and ColumnFamilyRecordReader
-----------------------------------------------------------------------------------------------------------
Key: CASSANDRA-1264
URL: https://issues.apache.org/jira/browse/CASSANDRA-1264
Project: Cassandra
Issue Type: Improvement
Components: Hadoop
Reporter: James Mello
Looking at the code for the ColumnFamilyRecordWriter and
ColumnFamilyRecordReader, if the service throws a backpressure exception, the
map/reduce framework will end up having to do retries to continue on the job.
This failure scenario is pretty expensive and doesn't address transient load
exceptions. If the hadoop integration can add retry logic internally for these
scenarios, then the overall system would be greatly improved (IMO).
A current idea on this would be to use a custom thread pool executor. Then the
afterExecution method could be overridden to do retry if a specific type of
retry class is provided in the job conf. I'll be trying to get a code example
out soon as we need this internally, but if it proves to be a reasonable model,
we'd like to try to help out.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
