no I don't think increasing the timeout will help as this is not the source of the problem. Cassandra has a very simple write path this is boosted by high write speeds and throughput. I would guess that your Cassandra pin e.g. http://host:9160-2 cannot be mapped to. is there any reason that you've added the trailing hyphen two? hath lewis
On Wednesday, August 28, 2013, kaveh minooie <[email protected]> wrote: > > Hi every one, > this is an example of what I am getting : > > 2013-08-28 16:41:09,520 INFO org.apache.gora.mapreduce.GoraRecordWriter: Flushing the datastore after 10000 records > 2013-08-28 16:41:19,556 WARN me.prettyprint.cassandra.connection.HConnectionManager: Could not fullfill request on this host CassandraClient<crawler1:9160-2> > 2013-08-28 16:41:19,557 WARN me.prettyprint.cassandra.connection.HConnectionManager: Exception: > me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException() > at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:35) > at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264) > at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97) > at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243) > at me.prettyprint.cassandra.model.MutatorImpl.insert(MutatorImpl.java:69) > at org.apache.gora.cassandra.store.HectorUtils.insertColumn(HectorUtils.java:52) > at org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:197) > at org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:353) > at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:228) > at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:70) > at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586) > at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at org.apache.nutch.crawl.DbUpdateReducer.reduce(DbUpdateReducer.java:202) > at org.apache.nutch.crawl.DbUpdateReducer.reduce(DbUpdateReducer.java:39) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177) > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: TimedOutException() > at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20269) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) > at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:922) > at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:908) > at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246) > at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243) > at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) > at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258) > ... 20 more > > > this is happening in virtually all of my reduce tasks, and I think as a result of this, I get this for about 15% of my reduce tasks: > > Task attempt_201308151359_0006_r_000014_0 failed to report status for 1800 seconds. Killing! > > > is there anything that I can do to improve this situation? maybe increasing the timeout time some where? > > thanks, > > > -- > Kaveh Minooie > -- *Lewis*
