[ https://issues.apache.org/jira/browse/GORA-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-210: -------------------------------------- Assignee: Roland > thread safety: java.util.ConcurrentModificationException > -------------------------------------------------------- > > Key: GORA-210 > URL: https://issues.apache.org/jira/browse/GORA-210 > Project: Apache Gora > Issue Type: Bug > Components: storage-cassandra > Affects Versions: 0.2 > Environment: nutch 2.1 / cassandra 1.2.1 / gora-cassandra 0.2 / > gora-core 0.2.1 > running fetch with parse=true > fetcher.threads.per.queue=2 / about 1 Exception per 100k URLs fetched > nutch on a 16 core AMD Opteron 2GHz. > Cassandra on 8 core Intel Xeon 3.3 GHz > Reporter: Roland > Assignee: Roland > Priority: Critical > Labels: patch > Fix For: 0.3 > > Attachments: GORA-210.patch, GORA-210-trunk.patch > > > This is the result of debugging one of my issues described in NUTCH-1534. > I think there is a wrong assumpation about thread safety of LinkedHashMap, it > is not enough to not iterate over the buffer (which is a LinkedHashMap). > My patch fixes this error for me: > java.util.ConcurrentModificationException > at > java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394) > at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:405) > at java.util.AbstractCollection.toArray(AbstractCollection.java:141) > at > org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:200) > at > org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:65) > at > org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587) > at > org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) > at > org.apache.nutch.fetcher.FetcherReducer$FetcherThread.output(FetcherReducer.java:664) > at > org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:534) > It may not be perfect from a performance point of view... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira