[
https://issues.apache.org/jira/browse/GORA-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589773#comment-13589773
]
Renato Javier Marroquín Mogrovejo edited comment on GORA-211 at 2/28/13 6:39 PM:
---------------------------------------------------------------------------------
Hi Roland,
You are totally right, I was thinking on what it is supposed to do, but I don't
think it wouldn't even make a difference, would it? I mean deep cloning the
objects in a multi-thread environment :-/ There would probably be some memory
footprint we should consider while deep cloning, I will look into this later on
tonight man.
Oh man that news about HectorClient just hit me like cold water in a cold day
): great catch though! Please try the patch you sent over the dev list, and let
us know how that works. Could you please explain to us a little bit about how
you are testing this? So we can also test it out the same way and be sure we
commit stable code (:
Thanks a ton Roland!
was (Author: renato2099):
Hi Roland,
You are totally right, I was thinking on what it is supposed to do, but I don't
think it wouldn't even make a difference, would it? I mean deep cloning the
objects in a multi-thread environment :-/
Oh man that news about HectorClient just hit me like cold water in a cold day
): great catch though! Please try the patch you sent over the dev list, and let
us know how that works. Could you please explain to us a little bit about how
you are testing this? So we can also test it out the same way and be sure we
commit stable code (:
Thanks a ton Roland!
> thread safety: java.lang.NullPointerException
> ---------------------------------------------
>
> Key: GORA-211
> URL: https://issues.apache.org/jira/browse/GORA-211
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-cassandra
> Affects Versions: 0.2
> Environment: nutch 2.1 / cassandra 1.2.1 / gora-cassandra 0.2 /
> gora-core 0.2.1
> running fetch with parse=true
> fetcher.threads.per.queue>1
> Reporter: Roland
> Priority: Critical
>
> This is the result of debugging one of my issues described in NUTCH-1534.
> example trace:
> java.lang.NullPointerException
> at
> me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243)
> at
> me.prettyprint.cassandra.model.MutatorImpl.insert(MutatorImpl.java:71)
> at
> org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:139)
> at
> org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:307)
> at
> org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:212)
> at
> org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:65)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.nutch.fetcher.FetcherReducer$FetcherThread.output(FetcherReducer.java:664)
> at
> org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:534)
> I'm suspecting CassandraStore.put() not taking enough precautions to copy all
> objects safely to it's buffer.
> {code}
> switch(type) {
> case RECORD:
> Persistent persistent = (Persistent) fieldValue;
> Persistent newRecord = persistent.newInstance(new
> StateManagerImpl());
> for (Field member: fieldSchema.getFields()) {
> newRecord.put(member.pos(), persistent.get(member.pos()));
> }
> fieldValue = newRecord;
> break;
> case MAP:
> StatefulHashMap<?, ?> map = (StatefulHashMap<?, ?>) fieldValue;
> StatefulHashMap<?, ?> newMap = new StatefulHashMap(map);
> fieldValue = newMap;
> break;
> }
> {code}
> case RECORD - do we not need to duplicate the object returned by
> "persistent.get(member.pos())":
> newRecord.put(member.pos(), persistent.get(member.pos()))
> case MAP - do we not need to duplicate all value-objects of the map?
> I had not time to write a patch or test this, so, please comment :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
