No that would even be worse.
A single BatchInserter and every graphdb-store that is currently written to by
a batch inserter MUST be accessed from only a single single threaded
environment.
Please use the normal EmbeddedGraphDbService for your multi-threaded MR jobs.
Cheers
Michael
Am 17.06.2011 um 23:38 schrieb sulabh choudhury:
> Are you referring that in a M/R environment each Map (or Reduce) process will
> try to have its own instance of batchInserter and hence it would fail ?
>
> WHen I say "local" I mean that the code works fine when I just use the M/R
> api but fails when I try to run in distributed mode.
>
> On Fri, Jun 17, 2011 at 2:25 PM, Michael Hunger
> <michael.hun...@neotechnology.com> wrote:
> Hi Sulabh,
>
> what do you mean by 'local' mode?
>
> The batch inserter can only be used in a single threaded environment. You
> shouldn't use it in a concurrent env as it will fail unpredictably.
>
> Please use the EmbeddedGraphDatabase instead.
>
> Michael
>
> Am 17.06.2011 um 23:20 schrieb sulabh choudhury:
>
>> Well as I mentioned the code does not fail anywhere, it runs it full course
>> and just skips the writing to the graph part.
>> I have just one graph and I pass just 1 instance of the batchInserter to
>> the map function.
>>
>> My code is in Scala, sample code attached below
>>
>>
>> class ExportReducer extends Reducer[Text,MapWritable,LongWritable,Text]{
>>
>> type Context = org.apache.hadoop.mapreduce.Reducer[Text, MapWritable,
>> LongWritable, Text]#Context
>>
>> @throws(classOf[Exception])
>> override def reduce(key: Text, value: java.lang.Iterable[MapWritable],
>> context: Context) {
>>
>> var keys: Array[String] = key.toString.split(":")
>> var uri1 = "first" + keys(0)
>> var uri2 = "last" + keys(1)
>> ExportReducerObject.propertiesUID.put("ID",uri1);
>> var node1 =
>> ExportReducerObject.batchInserter.createNode(ExportReducerObject.propertiesUID);
>>
>> ExportReducerObject.indexService.add(node1,ExportReducerObject.propertiesUID)
>> ExportReducerObject.propertiesCID.put("ID",uri2);
>> var node2 =
>> ExportReducerObject.batchInserter.createNode(ExportReducerObject.propertiesCID);
>>
>> ExportReducerObject.indexService.add(node2,ExportReducerObject.propertiesCID);
>>
>> ExportReducerObject.propertiesEdges.put("fullName","1.0");
>>
>> ExportReducerObject.batchInserter.createRelationship(node1,node2,DynamicRelationshipType.withName("fullName"),ExportReducerObject.propertiesEdges)
>>
>> }
>>
>> My graph properties are defined as below :-
>> val batchInserter = new BatchInserterImpl("graph",
>> BatchInserterImpl.loadProperties("neo4j.props"))
>> val indexProvider = new LuceneBatchInserterIndexProvider(batchInserter)
>> val indexService =
>> indexProvider.nodeIndex("ID",MapUtil.stringMap("type","exact"))
>>
>>
>> Mind it that the code works perfectly( writes to the graph) when running in
>> local mode.
>>
>> On Fri, Jun 17, 2011 at 11:32 AM, sulabh choudhury <sula...@gmail.com> wrote:
>> I am trying to write MapReduce job to do Neo4j Batchinserters.
>> It works fine when I just run it like a java file(runs in local mode) and
>> does the insert, but when I try to run it in the distributed mode it does
>> not write to the graph.
>> Is it issue related to permissions?
>> I have no clue where to look.
>>
>>
>>
>> --
>> --
>> Thanks and Regards,
>> Sulabh Choudhury
>>
>
>
>
>
> --
> --
> Thanks and Regards,
> Sulabh Choudhury
>
_______________________________________________
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
