sent_topics = LOAD '/tmp/pair_titles.avro' USING AvroStorage();
STORE sent_topics INTO 'mongodb://localhost/test.pigola' USING
MongoStorage();
That works. Why is it the case that MongoStorage only works if the
intermediate processing doesn't happen? Strangeness.
On Sun, Feb 5, 2012 at 12:31 AM, Russell Jurney <russell.jur...@gmail.com>wrote:
> MongoStorage is failing for me now, on a script that was failing before.
> Is anyone else using it? The schema is [from:chararray, to:chararray,
> pairs:{null:(subject:chararray)}], which worked before.
>
> 2012-02-05 00:27:54,991 [Thread-15] INFO
> com.mongodb.hadoop.pig.MongoStorage - Store Location Config:
> Configuration: core-default.xml, core-site.xml, mapred-default.xml,
> mapred-site.xml,
> /tmp/hadoop-rjurney/mapred/local/localRunner/job_local_0001.xml For URI:
> mongodb://localhost/test.pigola
> 2012-02-05 00:27:54,993 [Thread-15] INFO
> com.mongodb.hadoop.pig.MongoStorage - OutputFormat...
> com.mongodb.hadoop.MongoOutputFormat@4eb7cd92
> 2012-02-05 00:27:55,291 [Thread-15] INFO
> com.mongodb.hadoop.pig.MongoStorage - Preparing to write to
> com.mongodb.hadoop.output.MongoRecordWriter@333ec758
> Failed to parse: <line 1, column 35> rule identifier failed predicate:
> {!input.LT(1).getText().equalsIgnoreCase("NULL")}?
> at
> org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:79)
> at
> org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:93)
> at org.apache.pig.impl.util.Utils.parseSchema(Utils.java:175)
> at org.apache.pig.impl.util.Utils.getSchemaFromString(Utils.java:166)
> at
> com.mongodb.hadoop.pig.MongoStorage.prepareToWrite(MongoStorage.java:186)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
> 2012-02-05 00:27:55,320 [Thread-15] INFO
> com.mongodb.hadoop.pig.MongoStorage - Stored Schema: [from:chararray,
> to:chararray, pairs:{null:(subject:chararray)}]
> 2012-02-05 00:27:55,323 [Thread-15] WARN
> org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:464)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:427)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:407)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:261)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)
> Caused by: java.lang.NullPointerException
> at com.mongodb.hadoop.pig.MongoStorage.putNext(MongoStorage.java:68)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:508)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:462)
> ... 7 more
>
>
> --
> Russell Jurney
> twitter.com/rjurney
> russell.jur...@gmail.com
> datasyndrome.com
>
--
Russell Jurney
twitter.com/rjurney
russell.jur...@gmail.com
datasyndrome.com
