For whatever reason, the Configuration that was passed to the getRecordReader method is different than the one you called the setMockInstance on.
I'm no spark expert, but I've experienced confusion in other frameworks like Pig and Hive where the framework isn't always returning you the same Configuration object you had before. You may have to invoke these AccumuloOutputFormat methods more than once. On Jul 18, 2014 6:41 AM, "Jianshi Huang" <[email protected]> wrote: > I got the following errors using AccumuloOutputFormat from Spark: > > 14/07/18 18:32:21 ERROR Executor: Exception in task ID 0 > java.io.IOException: java.lang.IllegalStateException: Instance has not > been configured for AccumuloOutputFormat > at > org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.getRecordWriter(AccumuloOutputFormat.java:558) > at org.apache.spark.rdd.PairRDDFunctions.org > $apache$spark$rdd$PairRDDFunctions$$writeShard$1(PairRDDFunctions.scala:713) > at > org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731) > at > org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731) > at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111) > at org.apache.spark.scheduler.Task.run(Task.scala:51) > at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > However, AccumuloOutputFormat has already been set: > > val inst = new MockInstance("mock") > val conn = inst.getConnector("test", new PasswordToken("")) > conn.tableOperations().create(Conf.getString("accumulo.entity.table")) > > AccumuloOutputFormat.setMockInstance(job, inst.getInstanceName) > AccumuloOutputFormat.setConnectorInfo(job, "test", new > PasswordToken("")) > AccumuloOutputFormat.setCreateTables(job, true) > AccumuloOutputFormat.setDefaultTableName(job, > Conf.getString("accumulo.entity.table")) > > > Any idea why it happened? > > Cheers, > -- > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ >
