I got the following errors using AccumuloOutputFormat from Spark:
14/07/18 18:32:21 ERROR Executor: Exception in task ID 0
java.io.IOException: java.lang.IllegalStateException: Instance has not been
configured for AccumuloOutputFormat
at
org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.getRecordWriter(AccumuloOutputFormat.java:558)
at org.apache.spark.rdd.PairRDDFunctions.org
$apache$spark$rdd$PairRDDFunctions$$writeShard$1(PairRDDFunctions.scala:713)
at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
at
org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:731)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
at org.apache.spark.scheduler.Task.run(Task.scala:51)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:183)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
However, AccumuloOutputFormat has already been set:
val inst = new MockInstance("mock")
val conn = inst.getConnector("test", new PasswordToken(""))
conn.tableOperations().create(Conf.getString("accumulo.entity.table"))
AccumuloOutputFormat.setMockInstance(job, inst.getInstanceName)
AccumuloOutputFormat.setConnectorInfo(job, "test", new
PasswordToken(""))
AccumuloOutputFormat.setCreateTables(job, true)
AccumuloOutputFormat.setDefaultTableName(job,
Conf.getString("accumulo.entity.table"))
Any idea why it happened?
Cheers,
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
