[
https://issues.apache.org/jira/browse/SPARK-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-5356.
------------------------------
Resolution: Invalid
I don't think this is suitable as JIRA, even though you've marked it as a
Question. This would be good as a question on the user@ list. So far I don't
see a problem here that is related to Spark. You use a class you have not
imported.
> Write to Hbase from Spark
> -------------------------
>
> Key: SPARK-5356
> URL: https://issues.apache.org/jira/browse/SPARK-5356
> Project: Spark
> Issue Type: Question
> Components: Examples, Spark Shell
> Affects Versions: 1.1.0
> Environment: Linux
> Reporter: Ani
> Labels: hbase, spark-shell, write
>
> I am able to Read in Hbase from Spark, but I am not able to write rows in
> Hbase from Spark.
> I am on Cloudera 5.0 (Spark 1.1.0 and HBase 0.98.6) . So Far this is what I
> got.
> I have a RDD localData, how can save that to Hbase, how can I use
> saveAsHadoopDataset?
> import org.apache.hadoop.hbase.{HBaseConfiguration, HTableDescriptor}
> import org.apache.hadoop.hbase.mapreduce.TableInputFormat
> import org.apache.spark.rdd.NewHadoopRDD
> import org.apache.hadoop.hbase.io.ImmutableBytesWritable
> import org.apache.hadoop.hbase.client.Result
> import org.apache.hadoop.hbase.mapred.TableOutputFormat
> import org.apache.hadoop.mapred.JobConf
> //Create RDD
> val localData =
> sc.textFile("/home/hbase_example/antiwari/scala_code/resources/scala_load_file.txt")
> val conf = HBaseConfiguration.create()
> conf.set("hbase.zookeeper.quorum", "localhost")
> conf.set("hbase.zookeeper.property.clientPort","2181")
> val jobConfig: JobConf = new JobConf(conf, this.getClass)
> jobConfig.setOutputFormat(classOf[TableOutputFormat])
> jobConfig.set(TableOutputFormat.OUTPUT_TABLE, "spark_data")
> /*Contents of scala_load_file.txt
> 0000000001, Name01, Field1
> 0000000002, Name02, Field2
> 0000000003, Name03, Field3
> 0000000004, Name04, Field4
> /*
> I looked at many examples online including
> (http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_import.html...
> , i get the following error (may be because I am on spark 1.1.0 and this
> example is old)
> scala> def convert(triple: (Int, String, String)) = {
> | val p = new Put(Bytes.toBytes(triple._1))
> | p.add(Bytes.toBytes("cf"),
> | Bytes.toBytes("col_1"), Bytes.toBytes(triple._2))
> | p.add(Bytes.toBytes("cf"),
> | Bytes.toBytes("col_2"), Bytes.toBytes(triple._3))
> | (new ImmutableBytesWritable, p)
> | }
> <console>:18: error: not found: type Put
> val p = new Put(Bytes.toBytes(triple._1))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
