RE: Writing to HBase

2013-12-12 Thread Benjamin Kim 
Hi Philip,
I got this bit of code to work in the spark-shell using scala against our dev 
hbase cluster.
-bash-4.1$export 
SPARK_CLASSPATH=$SPARK_CLASSPATH:/opt/cloudera/parcels/CDH/lib/hbase/hbase.jar:/opt/cloudera/parcels/CDH/lib/hbase/conf:/opt/cloudera/parcels/CDH/lib/hadoop/conf
-bash-4.1$./spark-shellscala>import 
org.apache.hadoop.hbase.HBaseConfigurationscala>import 
org.apache.hadoop.hbase.client._scala>import 
org.apache.hadoop.hbase.util.Bytesscala>val conf = 
HBaseConfiguration.create()scala>val table = new HTable(conf, 
"my_items")scala>val p = new 
Put(Bytes.toBytes("strawberry-fruit"))scala>p.add(Bytes.toBytes("item"),Bytes.toBytes("item"),Bytes.toBytes("strawberry"))scala>p.add(Bytes.toBytes("item"),Bytes.toBytes("category"),Bytes.toBytes("fruit"))scala>p.add(Bytes.toBytes("item"),Bytes.toBytes("price"),Bytes.toBytes("0.35"))scala>table.put(p)
It put the new row "strawberry-fruit" into an hbase table.
Sorry, but I have another newbie question. How do I add those CLASSPATH 
dependencies when I want to compile a streaming jar in sbt so that the hbase 
configs are automatically used?
Thanks,Ben
Date: Thu, 5 Dec 2013 10:24:02 -0700
From: philip.og...@oracle.com
To: user@spark.incubator.apache.org
Subject: Re: Writing to HBase


  

  
  
Here's a good place to start:




http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201311.mbox/%3ccacyzca3askwd-tujhqi1805bn7sctguaoruhd5xtxcsul1a...@mail.gmail.com%3E





On 12/5/2013 10:18 AM, Benjamin Kim
  wrote:



  
  Does anyone have an example or some sort of
starting point code when writing from Spark Streaming into
HBase?



We currently stream ad server event log data using Flume-NG
  to tail log entries, collect them, and put them directly into
  a HBase table. We would like to do the same with Spark
  Streaming. But, we would like to do the data massaging and
  simple data analysis before. This will cut down the steps in
  prepping data and the number of tables for our data scientists
  and real-time feedback systems.



Thanks,
Ben

Re: Writing to HBase

2013-12-05 Thread Philip Ogren 

Here's a good place to start:

http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201311.mbox/%3ccacyzca3askwd-tujhqi1805bn7sctguaoruhd5xtxcsul1a...@mail.gmail.com%3E


On 12/5/2013 10:18 AM, Benjamin Kim wrote:
Does anyone have an example or some sort of starting point code when 
writing from Spark Streaming into HBase?


We currently stream ad server event log data using Flume-NG to tail 
log entries, collect them, and put them directly into a HBase table. 
We would like to do the same with Spark Streaming. But, we would like 
to do the data massaging and simple data analysis before. This will 
cut down the steps in prepping data and the number of tables for our 
data scientists and real-time feedback systems.


Thanks,
Ben

Writing to HBase

2013-12-05 Thread Benjamin Kim 
Does anyone have an example or some sort of starting point code when writing 
from Spark Streaming into HBase?
We currently stream ad server event log data using Flume-NG to tail log 
entries, collect them, and put them directly into a HBase table. We would like 
to do the same with Spark Streaming. But, we would like to do the data 
massaging and simple data analysis before. This will cut down the steps in 
prepping data and the number of tables for our data scientists and real-time 
feedback systems.
Thanks,Ben