Hello Rachana! I use HBase with Spark Streaming. My solution is to create a singleton with HConnection object for each JVM and refer there from foreachPartition(), creating table connection (it's cheap according to HBase documentation) for each streaming iteration. I close connections on SparkShutdownHook.
-- Kind regards/ Pozdrawiam, Wojciech Indyk http://datacentric.pl 2016-03-09 7:04 GMT+01:00 Rachana Srivastava <[email protected]>: > I am trying to integrate SparkStreaming with HBase. I am calling following > APIs to connect to HBase > > HConnection hbaseConnection = > HConnectionManager.createConnection(conf);hBaseTable = > hbaseConnection.getTable(hbaseTable); > Since I cannot get the connection and broadcast the connection each API call > to get data from HBase is very expensive. I tried using JavaHBaseContext > (JavaHBaseContext hbaseContext = new JavaHBaseContext(jsc, conf)) by using > hbase-spark library in CDH 5.5 but I cannot import the library from maven. > Has anyone been able to successfully resolve this issue.
