Connection object is not serialisable. You need to implement a getorcreate
function which would run on each executors to create hbase connection
locally.
On 22 Sep 2016 08:34, "KhajaAsmath Mohammed" <mdkhajaasm...@gmail.com>
wrote:
> Hello Everyone,
>
> I am running spark application to push data from kafka. I am able to get
> hbase kerberos connection successfully outside of functon before calling
> foreachrdd on Dstream.
>
> Job fails inside foreachrdd stating that hbaseconnection object is not
> serialized. could you please let me now how toresolve this.
>
> @transient val hbaseConnection=hBaseEntityManager.getConnection()
>
> appEventDStream.foreachRDD(rdd => {
> if (!rdd.isEmpty()) {
> rdd.foreach { entity =>
> {
>
> generatePut(hBaseEntityManager,hbaseConnection,entity.getClass.getSimpleName,entity.asInstanceOf[DataPoint])
>
> }
>
> }
>
>
> Error is thrown exactly at connection object inside foreachRdd saying it is
> not serialize. could anyone provide solution for it
>
> Asmath
>
>
