Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods

[Mich Talebzadeh]

Hi,

Spark version 2.4.3
Hbase 1.2.7

Data is stored in Hbase as Json. example of a row shown below
[image: image.png]
I am trying to read this table in Spark Scala

import org.apache.spark.sql.{SQLContext, _}
import org.apache.spark.sql.execution.datasources.hbase._
import org.apache.spark.{SparkConf, SparkContext}
import spark.sqlContext.implicits._
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.jackson.Serialization.{read => JsonRead}
import org.json4s.jackson.Serialization.{read, write}
def catalog = s"""{
     | "table":{"namespace":"trading", "name":"MARKETDATAHBASEBATCH",
     | "rowkey":"key",
     | "columns":{
     | "rowkey":{"cf":"rowkey", "col":"key", "type":"string"},
     |     |"ticker":{"cf":"PRICE_INFO", "col":"ticker", "type":"string"},
     |     |"timeissued":{"cf":"PRICE_INFO", "col":"timeissued",
"type":"string"},
     |     |"price":{"cf":"PRICE_INFO", "col":"price", "type":"string"}
     |     |}
     | |}""".stripMargin
def withCatalog(cat: String): DataFrame = {
           spark.sqlContext
           .read
           .options(Map(HBaseTableCatalog.tableCatalog->cat))
           .format("org.apache.spark.sql.execution.datasources.hbase")
           .load()
        }
val df = withCatalog(catalog)


However, I am getting this error

Spark session available as 'spark'.
java.lang.NoSuchMethodError:
org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;
  at
org.apache.spark.sql.execution.datasources.hbase.HBaseTableCatalog$.apply(HBaseTableCatalog.scala:257)
  at
org.apache.spark.sql.execution.datasources.hbase.HBaseRelation.<init>(HBaseRelation.scala:80)
  at
org.apache.spark.sql.execution.datasources.hbase.DefaultSource.createRelation(HBaseRelation.scala:51)
  at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
  at
org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
  at withCatalog(testme.scala:49)
  ... 65 elided

I have Googled it but with little luck!

Thanks,
Mich

http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.