Hi Puneet, Have you tried appending --jars $SPARK_HOME/lib/spark-examples-*.jar to the execution command?
Ram On Thu, Jul 7, 2016 at 5:19 PM, Puneet Tripathi < puneet.tripa...@dunnhumby.com> wrote: > Guys, Please can anyone help on the issue below? > > > > Puneet > > > > *From:* Puneet Tripathi [mailto:puneet.tripa...@dunnhumby.com] > *Sent:* Thursday, July 07, 2016 12:42 PM > *To:* user@spark.apache.org > *Subject:* Spark with HBase Error - Py4JJavaError > > > > Hi, > > > > We are running Hbase in fully distributed mode. I tried to connect to > Hbase via pyspark and then write to hbase using *saveAsNewAPIHadoopDataset > *, but it failed the error says: > > > > Py4JJavaError: An error occurred while calling > z:org.apache.spark.api.python.PythonRDD.saveAsHadoopDataset. > > : java.lang.ClassNotFoundException: > org.apache.spark.examples.pythonconverters.StringToImmutableBytesWritableConverter > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > > I have been able to create pythonconverters.jar and then did below: > > > > 1. I think we have to copy this to a location on HDFS, /sparkjars/ > seems a good a directory to create as any. I think the file has to be world > readable > > 2. Set the spark_jar_hdfs_path property in Cloudera Manager e.g. > hdfs:///sparkjars > > > > It still doesn’t seem to work can someone please help me with this. > > > > Regards, > > Puneet > > dunnhumby limited is a limited company registered in England and Wales > with registered number 02388853 and VAT registered number 927 5871 83. Our > registered office is at Aurora House, 71-75 Uxbridge Road, London W5 5SL. > The contents of this message and any attachments to it are confidential and > may be legally privileged. If you have received this message in error you > should delete it from your system immediately and advise the sender. > dunnhumby may monitor and record all emails. The views expressed in this > email are those of the sender and not those of dunnhumby. > dunnhumby limited is a limited company registered in England and Wales > with registered number 02388853 and VAT registered number 927 5871 83. Our > registered office is at Aurora House, 71-75 Uxbridge Road, London W5 5SL. > The contents of this message and any attachments to it are confidential and > may be legally privileged. If you have received this message in error you > should delete it from your system immediately and advise the sender. > dunnhumby may monitor and record all emails. The views expressed in this > email are those of the sender and not those of dunnhumby. >
