Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Filed https://issues.apache.org/jira/browse/SPARK-6708 to track this. Cheng On 4/4/15 10:21 PM, Cheng Lian wrote: I think this is a bug of Spark SQL dates back to at least 1.1.0. The json_tuple function is implemented as org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple. The ClassNo

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

I think this is a bug of Spark SQL dates back to at least 1.1.0. The json_tuple function is implemented as org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple. The ClassNotFoundException should complain with the class name rather than the UDTF function name. The problematic line shoul

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Hi Deepujain, I did include the jar file, I believe it is hive-exe.jar, through the --jars option: ./bin/spark-shell --master spark://radtech.io:7077 --total-executor-cores 2 --driver-class-path /usr/local/spark/lib/mysql-connector-java-5.1.34-bin.jar --jars /opt/apache-hive-0.13.1-bin/lib/hive-e

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

I think you need to include the jar file through --jars option that contains the hive definition (code) of UDF json_tuple. That should solve your problem. On Fri, Apr 3, 2015 at 3:57 PM, Todd Nist wrote: > I placed it there. It was downloaded from MySql site. > > On Fri, Apr 3, 2015 at 6:25 AM,

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

I placed it there. It was downloaded from MySql site. On Fri, Apr 3, 2015 at 6:25 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > Akhil > you mentioned /usr/local/spark/lib/mysql-connector-java-5.1.34-bin.jar . > how come you got this lib into spark/lib folder. > 1) did you place it there ? > 2) What is download lo

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Copy pasted his command in the same thread. Thanks Best Regards On Fri, Apr 3, 2015 at 3:55 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > Akhil > you mentioned /usr/local/spark/lib/mysql-connector-java-5.1.34-bin.jar . > how come you got this lib into spark/lib folder. > 1) did you place it there ? > 2) What is d

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Akhil you mentioned /usr/local/spark/lib/mysql-connector-java-5.1.34-bin.jar . how come you got this lib into spark/lib folder. 1) did you place it there ? 2) What is download location ? On Fri, Apr 3, 2015 at 3:42 PM, Todd Nist wrote: > Started the spark shell with the one jar from hive sugges

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Started the spark shell with the one jar from hive suggested: ./bin/spark-shell --master spark://radtech.io:7077 --total-executor-cores 2 --driver-class-path /usr/local/spark/lib/mysql-connector-java-5.1.34-bin.jar --jars /opt/apache-hive-0.13.1-bin/lib/hive-exec-0.13.1.jar Results in the same er

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Can you try building Spark with hive support? Before that try to run the following: ./bin/spark-shell --master spark://radtech.io:7077 --total-executor-cores 2 --driver

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Hi Akhil, This is for version 1.2.1. Well the other thread that you reference was me attempting it in 1.3.0 to see if the issue was related to 1.2.1. I did not build Spark but used the version from the Spark download site for 1.2.1 Pre Built for Hadoop 2.4 or Later. Since I get the error in bot

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

How did you build spark? which version of spark are you having? Doesn't this thread already explains it? https://www.mail-archive.com/user@spark.apache.org/msg25505.html Thanks Best Regards On Thu, Apr 2, 2015 at 11:10 PM, Todd Nist wrote: > Hi Akhil, > > Tried your suggestion to no avail. I a

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Hi Akhil, Tried your suggestion to no avail. I actually to not see and "jackson" or "json serde" jars in the $HIVE/lib directory. This is hive 0.13.1 and spark 1.2.1 Here is what I did: I have added the lib folder to the –jars option when starting the spark-shell, but the job fails. The hive-s

Re: Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

Try adding all the jars in your $HIVE/lib directory. If you want the specific jar, you could look fr jackson or json serde in it. Thanks Best Regards On Thu, Apr 2, 2015 at 12:49 AM, Todd Nist wrote: > I have a feeling I’m missing a Jar that provides the support or could this > may be related t

Spark Sql - Missing Jar ? json_tuple NoClassDefFoundError

I have a feeling I’m missing a Jar that provides the support or could this may be related to https://issues.apache.org/jira/browse/SPARK-5792. If it is a Jar where would I find that ? I would have thought in the $HIVE/lib folder, but not sure which jar contains it. Error: Create Metric Temporary