Re: confusing ScalaReflectionException with DataFrames in 1.4

2015-06-18 Thread Michael Armbrust
I saw another report so I filed it already: Filed as: https://issues.apache.org/jira/browse/SPARK-8470 On Thu, Jun 18, 2015 at 4:07 PM, Chad Urso McDaniel cha...@gmail.com wrote: We're using the normal command line: --- bin/spark-submit --properties-file ./spark-submit.conf --class

Re: confusing ScalaReflectionException with DataFrames in 1.4

2015-06-18 Thread Michael Armbrust
How are you adding com.rr.data.Visit to spark? With --jars? It is possible we are using the wrong classloader. Could you open a JIRA? On Thu, Jun 18, 2015 at 2:56 PM, Chad Urso McDaniel cha...@gmail.com wrote: We are seeing class exceptions when converting to a DataFrame. Anyone out there

confusing ScalaReflectionException with DataFrames in 1.4

2015-06-18 Thread Chad Urso McDaniel
We are seeing class exceptions when converting to a DataFrame. Anyone out there with some suggestions on what is going on? Our original intention was to use a HiveContext to write ORC and we say the error there and have narrowed it down. This is an example of our code: --- def

Re: confusing ScalaReflectionException with DataFrames in 1.4

2015-06-18 Thread Chad Urso McDaniel
We're using the normal command line: --- bin/spark-submit --properties-file ./spark-submit.conf --class com.rr.data.visits.VisitSequencerRunner ./mvt-master-SNAPSHOT-jar-with-dependencies.jar --- Our jar contains both com.rr.data.visits.orc.OrcReadWrite (which you can see in the stack trace) and