Re: spark1.3.1 using mysql error!

l or start-thriftserver and beeline Things would be better once SPARK-6966 is merged into 1.4.0 when you can use 1. use the --jars parameter for spark-shell, spark-sql, etc or 2. sc.addJar to add the driver after starting spark-shell. Good Luck, Anand Mohan -- View this message in context: http://ap

OutOfMemory error in Spark Core

We have our Analytics App built on Spark 1.1 Core, Parquet, Avro and Spray. We are using Kryo serializer for the Avro objects read from Parquet and we are using our custom Kryo registrator (along the lines of ADAM

Re: Spark SQL Percentile UDAF

Filed https://issues.apache.org/jira/browse/SPARK-3891 Thanks, Anand Mohan On Thu, Oct 9, 2014 at 7:13 PM, Michael Armbrust wrote: > Please file a JIRA:https://issues.apache.org/jira/browse/SPARK/ > <https://www.google.com/url?q=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPAR

Spark SQL Percentile UDAF

ntile(turnaroundtime,0.75) is working. (and so this issue is not of a high priority as there is this workaround for us) Thanks, Anand Mohan 0: jdbc:hive2://dev-uuppala.sfohi.philips.com> select name, percentile(turnaroundtime,array(0,0.25,0.5,0.75,1)) from exam group by name

Re: Spark SQL HiveContext Projection Pushdown

Thanks a lot. It is working like a charm now. Even predicate push down on optional fields is working. Awesome!!! Is there any plan to support windowing queries? I know that Shark supported it in its last release and expected it to be already included. Best regards, Anand Mohan On Oct 8, 2014 11

Spark SQL HiveContext Projection Pushdown

doesnt seem to work and it ends up reading the whole Parquet data for each query.(which slows down a lot) Please see attached the screenshot of this. Hive itself doesnt seem to have any issues with the projection pushdown. So this is weird. Is this due to any configuration problem? Thanks in advanc