Re: percentil UDAF in spark 1.2.0

: > Already fixed: https://github.com/apache/spark/pull/2802 > > > On Thu, Feb 19, 2015 at 3:17 PM, Mohnish Kodnani < > mohnish.kodn...@gmail.com> wrote: > >> Hi, >> I am trying to use percentile and getting the following error. I am using >> spark 1.2.0. Do

percentil UDAF in spark 1.2.0

Hi, I am trying to use percentile and getting the following error. I am using spark 1.2.0. Does UDAF percentile exist in that code line and do i have to do something to get this to work. java.util.NoSuchElementException: key not found: percentile at scala.collection.MapLike$class.default(M

Re: exception with json4s render

coder that seems to have been added in that version, but I'm not using >> it. I've downgraded to 3.2.10 and it seems to work. >> >> I searched through the spark repo and it looks like it's got 3.2.10 in a >> pom. I don't know the first thing about

Re: exception with json4s render

rk repo and it looks like it's got 3.2.10 in a > pom. I don't know the first thing about how dependencies are resolved but > I'm guessing it's related? > > On Wed Feb 11 2015 at 11:20:42 AM Mohnish Kodnani < > mohnish.kodn...@gmail.com> wrote: > >> I

Re: exception with json4s render

I was getting similar error after I upgraded to spark 1.2.1 from 1.1.1 Are you by any chance using json4s 3.2.11. I downgraded to 3.2.10 and that seemed to have worked. But I didnt try to spend much time debugging the issue than that. On Wed, Feb 11, 2015 at 11:13 AM, Jonathan Haddad wrote: >

Re: spark sql registerFunction with 1.2.1

5 at 3:02 PM, Mohnish Kodnani < > mohnish.kodn...@gmail.com> wrote: > >> actually i tried in spark shell , got same error and then for some reason >> i tried to back tick the "timestamp" and it worked. >> val result = sqlContext.sql("select toSeconds(`

Re: spark sql registerFunction with 1.2.1

seems sql context is supporting UDF. On Tue, Feb 10, 2015 at 2:32 PM, Michael Armbrust wrote: > The simple SQL parser doesn't yet support UDFs. Try using a HiveContext. > > On Tue, Feb 10, 2015 at 1:44 PM, Mohnish Kodnani < > mohnish.kodn...@gmail.com> wrote: > >&g

spark sql registerFunction with 1.2.1

Hi, I am trying a very simple registerFunction and it is giving me errors. I have a parquet file which I register as temp table. Then I define a UDF. def toSeconds(timestamp: Long): Long = timestamp/10 sqlContext.registerFunction("toSeconds", toSeconds _) val result = sqlContext.sql("select

Re: Spark SQL group by

Doh :) Thanks.. seems like brain freeze. On Fri, Feb 6, 2015 at 3:22 PM, Michael Armbrust wrote: > You can't use columns (timestamp) that aren't in the GROUP BY clause. > Spark 1.2+ give you a better error message for this case. > > On Fri, Feb 6, 2015 at 3:12 PM, Mo

Spark SQL group by

Hi, i am trying to issue a sql query against a parquet file and am getting errors and would like some help to figure out what is going on. The sql : select timestamp, count(rid), qi.clientname from records where timestamp > 0 group by qi.clientname I am getting the following error: *org.apache.sp

Load multiple parquet file as single RDD

Hi, I am trying to play around with Spark and Spark SQL. I have logs being stored in HDFS on a 10 minute window. Each 10 minute window could have as many as 10 files with random names of 2GB each. Now, I want to run some analysis on these files. These files are parquet files. I am trying to run Spa