RE: Do windowing functions require hive support?
SQLContext only implements a subset of the SQL function, not included the window function. In HiveContext it is fine though. From: Stephen Boesch [mailto:java...@gmail.com] Sent: Thursday, 19 November 2015 3:01 PM To: Michael Armbrust Cc: Jack Yang; user Subject: Re: Do windowing functions require hive support? Why is the same query (and actually i tried several variations) working against a hivecontext and not against the sql context? 2015-11-18 19:57 GMT-08:00 Michael Armbrust mailto:mich...@databricks.com>>: Yes they do. On Wed, Nov 18, 2015 at 7:49 PM, Stephen Boesch mailto:java...@gmail.com>> wrote: But to focus the attention properly: I had already tried out 1.5.2. 2015-11-18 19:46 GMT-08:00 Stephen Boesch mailto:java...@gmail.com>>: Checked out 1.6.0-SNAPSHOT 60 minutes ago 2015-11-18 19:19 GMT-08:00 Jack Yang mailto:j...@uow.edu.au>>: Which version of spark are you using? From: Stephen Boesch [mailto:java...@gmail.com<mailto:java...@gmail.com>] Sent: Thursday, 19 November 2015 2:12 PM To: user Subject: Do windowing functions require hive support? The following works against a hive table from spark sql hc.sql("select id,r from (select id, name, rank() over (order by name) as r from tt2) v where v.r >= 1 and v.r <= 12") But when using a standard sql context against a temporary table the following occurs: Exception in thread "main" java.lang.RuntimeException: [3.25] failure: ``)'' expected but `(' found rank() over (order by name) as r ^
Re: Do windowing functions require hive support?
Why is the same query (and actually i tried several variations) working against a hivecontext and not against the sql context? 2015-11-18 19:57 GMT-08:00 Michael Armbrust : > Yes they do. > > On Wed, Nov 18, 2015 at 7:49 PM, Stephen Boesch wrote: > >> But to focus the attention properly: I had already tried out 1.5.2. >> >> 2015-11-18 19:46 GMT-08:00 Stephen Boesch : >> >>> Checked out 1.6.0-SNAPSHOT 60 minutes ago >>> >>> 2015-11-18 19:19 GMT-08:00 Jack Yang : >>> >>>> Which version of spark are you using? >>>> >>>> >>>> >>>> *From:* Stephen Boesch [mailto:java...@gmail.com] >>>> *Sent:* Thursday, 19 November 2015 2:12 PM >>>> *To:* user >>>> *Subject:* Do windowing functions require hive support? >>>> >>>> >>>> >>>> >>>> >>>> The following works against a hive table from spark sql >>>> >>>> >>>> >>>> hc.sql("select id,r from (select id, name, rank() over (order by name) >>>> as r from tt2) v where v.r >= 1 and v.r <= 12") >>>> >>>> >>>> >>>> But when using a standard sql context against a temporary table the >>>> following occurs: >>>> >>>> >>>> >>>> >>>> >>>> Exception in thread "main" java.lang.RuntimeException: [3.25] >>>> >>>> failure: ``)'' expected but `(' found >>>> >>>> >>>> >>>> rank() over (order by name) as r >>>> >>>> ^ >>>> >>>> >>> >> >
Re: Do windowing functions require hive support?
Yes they do. On Wed, Nov 18, 2015 at 7:49 PM, Stephen Boesch wrote: > But to focus the attention properly: I had already tried out 1.5.2. > > 2015-11-18 19:46 GMT-08:00 Stephen Boesch : > >> Checked out 1.6.0-SNAPSHOT 60 minutes ago >> >> 2015-11-18 19:19 GMT-08:00 Jack Yang : >> >>> Which version of spark are you using? >>> >>> >>> >>> *From:* Stephen Boesch [mailto:java...@gmail.com] >>> *Sent:* Thursday, 19 November 2015 2:12 PM >>> *To:* user >>> *Subject:* Do windowing functions require hive support? >>> >>> >>> >>> >>> >>> The following works against a hive table from spark sql >>> >>> >>> >>> hc.sql("select id,r from (select id, name, rank() over (order by name) >>> as r from tt2) v where v.r >= 1 and v.r <= 12") >>> >>> >>> >>> But when using a standard sql context against a temporary table the >>> following occurs: >>> >>> >>> >>> >>> >>> Exception in thread "main" java.lang.RuntimeException: [3.25] >>> >>> failure: ``)'' expected but `(' found >>> >>> >>> >>> rank() over (order by name) as r >>> >>> ^ >>> >>> >> >
Re: Do windowing functions require hive support?
But to focus the attention properly: I had already tried out 1.5.2. 2015-11-18 19:46 GMT-08:00 Stephen Boesch : > Checked out 1.6.0-SNAPSHOT 60 minutes ago > > 2015-11-18 19:19 GMT-08:00 Jack Yang : > >> Which version of spark are you using? >> >> >> >> *From:* Stephen Boesch [mailto:java...@gmail.com] >> *Sent:* Thursday, 19 November 2015 2:12 PM >> *To:* user >> *Subject:* Do windowing functions require hive support? >> >> >> >> >> >> The following works against a hive table from spark sql >> >> >> >> hc.sql("select id,r from (select id, name, rank() over (order by name) >> as r from tt2) v where v.r >= 1 and v.r <= 12") >> >> >> >> But when using a standard sql context against a temporary table the >> following occurs: >> >> >> >> >> >> Exception in thread "main" java.lang.RuntimeException: [3.25] >> >> failure: ``)'' expected but `(' found >> >> >> >> rank() over (order by name) as r >> >> ^ >> >> >
Re: Do windowing functions require hive support?
Checked out 1.6.0-SNAPSHOT 60 minutes ago 2015-11-18 19:19 GMT-08:00 Jack Yang : > Which version of spark are you using? > > > > *From:* Stephen Boesch [mailto:java...@gmail.com] > *Sent:* Thursday, 19 November 2015 2:12 PM > *To:* user > *Subject:* Do windowing functions require hive support? > > > > > > The following works against a hive table from spark sql > > > > hc.sql("select id,r from (select id, name, rank() over (order by name) as > r from tt2) v where v.r >= 1 and v.r <= 12") > > > > But when using a standard sql context against a temporary table the > following occurs: > > > > > > Exception in thread "main" java.lang.RuntimeException: [3.25] > > failure: ``)'' expected but `(' found > > > > rank() over (order by name) as r > > ^ > >
RE: Do windowing functions require hive support?
Which version of spark are you using? From: Stephen Boesch [mailto:java...@gmail.com] Sent: Thursday, 19 November 2015 2:12 PM To: user Subject: Do windowing functions require hive support? The following works against a hive table from spark sql hc.sql("select id,r from (select id, name, rank() over (order by name) as r from tt2) v where v.r >= 1 and v.r <= 12") But when using a standard sql context against a temporary table the following occurs: Exception in thread "main" java.lang.RuntimeException: [3.25] failure: ``)'' expected but `(' found rank() over (order by name) as r ^
Do windowing functions require hive support?
The following works against a hive table from spark sql hc.sql("select id,r from (select id, name, rank() over (order by name) as r from tt2) v where v.r >= 1 and v.r <= 12") But when using a standard sql context against a temporary table the following occurs: Exception in thread "main" java.lang.RuntimeException: [3.25] failure: ``)'' expected but `(' found rank() over (order by name) as r ^