Hi Shirley, We can do Group By (without a distinct clause) in both 1 and 2 map-reduce jobs. If you "set hive.map.aggr=true", we will only have 1 map-reduce jobs.
For "Order By", as Ashish said, we already have a similar clause "Sort by" which provides orders in each of the reducers. Can you let us know what is your use case? It's very possible that we can already achieve what you want by "sort by" and "limit" clauses. Zheng On Mon, Dec 22, 2008 at 11:08 AM, Shirley Cohen <[email protected]>wrote: > Hi Jeff and Ashish, > > Thanks for your response. Basically, what I was curious about is how Hive > implements group by operations. Does it do so in one or two map reduce > stages? Also, are order by's supported in the current version? If not, when > will they be? > > I haven't had a chance to play with Hive yet, but I intend to do so pretty > soon :)) > > Shirley > > Ashish Thusoo wrote: > >> Hi Shirley, >> I think this query would give you an error currently, for two reasons: >> 1. The select list does not contain the group by column and distinct >> track group by user is indeterminate - what value of track do you want to >> report here on the group of users? >> 2. We do not have order by yet though you can sort of simulate it with a >> sort by clause and 1 reducer. >> Can you explain more in terms of what you want the SQL to achieve? >> Also the hive wiki contains a number of presentations that describe how >> SQL gets converted to Map/Reduce plans at a high level. Check them out at >> http://wiki.apache.org/hadoop/Hive/Presentations >> Ashish >> >> ------------------------------------------------------------------------ >> *From:* Jeff Hammerbacher [mailto:[email protected]] >> *Sent:* Saturday, December 20, 2008 11:35 AM >> *To:* [email protected] >> *Subject:* Re: question about mapred translation >> >> Hey Shirley, >> >> Welcome to Hive! Once you've gotten Hive up and running and have created >> the "music" table, you should be able to say "EXPLAIN <query>", where >> <query> is the query specified below (or any other query). For more detailed >> information, you can say "EXPLAIN EXTENDED <query>". >> >> The output from the EXPLAIN might be a little obtuse, so if you still have >> questions, I'm sure more knowledgeable Hive folks can give you insight into >> the planner. >> >> Regards, >> Jeff >> >> On Sat, Dec 20, 2008 at 11:48 AM, Shirley Cohen <[email protected]<mailto: >> [email protected]>> wrote: >> >> Hi, >> >> I'm just getting started with Hive. I was wondering if anyone can >> tell me how Hive would translate the following SQL query into >> MapReduce: >> >> select distinct track, count(track) >> from music >> where date_listened between '12-10-2008' and '12-11-2008' >> group by user >> order by count(track) desc >> >> How many mapred jobs would it use? How would the map and red >> functions looks like? Also, does Hive have a utility that gives >> you this information? >> >> Thanks, >> >> Shirley >> >> >> >> >> > > -- Yours, Zheng
