We will run a combiner in map phase soon right? Potentially I can just take advantage of that.
On Tue, Jul 1, 2008 at 7:35 AM, Olga Natkovich <[EMAIL PROTECTED]> wrote: > Pi, > > This is a great start! > > Looked a little but into proposed implementation - seems like we should > be able to do a lot of hash-based preaggregation in the map and then > sort based preaggregation in the combiner. > > Olga > > > -----Original Message----- > > From: Chris Olston [mailto:[EMAIL PROTECTED] > > Sent: Monday, June 30, 2008 8:49 AM > > To: [email protected] > > Subject: Re: OLAP ? > > > > Awesome idea, Pi! > > > > Bottom line: sooner or later somebody is going to package up > > a "olap- on-mapreduce" solution. Pig seems like a very good > > vehicle for it. > > > > -Chris > > > > > > On Jun 30, 2008, at 8:10 AM, pi song wrote: > > > > > Has anyone ever looked at OLAP Cube processing on Pig or > > MapReduce? > > > This feature should be beneficial for normal businesses > > (probably not > > > web scale). I find it pretty simple provided that we can work on > > > relational abstraction (but might not scale well on really big > > > datasets). I have done a sketch on how I think it should look like > > > here http://wiki.apache.org/pig/PigOLAPSketch . Welcome for all > > > comments/suggestions. > > > > > > Pi > > > > -- > > Christopher Olston, Ph.D. > > Sr. Research Scientist > > Yahoo! Research > > > > > > >
