Thanks Daniel and Dmitriy for your answers, now I have a much clear idea of what type of optimization work is being done on PIG. And just another quick question, do you guys know where in the code those optimizations are? I just want to give it a closer look (:
Renato M. 2010/7/1 Dmitriy Ryaboy <dvrya...@gmail.com> > Renato, > I just want to make sure folks know -- Pig already has a number of such > optimizations. Daniel's work is aimed at making it (much) easier to write > such rules and to add a couple new ones. But some of the classic > optimizations like projection and filter push-down already exist in the > released versions of Pig. > > D > > On Thu, Jul 1, 2010 at 5:32 PM, Daniel Dai <jiany...@yahoo-inc.com> wrote: > > > Yes, they are classic logical optimization plus some Pig only > optimization. > > All these are rule based. > https://issues.apache.org/jira/browse/PIG-1319is an umbrella Jira to track > all new optimization rules. > > > > Daniel > > > > > > Renato Marroquín Mogrovejo wrote: > > > >> Hi, > >> > >> I am also interested in this logical plan optimization framework > >> functionality. You mentioned that rules are being developed, could you > >> explain a little bit more about them? Are they like the classic logical > >> optimizations (early projection, early filtering, among others)? > >> Thanks in advance. > >> > >> Renato M. > >> > >> logical plan optimization framework > >> > >> 2010/6/28 Alan Gates <ga...@yahoo-inc.com> > >> > >> > >> > >>> On Jun 28, 2010, at 12:36 AM, Swati Jain wrote: > >>> > >>> Thanks for the prompt reply. As you mentioned optimization is in its > >>> > >>> > >>>> developing stage, does it mean optimization framework is not complete > or > >>>> only rules are in developing stage? In addition to that, I would > really > >>>> appreciate if you could give a rough idea when the patch will be > >>>> available > >>>> and what functionality will it contain? > >>>> > >>>> At this point we believe the framework is complete and rules are > being > >>>> > >>>> > >>> developed. But the framework has never been used in user testing > >>> situations > >>> (alpha or beta testing) so there will be a whole round of bugs to fix > >>> once > >>> that testing is done. > >>> > >>> The current plan is to switch to this code as the actual optimizer with > >>> 0.8, which we hope to release late this year (no promises). > >>> > >>> Alan. > >>> > >>> > >>> > >> > > >
