Re: optimizer hints in Pig

2009-11-16 Thread Alan Gates
In general I think optimizer hints fit well with Pig's approach to data processing, as expressed in our philosophic statement that Pigs are domestic animals (see http://hadoop.apache.org/pig/ philosophy.html ). At least in the examples you give, I don't see 'with' as binding. The user is

optimizer hints in Pig

2009-11-14 Thread Ashutosh Chauhan
Hi All, We would like to know what Pig devs feel about optimizer hints. Traditionally, optimizer hints have been received with mixed reactions in RDBMS world.  Oracle provides lots of knobs[1][2] to turn and tune, while postgres[3][4] have tried to stay away from them. Mysql have few of them

Re: optimizer hints in Pig

2009-11-14 Thread Mridul Muralidharan
This should be interesting to pursue. For non-trivial plans, it can be used to model how join's are to be optimized for example - instead of having users specify it, infer based on the hints from various stages. Similarly, an oft-repeated request is making pig infer the value of PARALLEL -