Yes.. Caching is set to 1000 . Number of mappers in pig is 21, don't know about number of reducers.. Probably less than that..
It does many things basically.. The logic consists of the MR job populating 2 tables, one table which has 4m rows and 1 table which has 50,000 rows.. On Tue, Sep 3, 2013 at 12:33 PM, Anoop John <[email protected]> wrote: > You are using Scan caching in ur MR java code? How many mapper and > reducers in case of pig? How is ur Java MR job written.. a bit more on > its logic pls. > > -Anoop- > > On Tue, Sep 3, 2013 at 11:33 AM, Pavan Sudheendra <[email protected] > >wrote: > > > Hi all, > > I'm doing a kind of table join across 3 tables in the MR job ( plus doing > > some computation).. It took nearly 19 hours to run with 21 mappers and 21 > > reducers.. But with pig it ran in less than 2 hours.. > > We are using HBase both as source and sink.. Is this normal? > > > > > > On Mon, Sep 2, 2013 at 7:20 PM, Adrien Mogenet <[email protected] > > >wrote: > > > > > You should have a kind of debug/explain mode in Pig, and will show you > > how > > > it does clever things to optimize its excution path. > > > > > > > > > On Mon, Sep 2, 2013 at 3:45 PM, Dhaval Shah < > [email protected] > > > >wrote: > > > > > > > Java MR code is not optimized/efficiently written while Pig is highly > > > > optimized? Can you give us more details on what exactly you are > trying > > to > > > > do and how your Java MR code is written, how many MR jobs for Java vs > > Pig > > > > and so on > > > > > > > > Sent from Yahoo! Mail on Android > > > > > > > > > > > > > > > > > -- > > > Adrien Mogenet > > > http://www.borntosegfault.com > > > > > > > > > > > -- > > Regards- > > Pavan > > > -- Regards- Pavan
