Hi Doug, Yes, that's why i had set that property as false in my mapred-site.xml. But, to my surprise i didnt know that setting that property would be useless for Hadoop jobs unless the mapred-site.xml is in classpath. The idea of client side property is a little confusing to me at present since there is no proper nomenclature for client side properties at present. Thanks for your reply.
~Anil On Fri, Mar 30, 2012 at 3:26 PM, Doug Meil <[email protected]>wrote: > > Speculative execution is on by default in Hadoop. One of the Performance > recommendations in the Hbase RefGuide is to turn it off. > > > > > > On 3/30/12 6:12 PM, "Jean-Daniel Cryans" <[email protected]> wrote: > > >Well that's not an HBase configuration, that's Hadoop. I'm not sure if > >this is listed anywhere, maybe in the book. > > > >BTW usually HBase has a "client" somewhere in the same to indicate > >it's client side. > > > >J-D > > > >On Fri, Mar 30, 2012 at 3:08 PM, anil gupta <[email protected]> wrote: > >> Thanks for the quick reply, Jean. Is there any link where i can find the > >> name of all client-side configuration for HBase? > >> > >> ~Anil > >> > >> On Fri, Mar 30, 2012 at 3:01 PM, Jean-Daniel Cryans > >><[email protected]>wrote: > >> > >>> This is a client-side configuration so if your mapred-site.xml is > >>> _not_ on your classpath when you start the bulk load, it's not going > >>> to pick it up. So either have that file on your classpath, or put it > >>> in whatever other configuration file you have. > >>> > >>> J-D > >>> > >>> On Fri, Mar 30, 2012 at 2:52 PM, anil gupta <[email protected]> > >>>wrote: > >>> > Hi All, > >>> > > >>> > I am using cdh3u2. I ran HBase bulk loading with property > >>> > "mapred.reduce.tasks.speculative.execution" set to "false" in > >>> > mapred-site.xml. Still, i can see 6 killed task in Bulk Loading job > >>>and > >>> > after short analysis i realized that these jobs are killed because > >>> another > >>> > worker node completed the task, hence it means that speculative > >>>execution > >>> > is still on. Why the HBase Bulk loader is doing speculative execution > >>> when > >>> > i have set it to false in mapred-site.xml? Please let me know if i am > >>> > missing something over here. > >>> > > >>> > -- > >>> > Thanks & Regards, > >>> > Anil Gupta > >>> > >> > >> > >> > >> -- > >> Thanks & Regards, > >> Anil Gupta > > > > > -- Thanks & Regards, Anil Gupta
