+common-user@hadoop.apache.org Hi Harsh,
Thanks for the information. Is there any way to differentiate between a client side property and server-side property?or a Document which enlists whether a property is server or client-side? Many times i have to speculate over this and try out test runs. Thanks, Anil On Fri, Mar 30, 2012 at 9:54 PM, Harsh J <ha...@cloudera.com> wrote: > Anil, > > You can also disable speculative execution on a per-job basis. See > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setMapSpeculativeExecution(boolean) > (Which is why it is called a client-sided property - it applies > per-job). > > If HBase strongly recommends turning it off, HBase should also, by > default, turn it off for its own offered jobs? > > On Sat, Mar 31, 2012 at 4:02 AM, anil gupta <anilg...@buffalo.edu> wrote: > > Hi Doug, > > > > Yes, that's why i had set that property as false in my mapred-site.xml. > > But, to my surprise i didnt know that setting that property would be > > useless for Hadoop jobs unless the mapred-site.xml is in classpath. The > > idea of client side property is a little confusing to me at present since > > there is no proper nomenclature for client side properties at present. > > Thanks for your reply. > > > > ~Anil > > > > On Fri, Mar 30, 2012 at 3:26 PM, Doug Meil < > doug.m...@explorysmedical.com>wrote: > > > >> > >> Speculative execution is on by default in Hadoop. One of the > Performance > >> recommendations in the Hbase RefGuide is to turn it off. > >> > >> > >> > >> > >> > >> On 3/30/12 6:12 PM, "Jean-Daniel Cryans" <jdcry...@apache.org> wrote: > >> > >> >Well that's not an HBase configuration, that's Hadoop. I'm not sure if > >> >this is listed anywhere, maybe in the book. > >> > > >> >BTW usually HBase has a "client" somewhere in the same to indicate > >> >it's client side. > >> > > >> >J-D > >> > > >> >On Fri, Mar 30, 2012 at 3:08 PM, anil gupta <anilg...@buffalo.edu> > wrote: > >> >> Thanks for the quick reply, Jean. Is there any link where i can find > the > >> >> name of all client-side configuration for HBase? > >> >> > >> >> ~Anil > >> >> > >> >> On Fri, Mar 30, 2012 at 3:01 PM, Jean-Daniel Cryans > >> >><jdcry...@apache.org>wrote: > >> >> > >> >>> This is a client-side configuration so if your mapred-site.xml is > >> >>> _not_ on your classpath when you start the bulk load, it's not going > >> >>> to pick it up. So either have that file on your classpath, or put it > >> >>> in whatever other configuration file you have. > >> >>> > >> >>> J-D > >> >>> > >> >>> On Fri, Mar 30, 2012 at 2:52 PM, anil gupta <anilgupt...@gmail.com> > >> >>>wrote: > >> >>> > Hi All, > >> >>> > > >> >>> > I am using cdh3u2. I ran HBase bulk loading with property > >> >>> > "mapred.reduce.tasks.speculative.execution" set to "false" in > >> >>> > mapred-site.xml. Still, i can see 6 killed task in Bulk Loading > job > >> >>>and > >> >>> > after short analysis i realized that these jobs are killed because > >> >>> another > >> >>> > worker node completed the task, hence it means that speculative > >> >>>execution > >> >>> > is still on. Why the HBase Bulk loader is doing speculative > execution > >> >>> when > >> >>> > i have set it to false in mapred-site.xml? Please let me know if > i am > >> >>> > missing something over here. > >> >>> > > >> >>> > -- > >> >>> > Thanks & Regards, > >> >>> > Anil Gupta > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Thanks & Regards, > >> >> Anil Gupta > >> > > >> > >> > >> > > > > > > -- > > Thanks & Regards, > > Anil Gupta > > > > -- > Harsh J > -- Thanks & Regards, Anil Gupta
