No Ravi, I am not running any MR job. Also, my configuration files are not big.
On Wed, Aug 7, 2013 at 11:12 PM, Ravi Prakash <ravi...@ymail.com> wrote: > I believe https://issues.apache.org/jira/browse/MAPREDUCE-5399 causes > performance degradation in cases where there are a lot of reducers. I can > imagine it causing degradation if the configuration files are super big / > some other weird cases. > > > ------------------------------ > *From:* Krishna Kishore Bonagiri <write2kish...@gmail.com> > *To:* user@hadoop.apache.org > *Sent:* Wednesday, August 7, 2013 10:03 AM > *Subject:* Re: Extra start-up overhead with hadoop-2.1.0-beta > > Hi Omkar, > > Can you please see if you can answer my question with this info or if you > need anything else from me? > > Also, does resource localization improve or impact any performance? > > Thanks, > Kishore > > > On Thu, Aug 1, 2013 at 11:20 PM, Omkar Joshi <ojo...@hortonworks.com>wrote: > > How are you making these measurements can you elaborate more? Is it on a > best case basis or on an average or worst case? How many resources are you > sending it for localization? were the sizes and number of these resources > consistent across tests? Were these resources public/private/application > specific? Apart from this is the other load on node manager same? is the > load on hdfs same? did you see any network bottleneck? > > More information will help a lot. > > > Thanks, > Omkar Joshi > *Hortonworks Inc.* <http://www.hortonworks.com/> > > > On Thu, Aug 1, 2013 at 2:19 AM, Krishna Kishore Bonagiri < > write2kish...@gmail.com> wrote: > > Hi, > Please share with me if you anyone has an answer or clues to my question > regarding the start up performance. > > Also, one more thing I have observed today is the time taken to run a > command on a container went up by more than a second in this latest version. > > When using 2.0.4-alpha, it used to take 0.3 to 0.5 seconds from the point > I call startContainer() to the point the command is started on the > container. > > where as > > When using 2.1.0-beta, it is taking around 1.5 seconds from the point it > came to the call back onContainerStarted() to the point the command is seen > started running on the container. > > Thanks, > Kishore > > > On Thu, Jul 25, 2013 at 8:38 PM, Krishna Kishore Bonagiri < > write2kish...@gmail.com> wrote: > > Hi, > > I have been using the hadoop-2.0.1-beta release candidate and observed > that it is slower in running my simple application that runs on 2 > containers. I have tried to find out which parts of it is really having > this extra overhead(compared to hadoop-2.0.4-alpha), and here is what I > found that. > > 1) From the point my Client has submitted the Application Master to RM, it > is taking 2 seconds extra > 2) From the point my container request are set up by Application Master, > till the containers are allocated, it is taking 2 seconds extra > > Is this overhead expected with the changes that went into the new version? > Or is there to improve it by changing something in configurations or so? > > Thanks, > Kishore > > > > > > >
