What is causing my mappers to execute so damn slow?
Hi everyone, I posted this question many time before and i've given full details on stackoverflow.. http://stackoverflow.com/q/19056712/938959 Please i need someone to guide me in the right direction here. Help much appreciated! -- Regards- Pavan
Re: What is causing my mappers to execute so damn slow?
Your details are missing important bits like you configurations, Hadoop/HBase versions, etc. Doing those random reads inside your MR job, especially if they are reading cold data, will indeed make it slower. Just to get an idea, if you skip doing the Gets, how fast does it became? J-D On Fri, Sep 27, 2013 at 10:33 AM, Pavan Sudheendra pavan0...@gmail.comwrote: Hi everyone, I posted this question many time before and i've given full details on stackoverflow.. http://stackoverflow.com/q/19056712/938959 Please i need someone to guide me in the right direction here. Help much appreciated! -- Regards- Pavan
Re: What is causing my mappers to execute so damn slow?
Hi Jean, HBase 0.94.6 and Hadoop 1.2.1 Cloudera Distributions.. I infact tried that out, in place of doing the get operations , i created stub data and returned that instead.. It was practically at the same speed. Nothing changed.. After 20 mins or so when i check the job status.. It hardly reached 1,000,000 rows.. On Fri, Sep 27, 2013 at 11:12 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: Your details are missing important bits like you configurations, Hadoop/HBase versions, etc. Doing those random reads inside your MR job, especially if they are reading cold data, will indeed make it slower. Just to get an idea, if you skip doing the Gets, how fast does it became? J-D On Fri, Sep 27, 2013 at 10:33 AM, Pavan Sudheendra pavan0...@gmail.com wrote: Hi everyone, I posted this question many time before and i've given full details on stackoverflow.. http://stackoverflow.com/q/19056712/938959 Please i need someone to guide me in the right direction here. Help much appreciated! -- Regards- Pavan -- Regards- Pavan
Re: What is causing my mappers to execute so damn slow?
I don't think there's a CDH that includes Hadoop 1.2.1 So either your code is doing something slow or it's the reading itself. For the latter, make sure you go through http://hbase.apache.org/book.html#perf.reading and we also recently had this thread on the list were you can see some live performance debugging http://www.mail-archive.com/user@hbase.apache.org/msg27174.html. For example, make sure you're not running on the local job tracker. J-D On Fri, Sep 27, 2013 at 11:07 AM, Pavan Sudheendra pavan0...@gmail.comwrote: Hi Jean, HBase 0.94.6 and Hadoop 1.2.1 Cloudera Distributions.. I infact tried that out, in place of doing the get operations , i created stub data and returned that instead.. It was practically at the same speed. Nothing changed.. After 20 mins or so when i check the job status.. It hardly reached 1,000,000 rows.. On Fri, Sep 27, 2013 at 11:12 PM, Jean-Daniel Cryans jdcry...@apache.org wrote: Your details are missing important bits like you configurations, Hadoop/HBase versions, etc. Doing those random reads inside your MR job, especially if they are reading cold data, will indeed make it slower. Just to get an idea, if you skip doing the Gets, how fast does it became? J-D On Fri, Sep 27, 2013 at 10:33 AM, Pavan Sudheendra pavan0...@gmail.com wrote: Hi everyone, I posted this question many time before and i've given full details on stackoverflow.. http://stackoverflow.com/q/19056712/938959 Please i need someone to guide me in the right direction here. Help much appreciated! -- Regards- Pavan -- Regards- Pavan
