On Fri, Jun 3, 2011 at 12:50 PM, Doug Meil <doug.m...@explorysmedical.com> wrote: > Thanks everybody for commenting on this thread. > > We'd certainly like to lobby for movement on these two tickets, and although > we don't have anybody that is familiar with the source code we'd be happy to > perform some tests get some performance numbers. > > Per Kihwal's comments, it sounds like HDFS-941 needs to get re-worked because > the patch is stale. >
Yes - bc Wong, the originally contributor, works with me but on unrelated projects. HDFS-941 was something he did as part of a "hackathon" but only gets occasional time to circle back on it. As we last left it, there were just a few things that had to be addressed. If someone wants to finish it up, and volunteer to test it under some real load, I'd be happy to review and commit. > The patch for HDFS-347 sounds like it's still usable. The current patch for 347 is unworkable since it doesn't do checksums or security. The FD-passing approach was working at some point but basically needs to be re-done on trunk. I think doing HDFS-941 and HDFS-918 first is best, then more drastic things like 347 can be considered. > > So what else is needed to push this effort forward? Is it beneficial to get > more numbers on HDFS-347 and keep lobbying on the ticket, and/or is there > another path that should be taken (plying with beer, free Cleveland Indians > tickets, harassing phone calls, etc.)? Testing! The thing that's scariest about HDFS-941 is that it was passing all of its unit tests, and then when I tried it under load for 2-3 days with YCSB on a 5 node cluster, I saw a couple of checksum exceptions come out of it. That implies there's a bug lurking somewhere. It may be fixed by now, but I'm hesitant to commit it unless there has been testing of that sort. -Todd > > > > -----Original Message----- > From: Dhruba Borthakur [mailto:dhr...@gmail.com] > Sent: Friday, June 03, 2011 3:00 PM > To: dev@hbase.apache.org > Subject: Re: HDFS-1599 status? (HDFS tickets to improve HBase) > > I completely agree with Ryan. Most of the measurements in HDFS-347 are point > comparisions.... data rate over socket, single-threaded sequential read from > datanode, single-threaded random read form datanode, etc. These measurements > are good, but when you run the entire Hbase system at load, you definitely > see a 3X performance improvement when reading data locally (instead of going > through the datanode). > > -dhruba > > On Fri, Jun 3, 2011 at 11:08 AM, Ryan Rawson <ryano...@gmail.com> wrote: > >> Could you explain your HDFS-347 comment more? I dont think people >> suggested that the socket itself was the primary issue, but dealing >> with the datanode and the socket and everything was really slow. It's >> hard to separate concerns and test only 1 thing at a time - for >> example you said 'local socket comm isnt the problem', but there is no >> way to build a test that uses a local socket but not the datanode. >> >> The basic fact is that datanode adds a lot of overhead, and under high >> concurrency that overhead grows. >> >> >> >> On Fri, Jun 3, 2011 at 7:07 AM, Kihwal Lee <kih...@yahoo-inc.com> wrote: >> > HDFS-941 >> > The trunk has moved on so the patch won't apply. There has been >> significant changes in HDFS lately, so it will require more than >> simple rebase/merge. If the original assignee is busy, I am willing to help. >> > >> > HDFS-347 >> > The analysis is pointing out that local socket communication is >> > actually >> not the problem. The initial assumption of local socket being slow >> should be ignored and the design should be revisited. >> > >> > I agree that improving local pread performance is critical. Based >> > on my >> experiments, HDFS-941 helps a lot and the communication channel became >> no longer the bottleneck. >> > >> > Kihwal >> > >> > >> > On 6/2/11 4:00 PM, "Doug Meil" <doug.m...@explorysmedical.com> wrote: >> > >> > Hi folks, I was wondering if there was any movement on any of these >> > HDFS >> tickets for HBase. The umbrella ticket is HDFS-1599, but the last >> comment from stack back in Feb highlighted interest in several tickets: >> > >> > >> > 1) HDFS-918 (use single selector) >> > >> > a. Last comment Jan 2011 >> > >> > >> > >> > 2) HDFS-941 (reuse of connection) >> > >> > a. Patch available as of April 2011 >> > >> > b. But ticket still unresolved. >> > >> > >> > >> > 3) HDFS-347 (local reads) >> > >> > a. Discussion seemed to end in March 2011 with a huge comment >> saying that there was no performance benefit. >> > >> > b. I'm working my way through this comment/report, but intuitively >> it seems like it would be a good idea since as the other comments in >> the ticket stated the RS reads locally just about every time. >> > >> > >> > Doug Meil >> > Chief Software Architect, Explorys >> > doug.m...@explorys.com >> > >> > >> > >> > > > > -- > Connect to me at http://www.facebook.com/dhruba > -- Todd Lipcon Software Engineer, Cloudera
