HDFS-941 just got committed to TRUNK in a coordinated effort led by our man Todd (Hopefully it makes it into 0.22!). HDFS-1148 is next! St.Ack
On Sun, Jun 5, 2011 at 10:09 PM, Todd Lipcon <t...@cloudera.com> wrote: > OK guys, I did my part: I rebased HDFS-941 and HDFS-1148 to trunk. Also > attempted to rebase HDFS-1323 (nee 918) but it's failing tests. > > If you want to help, please take a look at the failing tests on HDFS-1323 > and see if you can understand what might be going wrong. > > Unfortunately this isn't my top priority at work right now (HA for the > namenode is), but I'm happy to spend some nights and weekends to help push > these through if they really work. > > -Todd > > On Sun, Jun 5, 2011 at 3:40 PM, Todd Lipcon <t...@cloudera.com> wrote: > >> On Sat, Jun 4, 2011 at 1:46 AM, Andrew Purtell <apurt...@apache.org>wrote: >> >>> This is not discouraging. :-) >>> >>> HBasers patch CDH because trunk -- anything > 0.20 actually -- is not >>> trusted by consensus if you look at all of the production deployments. Does >>> ANYONE run trunk under anything approaching "production"? And trunk/upstream >>> has a history of ignoring any HBase specific concern. So the use of and >>> trading of patches will probably continue for a while, maybe forever. >>> >> >> Right - I wasn't suggesting that you run trunk in production as of yet. But >> there has been very little activity in terms of HBase people running trunk >> in dev/test clusters in the past. Stack has done some awesome work here in >> the last few weeks, so that should open it up for some more people to jump >> on board. >> >> I agree that HBase has been treated as a second-class citizen in recent >> years from HDFS's performance, but I think that has changed. All of the >> major HDFS contributors now have serious stakes in HBase, and so long as >> there are tests with sufficient testing that apply against trunk, I don't >> see a reason they wouldn't be included. >> >> >>> >>> Part of the problem is the expectation that any patch provided against >>> trunk may generate months of back and forth, as we have seen, which presents >>> difficulities to a potential contributor who does not work on e.g. HDFS >>> matters full time. Alternatively it may pick up a committer as sponsor and >>> then be vetoed by Yahoo because they're mad at Cloudera over some unrelated >>> issue and a patch appears to have a Cloudera sponsor and/or or vice versa. >>> Now, that situation I describe _is_ discouraging. It's not enough to say >>> that we must contribute through trunk. Trunk needs to earn back our trust. >>> >> >> Yes, there have been some unfortunate things in the past. There have also >> been some half-finished or untested patches proposed, and you can't blame >> HDFS folks for not taking a big patch that doesn't have a lot of confidence >> behind it. >> >> I've been thinking about this this afternoon, and have an idea. It may >> prove to be an awful one, but maybe it's a good one, only time will tell :) >> I'll create a branch off of HDFS trunk specifically for HBase performance >> work. We can commit these "90% done" patches there, which will make it >> easier for others to test and gain confidence. Branches also can make it >> easier to maintain patches over time with a changing trunk. >> >> How does this sound to the HBase community? If it seems like a good idea, >> *and* there are some people who would be willing to set it up on some small >> dev clusters and run load tests, I'll move forward with it. >> >> >>> I believe I recently saw discussion that append should be removed or >>> disabled by default on 0.22 or trunk. Did you see anything like this? If I >>> am mistaken, fine. If not, this is going in the wrong direction, for >>> example. >>> >> >> Not sure what you're referring to - I don't remember any discussion like >> this. >> >> -Todd >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
