Jeff, thanks very much for the response and for pushing forward with the gateway.
On the Python front, I just wrote an asynchronous client (Tornado-based). Commit: https://github.com/mjrusso/pyhbase/commit/07ed39527bd6752158b1293e8d2df528d649a613 Let me know if you have any feedback on this. If you don't feel that it is a fit for pyhbase, I'm happy to maintain a fork. (There's also the question of if the TornadoRequestor and TornadoHTTPTransceiver should be included in the Avro package itself.) Best, Michael On Sun, Dec 26, 2010 at 3:16 PM, Jeff Hammerbacher <[email protected]>wrote: > Okay, I felt guilty about not cleaning this code up in a while. > > I opened https://issues.apache.org/jira/browse/HBASE-3393 to track > updating > the server to use the new join() method rather than sleep()ing for a long > time. The tests are failing for a very strange reason right now (claiming > that a field is missing on an Avro record that's clearly present in the > .java file; I'm at a loss), but I'll put my patch up in case you want to > try > to make progress. > > Also, Maven can't seem to find the Netty jar needed by Avro. I've got an > email out to the Avro lists to figure out what's up with that. For now, you > can just add the JBoss repository to the HBase pom.xml as directed at > http://www.jboss.org/netty/downloads.html. > > I also fixed a bug where the server wasn't using the port number passed at > the command line. > > I also opened https://issues.apache.org/jira/browse/HBASE-3394 to change > the > build process to generate the Java code at compile time rather than forcing > us to check in the compiled Java code. That might not be preferred for some > who prefer source code to be checked in. Happy to discuss on that ticket. > > Hope that helps, > Jeff > > On Sun, Dec 26, 2010 at 10:50 AM, Jeff Hammerbacher <[email protected] > >wrote: > > > Hey Michael, > > > > Actually, it was something more basic: AVRO-544 changed the way servers > are > > started so that the gateway class could just join the Avro server thread > and > > run forever, rather than having to run for a finite amount of time. I > still > > need to get an HBase-side change in that uses this new feature of Avro--I > > just haven't had much time for coding of late. > > > > Other than that, I have been lax in keeping up with some of the new APIs > > introduced in 0.90 (the multi* stuff, for example). There are a few other > > cleanup tasks at https://issues.apache.org/jira/browse/HBASE-2400 that > are > > worth checking out. > > > > Thanks for sending the patches my way for the Python client. I'll do my > > best to get to some of these updates soon, and would be more than happy > to > > review patches for HBase, Avro, or the Python client on Github (thanks > for > > the patches you've already sent my way). > > > > Regards, > > Jeff > > > > > > > > On Sun, Dec 26, 2010 at 7:16 AM, Michael Russo <[email protected]> > wrote: > > > >> Jeff Hammerbacher stated on Quora [1] that "the Avro gateway is not > >> production quality" and made a reference to the improvements that an > >> upgrade > >> to Avro 1.4 would bring. > >> > >> I would like to understand a little better what this means. > >> > >> I reviewed the Avro changelog [2] for 1.4+, and very quickly > >> scanned o.a.h.h.avro.AvroServer.java [3] and the only Avro change that > >> really stood out in the context of significantly improving the > production > >> quality of the gateway is AVRO-405 (Netty-based Java RPC server) [4]. > >> > >> Is this a reasonable analysis or are there any other important elements > >> that > >> I'm missing? > >> > >> [1]: > >> > http://www.quora.com/What-is-the-current-status-for-using-Avro-with-HBase > >> [2]: https://github.com/apache/avro/blob/trunk/CHANGES.txt > >> [3]: > >> > >> > http://svn.apache.org/viewvc/hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java?view=markup > >> [4]: https://issues.apache.org/jira/browse/AVRO-405 > >> > >> Thanks, > >> Michael > >> > > > > >
