Proposed feature branch for HBase security

2010-07-02 Thread Gary Helmling
Hi folks, With Yahoo's latest security release on github ( http://github.com/yahoo/hadoop-common/tree/yahoo-hadoop-0.20.104), it looks like we now have a real-world usable version of secure Hadoop, based on 0.20. This is exciting stuff, because now we have something solid to start working

Re: RPC statistics from region servers...

2010-07-29 Thread Gary Helmling
Hmm, this sounds like HBASE-2146. That patch had added code to initialize the RPC metrics MBean with a know list of RPC method names maintained in HBaseRPC.Invocation. That way all per-method metrics would show up on server startup. I believe that that method name mapping was later removed

Review Request: HBASE-2742, HBASE-2016: Port of secure Hadoop RPC changes and integration with HBase RPC protocols

2010-07-29 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/406/ --- Review request for hbase. Summary --- This patch ports over the secure

Re: RPC statistics from region servers...

2010-07-31 Thread Gary Helmling
Fixed in HBASE-2890 On Jul 31, 2010, at 4:38 AM, kann...@gmail.com (Kannan Muthukkaruppan) kann...@gmail.com wrote: Gary: Got it. Thanks for the clarification. On Thu, Jul 29, 2010 at 4:39 PM, Gary Helmling ghelml...@gmail.com wrote: Ryan: Yep, agreed on removal of the code

Re: Review Request: HBASE-2742, HBASE-2016: Port of secure Hadoop RPC changes and integration with HBase RPC protocols

2010-08-03 Thread Gary Helmling
, Gary Helmling wrote: --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/406/ --- (Updated 2010-07-29 12:40:06) Review request for hbase

Re: Review Request: HBASE-2742, HBASE-2016: Port of secure Hadoop RPC changes and integration with HBase RPC protocols

2010-08-09 Thread Gary Helmling
--- On 2010-07-29 12:40:06, Gary Helmling wrote: --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/406

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-09-10 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-09-10 17:29:57.718053) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-09-17 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-09-17 15:20:58.111896) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-09-24 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-09-24 00:01:53.015127) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-09-28 Thread Gary Helmling
--- On 2010-09-24 00:01:53, Gary Helmling wrote: --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-09-30 Thread Gary Helmling
On 2010-09-25 16:18:25, Andrew Purtell wrote: src/main/java/org/apache/hadoop/hbase/client/HTable.java, line 1400 http://review.cloudera.org/r/816/diff/5/?file=12533#file12533line1400 Maybe for sake of clarity call this getStartKeysInRange? Gary Helmling wrote: Makes sense

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-01 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-10-01 13:48:46.472644) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-01 Thread Gary Helmling
On Fri, Oct 1, 2010 at 1:54 PM, Ryan Rawson ryano...@gmail.com wrote: I think it's tricky, because we dont really encourage people to think of regions, but think of rows instead. The fact that regions exist is a bit of an implementation detail, although like indexes in databases a critical

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-04 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-10-04 12:04:08.373390) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-05 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/#review1414 --- On 2010-10-04 16:28:39, Gary Helmling wrote

Re: dynamic RPC and coprocessor changes

2010-10-06 Thread Gary Helmling
On Tue, Oct 5, 2010 at 7:41 PM, Andrew Purtell apurt...@apache.org wrote: With that said, I expect Gary is already working to address this concern specifically: Looking at some of their names, they are generic sounding and it's not clear to me without digging in that they are part of

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-06 Thread Gary Helmling
On 2010-10-04 23:09:49, stack wrote: src/main/java/org/apache/hadoop/hbase/client/Exec.java, line 65 http://review.cloudera.org/r/816/diff/9/?file=13681#file13681line65 This is fair I suppose if only one coprocessor per region (Is that true)? Gary Helmling wrote: Correct

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-06 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-10-06 21:21:23.477207) Review request for hbase, Andrew Purtell

Re: Review Request: [HBASE-2321] [HBASE-2002] Add support for per-region dynamically registered RPC endpoints for coprocessors and allow configurable RPC client/server implementations

2010-10-06 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/ --- (Updated 2010-10-06 21:22:21.837569) Review request for hbase, stack, Andrew

Re: Review Request: HBASE-3102: Add Capability to Monitor Long Running Metrics

2010-10-26 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1055/#review1672 --- Looks pretty good. These look like great metrics to add! Only issue

Re: Review Request: HBASE-3102: Add Capability to Monitor Long Running Metrics

2010-10-26 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1055/#review1676 --- Ship it! Based on discussion in JIRA, I'm fine addressing naming

Re: hbase monitoring through stargate

2010-10-27 Thread Gary Helmling
One other option, since a read-only configuration isn't currently available, you could configure Apache HTTPD as a reverse proxy in front of the stargate server and use mod_rewrite to only allow get requests through. Since I believe stargate follows the RESTful conventions for request methods,

Re: Review Request: HBASE-3194: Allow HBase to run against both regular Hadoop 0.20 and secure Hadoop

2010-11-08 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1184/ --- (Updated 2010-11-08 17:18:26.166428) Review request for hbase. Changes

Re: Hbase security: Encryption of Data before storage on physical disk

2010-11-16 Thread Gary Helmling
Hi Preetam, A group of us at Trend Micro have been working on adding some security features to HBase. We've been focused on authentication and access control though, as opposed to data encryption. HBase sits on top of HDFS and relies on it as the underlying filesystem. HBase implements it's own

Re: Please welcome two new hbase committers, Gary Helmling and Nicolas Spiegelberg

2010-12-02 Thread Gary Helmling
On Fri, Dec 3, 2010 at 7:03 AM, Andrew Purtell apurt...@apache.org wrote: Congratulations Gary and Nicholas! - Andy --- On Thu, 12/2/10, Stack st...@duboce.net wrote: From: Stack st...@duboce.net Subject: Please welcome two new hbase committers, Gary Helmling and Nicolas Spiegelberg

Review Request: HBASE-3260: Add explicit lifecycle methods to Coprocessor interface

2010-12-16 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1306/ --- Review request for hbase, stack and Andrew Purtell. Summary --- This patch

Review Request: HBASE-3256: Coprocessors: Coprocessor host and observer for HMaster

2010-12-20 Thread Gary Helmling
--- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/1321/ --- Review request for hbase, stack, Andrew Purtell, and Jonathan Gray. Summary

Re: hadoop version for hbase 0.90

2010-12-28 Thread Gary Helmling
Hi Ted, Yes, you will need to replace the hadoop-core jar in hbase/lib/ with the cdh3b2 version. The jar in hbase/lib/ always needs to be exactly in sync with what you are running on the DNs. I believe there are a few additional HDFS patches applied to the Hadoop 0.20-append branch that HBase

Re: Build failed in Hudson: HBase-TRUNK #1706

2011-01-06 Thread Gary Helmling
From my RPC metrics test. My bad. Fixed. On Thu, Jan 6, 2011 at 4:25 PM, Apache Hudson Server hud...@hudson.apache.org wrote: See https://hudson.apache.org/hudson/job/HBase-TRUNK/1706/changes Changes: [todd] HBASE-3401. Region IPC operations should be high priority [todd] HBASE-3407

Re: RegionObserver Coprocessor PreGet

2011-02-02 Thread Gary Helmling
Hi JS, For a real (though a little complicated) example of coprocessors, you can check out the HBase security code. The latest working code is up on github: https://github.com/trendmicro/hbase/tree/security See the org.apache.hadoop.hbase.security.rbac.AccessController class for the actual

Re: Let's Switch to TestNG

2011-02-23 Thread Gary Helmling
See http://jira.codehaus.org/browse/SUREFIRE-656 for the missing surefire support for JUnit Categories. TestNG can run existing JUnit tests, though I don't know if the surefire TestNG support can do this transparently or if it requires some additional config (in testng.xml)? Other nice looking

Re: Proposal: HBase Hackathon on the 21st of March in SF

2011-02-28 Thread Gary Helmling
+1 as well. The 21st is good. Look forward to it! On Mon, Feb 28, 2011 at 3:26 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: +1 On Mon, Feb 28, 2011 at 3:24 PM, Stack st...@duboce.net wrote: Chatting on IRC a few of us are thinking of having an HBase Hackathon on the 21st of March

Re: Coprocessor tax?

2011-03-01 Thread Gary Helmling
, Gary Helmling ghelml...@gmail.com wrote: All the CoprocessorHost invocations should be wrapped in if (cpHost != null). We could just added an extra check for whether any coprocessors are loaded -- if (cpHost != null cpHost.isActive()), something like that? Or the CoprocessorHost methods

Re: Coprocessor tax?

2011-03-01 Thread Gary Helmling
Sure, even better (it's doing the locking for you). On Tue, Mar 1, 2011 at 12:49 PM, Ryan Rawson ryano...@gmail.com wrote: I don't think we need a lock even for updating, check it copy on write array list. On Mar 1, 2011 12:45 PM, Gary Helmling ghelml...@gmail.com wrote: Yeah, I was just

Re: Coprocessor tax?

2011-03-01 Thread Gary Helmling
Created HBASE-3587: https://issues.apache.org/jira/browse/HBASE-3587 https://issues.apache.org/jira/browse/HBASE-3587Please comment/correct if I left anything out or summed it up wrong. On Tue, Mar 1, 2011 at 12:59 PM, Gary Helmling ghelml...@gmail.com wrote: Sure, even better (it's doing

Re: putting a border around 0.92 release

2011-03-02 Thread Gary Helmling
I think it's very relevant that the linux kernel moved from one approach to the other as it reached a certain scale of developer participation and broader ecosystem support. I think what really enables the new linux approach is the fact that there are multiple layers of stabilization between the

Re: hbase-2001 added getZooKeeperWatcher and getCatalogTracker to RegionServerServices but these two methods are in Server

2011-03-24 Thread Gary Helmling
Seems odd that a CP, a region-scoped thing, would want to make use of the catalogtracker and/or zk? With AccessController we found a need to coordinate some state (ACL permissions) across all region CP instances. Seems pretty likely that other implementors will run into a similar need. I

Re: HbaseObjectWritable and UnsupportedOperationException

2011-03-31 Thread Gary Helmling
Hi Ted, First, just to make sure, LongColumnInterpreter implements either Writable or Serializable, right? If so (I assume this is based on current trunk), sounds like a bug to me. There have been a couple changes over the past several months to allow HbaseObjectWritable to serialize Writables

Re: ColumnInterpreter and HbaseObjectWritable Was: HbaseObjectWritable and UnsupportedOperationException

2011-03-31 Thread Gary Helmling
On Thu, Mar 31, 2011 at 11:37 AM, Ted Yu yuzhih...@gmail.com wrote: Renaming the subject to better reflect the nature of further discussion. There're two considerations for my current implementation attached to HBASE-1512. 1. User shouldn't modify HbaseObjectWritable directly for the new

reviews.apache.org

2011-04-05 Thread Gary Helmling
Since ASF Infra is now running a version of review board for projects, should we consider moving to it? review.cloudera.org has served us well (thanks Todd and Cloudera!), but the email rejection based on spam scoring is becoming a big pain and making it harder to keep track of updates.

Re: reviews.apache.org

2011-04-05 Thread Gary Helmling
St.Ack On Tue, Apr 5, 2011 at 10:49 AM, Stack st...@duboce.net wrote: On Tue, Apr 5, 2011 at 10:29 AM, Gary Helmling ghelml...@gmail.com wrote: Since ASF Infra is now running a version of review board for projects, should we consider moving to it? Yes. Sorry, I initiated the move w

Re: Exec implementing Row

2011-04-27 Thread Gary Helmling
This is a remnant from initial plans to do single-RPC-per-RS batching of coprocessor RPCs via the parameterized HConnectionManager.HConnectionImplementation.processBatchCallback() method. Currently we do a single RPC per region for HTable.coprocessorExec() invocations. processBatchCallback()

Re: Exec implementing Row

2011-04-28 Thread Gary Helmling
On Wed, Apr 27, 2011 at 10:47 PM, Gary Helmling ghelml...@gmail.com wrote: This is a remnant from initial plans to do single-RPC-per-RS batching of coprocessor RPCs via the parameterized HConnectionManager.HConnectionImplementation.processBatchCallback() method. Currently we do a single RPC

Re: Exec implementing Row

2011-04-28 Thread Gary Helmling
On Wed, Apr 27, 2011 at 11:14 PM, Himanshu Vashishtha hvash...@cs.ualberta.ca wrote: For the least of it, one can get rid of private byte[] referenceRow attribute in Exec class I'd say (?). Um, what would Row.getRow() return in that case? Sure, if you drop the Row implementation, you can

Re: Add FB comments to tail of all pages in our hbase website book?

2011-04-28 Thread Gary Helmling
+1! Yeah, I've also found the mysql doc comments very useful. I think this would be great to add to our book. If you need a hand hacking it up let me know. I did some FB app stuff in the past. --gh On Thu, Apr 28, 2011 at 1:09 PM, Stack st...@duboce.net wrote: Why I was a mysql'r, often

Re: Switching to Jamon templates?

2011-05-02 Thread Gary Helmling
on the +1 train as well. Haven't worked with Jamon, but anything to clean up the web UI pages is much appreciated! On Mon, May 2, 2011 at 12:35 PM, Andrew Purtell apurt...@yahoo.com wrote: +1 From: Todd Lipcon t...@cloudera.com I filed HBASE-3835 over the weekend to propose switching

Re: region vs regionserver coprocessors

2013-05-14 Thread Gary Helmling
Looking at RegionServerObserver, it only defines a single method -- preStopRegionServer(). AccessController mediates access on all normal client operations on data, which happen on regions and are only represented in RegionObserver. So what it sounds like you're asking is if RegionServerObserver

Re: BBUZZ Berlin Sightseeing?

2011-05-26 Thread Gary Helmling
I'd be up for it as well. I arrive Saturday the 4th, so Sunday would be good. Any chance of a bigger bus if we wind up with more than 9 people? On Thu, May 26, 2011 at 2:25 PM, Andrew Purtell apurt...@apache.org wrote: I'd be up for this, along with two friends. So that's more $$$ than

Help with 0.92 new feature list

2011-06-05 Thread Gary Helmling
Hi folks, As part of my presentation for Hadoop Summit, I'd like to include a slide highlighting new features or major improvements in 0.92. I've admittedly had my head in coprocessor/security land for a while, so I'm not sure I'm up to date on everything that's been happening, and I'd like to

Re: Please welcome our newest committer, Ted Yu!

2011-06-13 Thread Gary Helmling
Congratulations, Ted! Keep those patches coming! Gary On Mon, Jun 13, 2011 at 1:49 PM, Stack st...@duboce.net wrote: Ted just got his committer badge (One of us!). Ted, usually the first thing fellas do is add their details to pom.xml -- see around line 120 (You can write me offline if you

Re: Retry HTable.put() on client-side to handle temp connectivity problem

2011-06-28 Thread Gary Helmling
I'd also be wary of changing the default to retry forever. This might be hard to differentiate from a hang or deadlock for new users and seems to violate least surprise. In many cases it's preferable to have some kind of predictable failure as well. So I think this would appear to be a

Re: Proposal: HBase Hackathon Aug 22 @ FB

2011-08-04 Thread Gary Helmling
+1 On Thu, Aug 4, 2011 at 2:02 PM, Todd Lipcon t...@cloudera.com wrote: Works for me. On Thu, Aug 4, 2011 at 1:50 PM, Nicolas Spiegelberg nspiegelb...@fb.com wrote: Chatted on IRC with Stack a couple others. We would like to host a Hackathon + Meetup on August 22 at Facebook in Palo

Re: Please welcome our new committer Doug Meil!

2011-08-08 Thread Gary Helmling
Congrats Doug! Welcome aboard! Gary On Mon, Aug 8, 2011 at 2:00 PM, Stack st...@duboce.net wrote: Good on you Doug! St.Ack

Re: coprocessor table operations

2011-08-27 Thread Gary Helmling
On Fri, Aug 26, 2011 at 5:19 PM, Ma, Ming min...@ebay.com wrote: Thanks, Gray, Ted. I will open a jira and provide the fix later. About where to call such post methods for table operations, after request is queued to executor or after EventHandler.process has finished: 1. It applies to

Re: New HBase Logo

2011-09-01 Thread Gary Helmling
Thanks Stack for organizing and herding this along, and thanks StumbleUpon for sponsoring it! On Wed, Aug 31, 2011 at 9:30 PM, Stack st...@duboce.net wrote: Your PMC voted the following as the new HBase logo: https://issues.apache.org/jira/secure/attachment/12492477/01.jpg It will replace

Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal

2011-09-02 Thread Gary Helmling
Some comments on the proposal and differentiation vs HBase: Access Labels: The proposal claims that this is unlikely to be adopted [in HBase]. This is completely untrue. This has been discussed many times in the past in relation to our security implementation. It's just been deferred at the

HBASE-3842: RegionObserver compaction API changes

2011-09-02 Thread Gary Helmling
I plan to commit the patch for refactoring RegionObserver coprocessor hooks at the end of day today: https://reviews.apache.org/r/1686/ If anyone wants to comment, please do so before then. (Nicolas, I'm looking at you!) --gh

Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal

2011-09-02 Thread Gary Helmling
I've been in touch with this team for the last 18 months or so. They're good people, smart, and have a healthy respect for HBase and our team. Though they haven't contributed code or participated on the lists, I can vouch that they do follow our development and generally do understand HBase

Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal

2011-09-06 Thread Gary Helmling
Seems like committing it will disrupt the build and src tree layout. Gary was holding off till we branched but 0.92 branching is taking too long. + Lets branch this friday, or next? + And or, run a vote on whether we should commit security now before we branch or after This is getting

Re: Branching for 0.92 [WAS - Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal]

2011-09-07 Thread Gary Helmling
, then a parallel refactor of the build could happen, with a final merge step. Best regards, - Andy On Tue Sep 6th, 2011 12:02 PM PDT Gary Helmling wrote: Seems like committing it will disrupt the build and src tree layout. Gary was holding off till we branched but 0.92 branching

Re: Branching for 0.92 [WAS - Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal]

2011-09-07 Thread Gary Helmling
, Sep 7, 2011 at 9:26 AM, Gary Helmling ghelml...@gmail.com wrote: I was going to start a thread on modularizing pre-0.92, but given that's there's no actual patch ready, only a JIRA (HBASE-4336), that seems premature. We can race and see how long it takes to put something together, then call

Re: Branching for 0.92 [WAS - Re: [DISCUSSION] Accumulo, another BigTable clone, has shown up on Apache Incubator as a proposal]

2011-09-07 Thread Gary Helmling
On Wed, Sep 7, 2011 at 10:49 AM, Gary Helmling ghelml...@gmail.com wrote: The secure RPC engine would still have to be split off into a separate tree, since there are some classes that derive from the SASL support and token authentication in secure Hadoop. Can't break compilation of the non

Re: Build failed in Jenkins: HBase-TRUNK #2186

2011-09-08 Thread Gary Helmling
Looks like TestHLogUtils didn't get added with the HBASE-4260 patch? On Wed, Sep 7, 2011 at 10:20 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: See https://builds.apache.org/job/HBase-TRUNK/2186/changes Changes: [stack] HBASE-4296 Deprecate

Re: Secure Hadoop and non-secure HBase

2011-09-11 Thread Gary Helmling
Hi Eric, If you configure hbase.master.keytab.file hbase.master.kerberos.principal hbase.regionserver.keytab.file hbase.regionserver.kerberos.principal in your hbase-site.xml, then the master and region server processes should login from the keytab files on startup, as Todd mentions. It's also

Re: Secure Hadoop and non-secure HBase

2011-09-12 Thread Gary Helmling
. Thank you. regards, Eric On Sep 11, 2011, at 3:57 PM, Gary Helmling wrote: Hi Eric, If you configure hbase.master.keytab.file hbase.master.kerberos.principal hbase.regionserver.keytab.file hbase.regionserver.kerberos.principal in your hbase-site.xml, then the master and region

Re: HBASE-4373

2011-09-16 Thread Gary Helmling
Thanks for the heads up Ted. On Thu, Sep 15, 2011 at 11:13 PM, Ted Yu yuzhih...@gmail.com wrote: Andy/Gary/Mingjie: HBASE-4373 touched certain coprocessor APIs. Please take a look when you have time. Thanks

Re: Please welcome Ramkrishna S. Vasudevan, our newest hbase committer

2011-09-28 Thread Gary Helmling
Congrats Ram! Great work! On Wed, Sep 28, 2011 at 9:41 AM, Jean-Daniel Cryans jdcry...@apache.orgwrote: Great job Ram! First commit should be adding yourself here: http://hbase.apache.org/team-list.html J-D On Wed, Sep 28, 2011 at 9:15 AM, Stack st...@duboce.net wrote: Please welcome

Re: backporting HBASE-3777 to 0.90

2011-09-28 Thread Gary Helmling
Changing the connection identity behavior in the middle of a release series seems like a bad idea. The 0.20 releases did connection identity based on Configuration contents, 0.90 changed this to Configuration instance identity, then 0.90.5 would be going back to contents again (acknowledged with

Re: Forcing separate connections Was: backporting HBASE-3777 to 0.90

2011-09-28 Thread Gary Helmling
of the other connection * parameters differ. */ public static String HBASE_CLIENT_INSTANCE_ID = hbase.client.instance.id ; FYI On Wed, Sep 28, 2011 at 12:06 PM, Gary Helmling ghelml...@gmail.com wrote: Changing the connection identity behavior in the middle of a release series

Re: backporting HBASE-3777 to 0.90

2011-09-28 Thread Gary Helmling
On Wed, Sep 28, 2011 at 1:39 PM, Andrew Purtell apurt...@apache.org wrote: I'd switch from -1 to +1 if we can get +1s from people who have tried it on clusters with several different real existing apps written by several different teams. This makes sense. My +1 was partly an agreement

Re: Build failed in Jenkins: HBase-0.92 #30

2011-09-29 Thread Gary Helmling
Created HBASE-4518 to deal with TestServerCustomProtocol flakiness. On Thu, Sep 29, 2011 at 3:54 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: See https://builds.apache.org/job/HBase-0.92/30/changes Changes: [stack] HBASE-4513 NOTICES.txt refers to Facebook for Thrift

Re: hadoop 0.20.205.0 release candidate testing

2011-10-04 Thread Gary Helmling
HBASE-4515 mistakenly brought in the UnixUserGroupInformation reference as an import. Created HBASE-4537 to clean it up. On Tue, Oct 4, 2011 at 3:09 PM, Stack st...@duboce.net wrote: I've been doing a bit of hadoop 0.20.205.0 release candidate testing running tip of 0.90 branch on it. I've

Re: hadoop 0.20.205.0 release candidate testing

2011-10-04 Thread Gary Helmling
mvn-install, the HBase build then picked it up as a dependency. If you run into other problems, please share! On Tue, Oct 4, 2011 at 3:53 PM, Gary Helmling ghelml...@gmail.com wrote: HBASE-4515 mistakenly brought in the UnixUserGroupInformation reference as an import. Created HBASE-4537

TestHLog hanging in trunk

2011-10-05 Thread Gary Helmling
I've noticed that TestHLog is currently hanging in trunk (haven't checked other branches). Oddly the tests actually complete, but then the test hangs in teardown. Seems to be something in the server shutdown hooks. git bisect tracks down the hang to this commit: commit

Re: TestHLog hanging in trunk

2011-10-05 Thread Gary Helmling
()); } + @AfterClass + public static void tearDownAfterClass() throws Exception { +TEST_UTIL.shutdownMiniCluster(); + } + private static String getName() { // TODO Auto-generated method stub return TestHLog; On Wed, Oct 5, 2011 at 11:23 AM, Gary Helmling ghelml...@gmail.com wrote: I've noticed

Re: TestHLog hanging in trunk

2011-10-05 Thread Gary Helmling
wrote: +CC Roman who worked on the patch identified by the bisect. Roman, does Gary's analysis make sense to you? -Todd On Wed, Oct 5, 2011 at 11:55 AM, Gary Helmling ghelml...@gmail.com wrote: Somehow TestHLog was never actually shutting down the mini-cluster? The following change lets

Re: TestHLog hanging in trunk

2011-10-05 Thread Gary Helmling
) Any idea what's causing the region closing to hang? The surefire plugin doesn't seem to write the TestHLog-output.txt file if the test is hanging. Any way to force it to do so? Kind of useless. On Wed, Oct 5, 2011 at 12:13 PM, Gary Helmling ghelml...@gmail.com wrote: Something else seems

Re: TestHLog hanging in trunk

2011-10-05 Thread Gary Helmling
I created HBASE-4545 to clean up TestHLog. Among other things, it didn't even need to spin up a MiniHBaseCluster, only a MiniDFSCluster. Several of the tests were also leaking unclosed HLog instances. Will post a patch shortly. On Wed, Oct 5, 2011 at 5:52 PM, Gary Helmling ghelml...@gmail.com

Re: Including specific tests in the latest pom.xml

2011-10-06 Thread Gary Helmling
Ram, If you just want to run a set of tests together, I think you can do that from the command line: mvn test -Dtest=TestLogRolling,TestOpenRegionHandler,TestSplitTransaction Does that get you what you're after? --gh On Thu, Oct 6, 2011 at 11:08 AM, Ramakrishna S Vasudevan 00902313

Re: expecting X results spam

2011-10-06 Thread Gary Helmling
+1 to removing it It was intentional at one point... :) But that doesn't mean it should have stayed in the final patch for commit. On Thu, Oct 6, 2011 at 5:26 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: Hey devs, I was testing out 0.92 and HCM got this like this a year ago: +

Re: Please welcome our newest committer, Lars Hofhansl

2011-10-07 Thread Gary Helmling
Congrats, Lars! On Fri, Oct 7, 2011 at 10:29 AM, Doug Meil doug.m...@explorysmedical.comwrote: Congrats, and welcome! On 10/7/11 12:53 PM, Stack st...@duboce.net wrote: One of us! As per tradition, your first commit should be adding yourself to the pom.xml Lars in the committers

Re: Build failed in Jenkins: hbase-0.90 #324

2011-10-14 Thread Gary Helmling
this fail because of your change Gary? Good on you, St.Ack On Fri, Oct 14, 2011 at 6:06 AM, Apache Jenkins Server jenk...@builds.apache.org wrote: See https://builds.apache.org/job/hbase-0.90/324/changes Changes: [Gary Helmling] HBASE-4282 RegionServer should abort when WAL close fails

Re: Build failed in Jenkins: hbase-0.90 #324

2011-10-14 Thread Gary Helmling
make sense, since we're testing when we should abort vs. when we shouldn't. I'll open a follow up JIRA. --gh On Fri, Oct 14, 2011 at 12:39 AM, Gary Helmling ghelml...@gmail.com wrote: From Jenkins, the failure was in TestLogRolling.testLogRollOnPipelineRestart, so looks like

Re: adding constraints

2011-10-18 Thread Gary Helmling
There is an example of how to do Constraints as a jar with CPs already attached to the ticket, and its pretty simple. However, the ticket goes into the plusses and minuses for a top-level or just basic CP based implementation. For me, the best reason for top level is top make HBase easy to

Re: adding constraints

2011-10-18 Thread Gary Helmling
Hmm, I wasn't really reading the two implementation options for constraints as a choice between a built-in feature and CP based. Either way it would be CP based, but the 'built-in' would just have some 'nice' ways of adding things. In short, its a question of adding a method to the HTD for

Re: adding constraints

2011-10-19 Thread Gary Helmling
Do you think you can put the core what we are agreeing on into 4605?  I want to make sure we don't lose any of your comments Sure I'll try to summarize in a comment on 4605. I think we'll need to open a new JIRA for the shell aspects of this as well, since it looks like 4554 is only handling

Re: proposal for naming convention of patches for TRUNK

2011-10-26 Thread Gary Helmling
I prefer to default to trunk, and require a -0.90 or -0.92 to delineate a different branch. Most patches should be against trunk, so let's optimize for the common case. -Todd +1

Re: startMiniDFSCluster and file permissions

2011-10-27 Thread Gary Helmling
This is fixed by HDFS-1560, though unfortunately it's not in 0.20.205.0. I've just been running with umask set to 022. On Thu, Oct 27, 2011 at 4:16 PM, lars hofhansl lhofha...@yahoo.com wrote: Hmm... checkDir eventually calls checkPermission and that does an equals check on the expected and

Re: Please welcome the dynamic K-squared duo, Kannan and Karthick, our latest hbase committers!

2011-10-31 Thread Gary Helmling
Welcome, Kannan and Karthik! Glad to have both of you on board! --gh On Mon, Oct 31, 2011 at 12:00 PM, Kannan Muthukkaruppan kan...@fb.com wrote: Thanks Stack and HBase team! Look fwd to doing exciting work as part of this team. Regards, Kannan Muthukkaruppan On 10/31/11 10:19 AM, Stack

Re: Please welcome the dynamic K-squared duo, Kannan and Karthick, our latest hbase committers!

2011-11-01 Thread Gary Helmling
We should make you guys t-shirts with some of these choice quotes! On Tue, Nov 1, 2011 at 10:28 AM, Stack st...@duboce.net wrote: 'K2 has variously been described as the awesome, killer and savage' On Tue, Nov 1, 2011 at 10:16 AM, Dhruba Borthakur dhr...@gmail.com wrote: Good stuff K^2.-

Re: An 0.92.0 Release Candidate this week?

2011-11-01 Thread Gary Helmling
I'd like to lobby for the security patches (HBASE-2742 and HBASE-3025) going in to 0.92. This may sound like a big change at this stage, but in reality the impact on current HBase core is pretty contained: * HBASE-2742 adds a security profile to the build, which only includes the security

Re: HBase with security build Was: An 0.92.0 Release Candidate this week?

2011-11-01 Thread Gary Helmling
But to make HBase with security available to wider audience, we really should create a Jenkins build which enables security profile. Completely agree. This should be pretty easy to do by cloning the existing build and adding -P security. Also, it would be nice if developers outside

Re: HBase with security build Was: An 0.92.0 Release Candidate this week?

2011-11-01 Thread Gary Helmling
Are we talking about a tarball that has both secure and insecure hbase or are we talking of two tarballs, insecure and secure? With the current POM/profile, we would do two tarballs, insecure and secure. The security profile adds the paths for the security classes to the source and

Re: Omid: Transactional Support for HBase

2011-11-04 Thread Gary Helmling
Hi Daniel, Cool stuff. You might want to link your project on the supporting projects wiki page as well: http://wiki.apache.org/hadoop/SupportingProjects Can't guarantee it'll drive a lot of traffic, but it's a good reference point for new users looking for options. --gh On Fri, Nov 4, 2011

Re: HBase .92 maven artifacts compiled against different releases of Hadoop

2011-11-11 Thread Gary Helmling
Some effort was put into restore and forward porting features to ensure HBase 0.90.x and Hadoop 0.20.205.0 can work together.  I recommend that one HBase release should be certified for one major release of Hadoop to reduce risk.   Perhaps when public Hadoop API are rock solid, then it will

Re: [unit tests] small tests vs. others

2011-11-14 Thread Gary Helmling
in a maven repo. Surefire works with maven, so it should be easy, JUnit does not, so there could be more work to do. I copy the dev list for information. Regards, N. -- Forwarded message -- From: Gary Helmling ghelml...@gmail.com Date: Thu, Oct 20, 2011 at 11:50 PM Subject: Re

Re: HBase .92 maven artifacts compiled against different releases of Hadoop

2011-11-16 Thread Gary Helmling
IMO a Hbase release should not include Hadoop JARs. Then, it would require a Hadoop installed to run. And it would use the Hadoop 'hadoop' CLI to start. I agree with not bundling Hadoop for RPM/deb packages where the package management system can take care of the dependencies. I'm -1 on

Security patches: HBASE-2742 and HBASE-3025

2011-11-17 Thread Gary Helmling
Hi all, I intend to commit the following patches for HBase security to the 0.92 branch and trunk at the end of day today: HBASE-2742: Provide a secure RPC engine for HBase https://reviews.apache.org/r/1991/ HBASE-3025: Coprocessor based simple access control https://reviews.apache.org/r/2041/

Re: Security patches: HBASE-2742 and HBASE-3025

2011-11-17 Thread Gary Helmling
Yes, the ZK authentication in 2418 is the final piece of the puzzle. On Thu, Nov 17, 2011 at 10:44 AM, Stack st...@duboce.net wrote: On Thu, Nov 17, 2011 at 10:33 AM, Gary Helmling ghelml...@gmail.com wrote: Hi all, I intend to commit the following patches for HBase security to the 0.92

Re: Build failed in Jenkins: HBase-TRUNK #2456

2011-11-18 Thread Gary Helmling
Results : Tests in error:  testLeaderSelection(org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager): KeeperErrorCode = Session expired for /test/TestZKLeaderManager There's a dumb assumption in the test that sometimes leads to ZKUtil.getData() being called on a ZooKeeperWatcher after

  1   2   3   >