Done for trunk on HBASE-6783. I will need to modify the tests script to unactivate it on HBase-0.94-security and HBase-TRUNK-on-Hadoop-2.0.0.
As a side note, there is a difference in behavior between the two modes: we don't see anymore a datanode error. I've seen it mentioned by JD somewhere, and I remember that the conclusion was "it's ok". Can't find it again, though. Conclusion; the TestFSErrorsExposed#testFullSystemBubblesFSErrors is not executed if short circuit is on. On Thu, Sep 13, 2012 at 11:07 PM, Andrew Purtell <[email protected]>wrote: > Yeah, on HBase-0.94-security > > On Thu, Sep 13, 2012 at 1:34 PM, n keywal <[email protected]> wrote: > > We've got two security builds on 0.94: HBase-0.94-security and > > HBase-0.94-security-on-Hadoop-23 > > We can change the setting on one of them? > > > > On Thu, Sep 13, 2012 at 10:20 PM, Andrew Purtell <[email protected] > >wrote: > > > >> Hey N, > >> > >> It makes sense to have a 0.94 build with -Psecurity enabled and local > >> shortcut disabled. What do you think? For trunk builds your points sound > >> good, and I have no opinion. > >> > >> On Thursday, September 13, 2012, n keywal wrote: > >> > >> > If I sum-up : > >> > > >> > We make it the default: > >> > 1) Documented as such (what JD wrote already + a reference to > HDFS-3907) > >> > 2) Set in the config we use for the unit tests. > >> > > >> > Andrew, you said that we should keep some test with it off ? May be we > >> > could do that in one of the existing build on the trunk, for example > >> > HBase-TRUNK-on-Hadoop-2.0.0? i.e.: We run the tests on trunk with > >> > short-circuit on; we run the test on hadoop-2 with short circuit off. > >> > > >> > I propose it this way because may be the tests will be faster with > >> > short-circuit on, and as well if it's the default production config it > >> > should be the one in the "main" test path. > >> > > >> > If that's ok, I can do test it on our test suite and do the jira for > the > >> > integration. > >> > > >> > > >> > > >> > > >> > On Thu, Sep 13, 2012 at 8:31 PM, Andrew Purtell <[email protected] > >> <javascript:;>> > >> > wrote: > >> > > >> > > That's https://issues.apache.org/jira/browse/HBASE-4867 > >> > > > >> > > On Thu, Sep 13, 2012 at 10:57 AM, Jesse Yates < > [email protected] > >> <javascript:;> > >> > > > >> > > wrote: > >> > > > There was a jira a while back (can't remember the number) that I > >> think > >> > > came > >> > > > from the Facebook fellas to have multiple hbase-site.xmls, with a > >> > naming > >> > > > scheme such that you can do layering of configurations. > >> > > > > >> > > > What ever happened to that? > >> > > > ------------------- > >> > > > Jesse Yates > >> > > > @jesse_yates > >> > > > jyates.github.com > >> > > > > >> > > > > >> > > > On Thu, Sep 13, 2012 at 10:44 AM, Andrew Purtell < > >> [email protected]<javascript:;> > >> > > >wrote: > >> > > > > >> > > >> Making it the default seems reasonable. > >> > > >> > >> > > >> For <= 0.94, the security profile has a separate hbase-site.xml > file > >> > > >> under test resources where there it could be defaulted as off. > >> Reason > >> > > >> is the local block sharing requires some shared group membership > >> > > >> between HBase and HDFS service accounts that might be > objectionable > >> in > >> > > >> a secure setting. So we should have test coverage with it off. > >> > > >> > >> > > >> On Thu, Sep 13, 2012 at 9:42 AM, n keywal <[email protected] > >> <javascript:;>> > >> > wrote: > >> > > >> > It's all in the "pretty much" :-) > >> > > >> > Should we make it the default (i.e. recommending it in the doc, > >> > > emitting > >> > > >> a > >> > > >> > warning if it's not set) for all installation ? > >> > > >> > Or would it make sense to separate the store and wal cases, > >> meaning > >> > > that > >> > > >> we > >> > > >> > need a change in hdfs to make this possible? > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Sep 13, 2012 at 6:30 PM, Jean-Daniel Cryans < > >> > > [email protected] <javascript:;> > >> > > >> >wrote: > >> > > >> > > >> > > >> >> On Thu, Sep 13, 2012 at 7:58 AM, n keywal <[email protected] > >> <javascript:;>> > >> > wrote: > >> > > >> >> > Hi, > >> > > >> >> > > >> > > >> >> > Is there any known counter indication for read short > circuit? > >> > > >> >> > I had a look at JD's presentation from January. It was > tested > >> vs. > >> > > >> HBase > >> > > >> >> as > >> > > >> >> > a whole; and still recommended for nearly all cases. > >> > > >> >> > >> > > >> >> Pretty much. > >> > > >> >> > >> > > >> >> > > >> > > >> >> > We could make it at least the default when reading HLog > files > >> > (even > >> > > >> if it > >> > > >> >> > seems it's a 'per client" config, there is may be something > to > >> > look > >> > > >> at as > >> > > >> >> > well)? > >> > > >> >> > >> > > >> >> You have to add 2 configs in hdfs-site.xml, which we can't do > >> from > >> > > >> HBase. > >> > > >> >> > >> > > >> >> J-D > >> > > >> >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> -- > >> > > >> Best regards, > >> > > >> > >> > > >> - Andy > >> > > >> > >> > > >> Problems worthy of attack prove their worth by hitting back. - > Piet > >> > > >> Hein (via Tom White) > >> > > >> > >> > > > >> > > > >> > > > >> > > -- > >> > > Best regards, > >> > > > >> > > - Andy > >> > > > >> > > Problems worthy of attack prove their worth by hitting back. - Piet > >> > > Hein (via Tom White) > >> > > > >> > > >> > >> > >> -- > >> Best regards, > >> > >> - Andy > >> > >> Problems worthy of attack prove their worth by hitting back. - Piet Hein > >> (via Tom White) > >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) >
