+1 on what Mikhail said below. On Wed, Mar 7, 2012 at 11:42 AM, Mikhail Bautin < [email protected]> wrote:
> The current support for multiple versions of HDFS is in my opinion actually > one of the strengths of HBase, and the project will lose that advantage if > we cut support for earlier versions of Hadoop. I think HBase should only > require the simplest possible universally available subset of HDFS API, and > security should be an optional feature, discovered through reflection or > enabled in some other ways. > > We have a custom version of Hadoop at Facebook that is not planning to > implement security any time soon. This version of Hadoop runs underneath > what we believe to be some of the largest existing production HBase > deployments. We are currently running the 0.89-fb version of HBase in > production, but are considering moving to a more recent version of HBase at > some point, and it would be great to be able to do that independently of > changing the underlying Hadoop distribution for migration complexity > reasons. Currently we are able to run public HBase trunk on our version of > Hadoop, but once in a while we have to satisfy new dependences on Hadoop > features that are added to HBase. If the changes proposed in this thread > happen, we would have to pull in a lot more security-related dependencies > into our version of Hadoop and, most likely, implement a lot of no-op > stubs. However, that may not be a trivial project, and it certainly would > not add any clarity or value to our Hadoop codebase or HBase / HDFS > interaction. > > I imagine there are other custom flavors of Hadoop out there where HBase > support would be desirable. For example, does MapR implement the same > security API as Hadoop 1.0.0 does? Restricting HBase to a smaller subset of > Hadoop versions complicates life for existing users, and makes HBase a less > likely choice for new users, who could go with something like Hypertable > where they have an extra abstraction layer between the database and the > underlying distributed file system implementation. > > Thanks, > --Mikhail > > On Wed, Mar 7, 2012 at 10:20 AM, Devaraj Das <[email protected]> wrote: > > > Given that the token/ugi APIs are being used in other ecosystem > components > > too (like Hive, HCatalog & Oozie), and in general, that security model > will > > probably hold for other projects too, I think that its not an unfair > > expectation from Hadoop that it should maintain compatibility on > UGI/Token* > > interfaces (*smile*). > > > > On Mar 6, 2012, at 11:57 AM, Arun C Murthy wrote: > > > > > Andy - could you please start a discussion? > > > > > > We could, at the very least, mark UGI as LimitedPrivate for HBase and > > work with you guys to maintain compatibility for the future. Makes sense? > > > > > > thanks, > > > Arun > > > > > > On Mar 6, 2012, at 10:21 AM, Andrew Purtell wrote: > > > > > >> After that, I believe we can merge the security sources in. However we > > may have an issue going forward because UGI is an unstable/private API. > > Needs sorting out with core at some point. > > >> > > >> Best regards, > > >> > > >> - Andy > > >> > > >> > > >> On Mar 6, 2012, at 9:55 AM, Stack <[email protected]> wrote: > > >> > > >>> On Tue, Mar 6, 2012 at 9:10 AM, Andrew Purtell <[email protected]> > > wrote: > > >>>> ...however we can't easily build a single artifact because the > secure > > RPC engine, as it interacts with the Hadoop auth framework, must use > > UserGroupInformation. > > >>>> > > >>> > > >>> OK. So security story needs a bit of work. Sounds like we have > > >>> enough votes though to require hadoop 1.0.0 at least in 0.96. > > >>> > > >>> St.Ack > > > > > > -- > > > Arun C. Murthy > > > Hortonworks Inc. > > > http://hortonworks.com/ > > > > > > > > > > >
