Regarding HBase 7941 - client API support.. In the past year I wrote a lot of client code for HBase and which led to writing a helper class for my specific needs, and since it was brought up here I guess I'm not the only one who did something similar...I personally like the idea brought up by Nick in the JIRA of using some kind of <SerializationType> interface and having HBase shipped with primitive support - and anyone who wants will implement for their needs. Same idea ad Hadoop's Writable interface - shipping with IntWritable, LongWritable, etc.
Anyway I'd be happy to help. On Sun, Mar 17, 2013 at 7:12 PM, Andrew Purtell <[email protected]> wrote: > > This then leads to another question... suppose Apache does add encryption > to Hadoop. While the Apache organization does have the proper paperwork in > place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ? > > Well I can't put that question aside since you've brought it up now > twice and encryption feature candidates for Apache Hadoop and Apache HBase > are something I have been working on. Its a valid question but since as you > admit you don't know what you are talking about, perhaps stating uninformed > opinions can be avoided. Only the latter is what I object to. I think the > short answer is as an Apache contributor I'm concerned about the Apache > product. Downstream repackagers can take whatever action needed including > changes, since it is open source, or feedback about it representing a > hardship. At this point I have heard nothing like that. I work for Intel > and can say we are good with it. > > On Sunday, March 17, 2013, Michael Segel wrote: > > > Its not a question of FUD, but that certain types of > encryption/decryption > > code falls under the munitions act. > > See: http://www.fas.org/irp/offdocs/eo_crypt_9611_memo.htm > > > > Having said that, there is this: > > http://www.bis.doc.gov/encryption/encfaqs6_17_02.html > > > > In short, I don't as a habit export/import encryption technology so I am > > not up to speed on the current state of the laws. > > Which is why I have to question the current state of the US encryption > > laws. > > > > This then leads to another question... suppose Apache does add encryption > > to Hadoop. While the Apache organization does have the proper paperwork > in > > place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ? > > > > But lets put that question aside. > > > > The point I was trying to make was that the core Sun JVM does support MD5 > > and SHA-1 out of the box, so that anyone running Hadoop and using the > > 1.6_xx or the 1.7_xx versions of the JVM will have these packages. > > > > Adding hooks that use these classes are a no brainer. However, beyond > > this... you tell me. > > > > -Mike > > > > On Mar 16, 2013, at 7:59 AM, Andrew Purtell <[email protected]> wrote: > > > > > The ASF avails itself of an exception to crypto export which only > > requires > > > a bit of PMC housekeeping at release time. So "is not [ok]" is FUD. I > > > humbly request we refrain from FUD here. See > > > http://www.apache.org/dev/crypto.html. To the best of our knowledge we > > > expect this to continue, though the ASF has not updated this policy yet > > for > > > recent regulation updates. > > > > > > On Saturday, March 16, 2013, Michel Segel wrote: > > > > > >> I also want to add that you could add MD5 and SHA-1, but I'd check on > us > > >> laws... I think these are ok, however other encryption/decryption code > > is > > >> not. > > >> > > >> They are part of the std sun java libraries ... > > >> > > >> Sent from a remote device. Please excuse any typos... > > >> > > >> Mike Segel > > >> > > >> On Mar 16, 2013, at 7:18 AM, Michel Segel <[email protected]> > > >> wrote: > > >> > > >>> Isn't that what you get through add on frameworks like TSDB and Kiji > ? > > >> Maybe not on the client side, but frameworks that extend HBase... > > >>> > > >>> Sent from a remote device. Please excuse any typos... > > >>> > > >>> Mike Segel > > >>> > > >>> On Mar 16, 2013, at 12:45 AM, lars hofhansl <[email protected]> > wrote: > > >>> > > >>>> I think generally we should keep HBase a byte[] based key value > store. > > >>>> What we should add to HBase are tools that would allow client side > > apps > > >> (or libraries) to built functionality on top of plain HBase. > > >>>> > > >>>> Serialization that maintains a correct semantic sort order is > > important > > >> as a building block, so is code that can build up correctly serialized > > and > > >> sortable compound keys, as well as hashing algorithms. > > >>>> > > >>>> Where I would draw the line is adding types to HBase itself. As long > > as > > >> one can write a client, or Filters, or Coprocessors with the tools > > provided > > >> by HBase we're good. Higher level functionality can then be built of > on > > top > > >> of HBase. > > >>>> > > >>>> > > >>>> For example, maybe we need to add better access API to the HBase WAL > > in > > >> order to have an external library implement idempotent transactions > > (which > > >> can be used to implement 2ndary indexes). > > >>>> Maybe some other primitives have to be exposed in order to allow an > > >> external library to implement full transactions. > > >>>> Or we might need a statistics framework (such as the one that Jesse > is > > >> working on). > > >>>> > > >>>> These are all building blocks that do not presume specific access > > >> patterns or clients, but can be used to implement them. > > >>>> > > >>>> > > >>>> As usual, just my $0.02. > > >>>> > > >>>> -- Lars > > >>>> > > >>>> > > >>>> > > >>>> ________________________________ > > >>>> From: Nick Dimiduk <[email protected]> > > >>>> To: [email protected] > > >>>> Sent: Friday, March 15, 2013 10:57 AM > > >>>> Subject: Re: HBase type support > > >>>> > > >>>> I'm talking about MD5, SHA1, etc. It's something explicitly > mentioned > > >>>> in HBASE-7221. > > >>>> > > >>>> On Fri, Mar 15, 2013 at 10:55 AM, James Taylor < > > [email protected] > > >>> wrote: > > >>>> > > >>>>> Hi Nick, > > >>>>> What do you mean by "hashing algorithms"? > > >>>>> Thanks, > > >>>>> James > > >>>>> > > >>>>> > > >>>>> On 03/15/2013 10:11 AM, Nick Dimiduk wrote: > > >>>>> > > >>>>>> Hi David, > > >>>>>> > > >>>>>> Native support for a handful of hashing algorithms has also been > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
