HBase shipping a generic framework for different interfaces is needed for ease of use for the users. +1 on the idea. Getting out the correct result for float values, positive and negative integers had to be taken care by the users or by using some wrappers.
This will help to solve that problem to a great extent. Regards Ram On Sun, Mar 17, 2013 at 10:54 PM, Mohamed Ibrahim <[email protected]>wrote: > I'm not a lawyer, but I think we're ok as long as it's in source code as > that is protected under freedom of speech in the US. See here ( > http://en.wikipedia.org/wiki/Cryptography ) under Export Control, the part > related to Bernstein v. United States . I don't know about binaries like > deb, but I can tell that we download binaries for browsers every day and > they use encryption in lots of places. I believe if there's any real issues > it would have surfaced up by now. > > As far as types in HBase, I think it is an excellent idea. I would suggest > to enable us to add a custom type, just like we can add our custom filters. > Some types that I had to code myself include CSV. There can be other custom > types that I need in the future, may be json, so the ability to add a > custom type might be a good feature. > > Thanks, > Mohamed > > > On Sun, Mar 17, 2013 at 1:12 PM, Andrew Purtell <[email protected]> > wrote: > > > > This then leads to another question... suppose Apache does add > encryption > > to Hadoop. While the Apache organization does have the proper paperwork > in > > place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ? > > > > Well I can't put that question aside since you've brought it up now > > twice and encryption feature candidates for Apache Hadoop and Apache > HBase > > are something I have been working on. Its a valid question but since as > you > > admit you don't know what you are talking about, perhaps stating > uninformed > > opinions can be avoided. Only the latter is what I object to. I think the > > short answer is as an Apache contributor I'm concerned about the Apache > > product. Downstream repackagers can take whatever action needed including > > changes, since it is open source, or feedback about it representing a > > hardship. At this point I have heard nothing like that. I work for Intel > > and can say we are good with it. > > > > On Sunday, March 17, 2013, Michael Segel wrote: > > > > > Its not a question of FUD, but that certain types of > > encryption/decryption > > > code falls under the munitions act. > > > See: http://www.fas.org/irp/offdocs/eo_crypt_9611_memo.htm > > > > > > Having said that, there is this: > > > http://www.bis.doc.gov/encryption/encfaqs6_17_02.html > > > > > > In short, I don't as a habit export/import encryption technology so I > am > > > not up to speed on the current state of the laws. > > > Which is why I have to question the current state of the US encryption > > > laws. > > > > > > This then leads to another question... suppose Apache does add > encryption > > > to Hadoop. While the Apache organization does have the proper paperwork > > in > > > place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, > etc ? > > > > > > But lets put that question aside. > > > > > > The point I was trying to make was that the core Sun JVM does support > MD5 > > > and SHA-1 out of the box, so that anyone running Hadoop and using the > > > 1.6_xx or the 1.7_xx versions of the JVM will have these packages. > > > > > > Adding hooks that use these classes are a no brainer. However, beyond > > > this... you tell me. > > > > > > -Mike > > > > > > On Mar 16, 2013, at 7:59 AM, Andrew Purtell <[email protected]> > wrote: > > > > > > > The ASF avails itself of an exception to crypto export which only > > > requires > > > > a bit of PMC housekeeping at release time. So "is not [ok]" is FUD. I > > > > humbly request we refrain from FUD here. See > > > > http://www.apache.org/dev/crypto.html. To the best of our knowledge > we > > > > expect this to continue, though the ASF has not updated this policy > yet > > > for > > > > recent regulation updates. > > > > > > > > On Saturday, March 16, 2013, Michel Segel wrote: > > > > > > > >> I also want to add that you could add MD5 and SHA-1, but I'd check > on > > us > > > >> laws... I think these are ok, however other encryption/decryption > code > > > is > > > >> not. > > > >> > > > >> They are part of the std sun java libraries ... > > > >> > > > >> Sent from a remote device. Please excuse any typos... > > > >> > > > >> Mike Segel > > > >> > > > >> On Mar 16, 2013, at 7:18 AM, Michel Segel < > [email protected]> > > > >> wrote: > > > >> > > > >>> Isn't that what you get through add on frameworks like TSDB and > Kiji > > ? > > > >> Maybe not on the client side, but frameworks that extend HBase... > > > >>> > > > >>> Sent from a remote device. Please excuse any typos... > > > >>> > > > >>> Mike Segel > > > >>> > > > >>> On Mar 16, 2013, at 12:45 AM, lars hofhansl <[email protected]> > > wrote: > > > >>> > > > >>>> I think generally we should keep HBase a byte[] based key value > > store. > > > >>>> What we should add to HBase are tools that would allow client side > > > apps > > > >> (or libraries) to built functionality on top of plain HBase. > > > >>>> > > > >>>> Serialization that maintains a correct semantic sort order is > > > important > > > >> as a building block, so is code that can build up correctly > serialized > > > and > > > >> sortable compound keys, as well as hashing algorithms. > > > >>>> > > > >>>> Where I would draw the line is adding types to HBase itself. As > long > > > as > > > >> one can write a client, or Filters, or Coprocessors with the tools > > > provided > > > >> by HBase we're good. Higher level functionality can then be built of > > on > > > top > > > >> of HBase. > > > >>>> > > > >>>> > > > >>>> For example, maybe we need to add better access API to the HBase > WAL > > > in > > > >> order to have an external library implement idempotent transactions > > > (which > > > >> can be used to implement 2ndary indexes). > > > >>>> Maybe some other primitives have to be exposed in order to allow > an > > > >> external library to implement full transactions. > > > >>>> Or we might need a statistics framework (such as the one that > Jesse > > is > > > >> working on). > > > >>>> > > > >>>> These are all building blocks that do not presume specific access > > > >> patterns or clients, but can be used to implement them. > > > >>>> > > > >>>> > > > >>>> As usual, just my $0.02. > > > >>>> > > > >>>> -- Lars > > > >>>> > > > >>>> > > > >>>> > > > >>>> ________________________________ > > > >>>> From: Nick Dimiduk <[email protected]> > > > >>>> To: [email protected] > > > >>>> Sent: Friday, March 15, 2013 10:57 AM > > > >>>> Subject: Re: HBase type support > > > >>>> > > > >>>> I'm talking about MD5, SHA1, etc. It's something explicitly > > mentioned > > > >>>> in HBASE-7221. > > > >>>> > > > >>>> On Fri, Mar 15, 2013 at 10:55 AM, James Taylor < > > > [email protected] > > > >>> wrote: > > > >>>> > > > >>>>> Hi Nick, > > > >>>>> What do you mean by "hashing algorithms"? > > > >>>>> Thanks, > > > >>>>> James > > > >>>>> > > > >>>>> > > > >>>>> On 03/15/2013 10:11 AM, Nick Dimiduk wrote: > > > >>>>> > > > >>>>>> Hi David, > > > >>>>>> > > > >>>>>> Native support for a handful of hashing algorithms has also been > > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > >
