Encrypting the _tokens_ inevitably leads to reduced capabilities BTW. Trivial example: I have these tokens in my index run runner running runs
Any non-trivial encryption algorithm will not encrypt the first three letters "run" identically in all three so searching for run* simply won't work. As you can see, there's quite a bit of back-and-forth with that JIRA and it is pretty much been abandoned. Best, Erick On Thu, Aug 10, 2017 at 11:17 PM, Kumaran Ramasubramanian <kums....@gmail.com> wrote: > Hi Ishan, thank you :-) > > - > - > Kumaran R > > > > On Mon, Aug 7, 2017 at 10:53 PM, Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> Harry Ochiai (Hitachi) has some index encryption solution, >> https://www.slideshare.net/maggon/securing-solr-search-data-in-the-cloud >> I think it is proprietary, but I'm not sure. Maybe more googling might help >> find the exact page where his solution is described. >> >> On Mon, Aug 7, 2017 at 9:59 PM, Kumaran Ramasubramanian < >> kums....@gmail.com> >> wrote: >> >> > Hi Erick, i want to encrypt some fields of an document which has personal >> > identifiable information ( both indexed and stored data)... for eg: >> email, >> > mobilenumber etc.. i am able to find LUCENE-6966 alone while googling >> it.. >> > any related pointers in solr or latest lucene version? >> > >> > >> > - >> > - >> > Kumaran R >> > >> > On Mon, Aug 7, 2017 at 9:52 PM, Erick Erickson <erickerick...@gmail.com> >> > wrote: >> > >> > > No, since you haven't defined what you want to encrypt, what your >> > > requirements are, what you hope to get out of "encryption" etc. >> > > >> > > Put the index on an encrypting filesystem and forget about it if you >> > > possibly can, because anything else is a significant amount of work. >> > > To encrypt the searchable tokens on a per-user basis in memory is a >> > > _lot_ of work. It depends on your security needs. >> > > >> > > Otherwise, as I said, please ask specific questions as the topic is >> > > quite large, much too large to conduct a seminar through the user's >> > > list. >> > > >> > > Best, >> > > Erick >> > > >> > > On Mon, Aug 7, 2017 at 9:07 AM, Kumaran Ramasubramanian >> > > <kums....@gmail.com> wrote: >> > > > Hi Erick, >> > > > >> > > > Thanks for the information. Any pointers about encryption options >> > in >> > > > solr? >> > > > >> > > > >> > > > -- >> > > > Kumaran R >> > > > >> > > > >> > > > >> > > > On Mon, Aug 7, 2017 at 9:17 PM, Erick Erickson < >> > erickerick...@gmail.com> >> > > > wrote: >> > > > >> > > >> Encryption in Solr has a bunch of ramifications. Do you care about >> > > >> >> > > >> - encryption at rest or in memory? >> > > >> - encrypting the _searchable_ tokens? >> > > >> - encrypting the searchable tokens per-user? >> > > >> - encrypting the stored data (which a filter won't do BTW). >> > > >> >> > > >> It's actually a fairly complex topic the discussion at LUCENE-6966 >> > > >> outlines much of it. Please ask specific questions as you research >> the >> > > >> topic. One per-user encryption package that I know of is by Hitachi >> > > >> Solutions (commercial) and it explicitly does _not_ support, for >> > > >> instance, wildcards (there are other limitations too). See: >> > > >> http://www.hitachi-solutions.com/securesearch/ >> > > >> >> > > >> Most of the time when people ask for encryption they soon discover >> > > >> it's much more difficult than they imagine and settle for just >> putting >> > > >> the indexes on an encrypting file system. When they move beyond that >> > > >> it gets complex and you'd be well advised to consult with Solr >> > > >> security experts. >> > > >> >> > > >> Best, >> > > >> Erick >> > > >> >> > > >> On Sun, Aug 6, 2017 at 11:30 PM, Kumaran Ramasubramanian >> > > >> <kums....@gmail.com> wrote: >> > > >> > Hi All, >> > > >> > >> > > >> > >> > > >> > After looking at all below discussions, i have one doubt which may >> > be >> > > >> silly >> > > >> > or novice but i want to throw this to lucene user list. >> > > >> > >> > > >> > if we have encryption layer included in our analyzer's flow of >> > filters >> > > >> like >> > > >> > EncryptionFilter to control field-level encryption. what are the >> > > >> > consequences ? am i missing anything basic? >> > > >> > >> > > >> > Thanks in advance.. >> > > >> > >> > > >> > >> > > >> > Related links: >> > > >> > >> > > >> > https://issues.apache.org/jira/browse/LUCENE-2228 : AES Encrypted >> > > >> Directory >> > > >> > - in lucene 3.x >> > > >> > >> > > >> > https://issues.apache.org/jira/browse/LUCENE-6966 : Codec for >> > > >> index-level >> > > >> > encryption - at codec level, to have control on which column / >> field >> > > have >> > > >> > personal identifiable information >> > > >> > >> > > >> > https://security.stackexchange.com/questions/ >> > > 111153/is-a-lucene-search- >> > > >> index-effectively-a-backdoor-for-field-level-encryption >> > > >> > >> > > >> > >> > > >> > A decent encrypting algorithm will not produce, say, the same >> first >> > > >> portion >> > > >> >> for two tokens that start with the same letters. So wildcard >> > searches >> > > >> won't >> > > >> >> work. Consider "runs", "running", "runner". A search on "run*" >> > would >> > > be >> > > >> >> expected to match all three, but wouldn't unless the encryption >> > were >> > > so >> > > >> >> trivial as to be useless. Similar issues arise with sorting. >> "More >> > > Like >> > > >> >> This" would be unreliable. There are many other features of a >> > robust >> > > >> search >> > > >> >> engine that would be impacted, and an index with encrypted terms >> > > would >> > > >> be >> > > >> >> useful for only exact matches, which usually results in a poor >> > search >> > > >> >> experience. >> > > >> > >> > > >> > >> > > >> > https://stackoverflow.com/questions/36604551/adding- >> > > >> encryption-to-solr-lucene-indexes >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > -- >> > > >> > Kumaran R >> > > >> >> > > >> ------------------------------------------------------------ >> --------- >> > > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> > > >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > > >> >> > > >> >> > > >> > > --------------------------------------------------------------------- >> > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> > > For additional commands, e-mail: java-user-h...@lucene.apache.org >> > > >> > > >> > >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
