On 9/5/2015 5:06 AM, Adam Retter wrote: > I wondered if there is any facility already existing in Lucene for > encrypting the values stored into the index and still being able to > search them? > > If not, I wondered if anyone could tell me if this is impossible to > implement, and if not to point me perhaps in the right direction? > > I imagine that just the text values and document fields to index (and > optionally store) in the index would be either encrypted on the fly by > Lucene using perhaps a public/private key mechanism. When a user issues > a search query to Lucene they would also provide a key so that Lucene > can decrypt the values as necessary to try and answer their query.
I think you could probably add transparent encryption/decryption at the Lucene level in a custom codec. That probably has implications for being able to read the older index when it's time to upgrade Lucene, with a complete reindex being the likely solution. Others will need to confirm ... I'm not very familiar with Lucene code, I'm here for Solr. Any verification of user identity/permission is probably best done in your own code, before it makes the Lucene query, and wouldn't necessarily be related to the encryption. Requirements like this are usually driven by paranoid customers or product managers. I think that when you really start to examine what an attacker has to do to actually reach the unencrypted information (Lucene index in this case), they already have acquired so much access that the system is completely breached and it won't matter what kind of encryption is added. I find many of these requirements to be silly, and put an incredible burden on admin and developer resources with little or no benefit. Here's an example of similar customer encryption requirement which I encountered recently: We have a web application that has three "hops" involved. A user talks to a load balancer, which talks to Apache, where the connection is then proxied to a Tomcat server with the AJP protocol. The customer wanted all three hops encrypted. The first hop was already encrypted, the second was easy, but the third proved to be very difficult. Finally we decided that we did not need load balancing on that last hop, and it could simply talk to localhost, eliminating the need to encrypt it. The customer was worried about an attacker sniffing the traffic on the LAN and seeing details like passwords. I consider this to be an insane requirement. In order to sniff that traffic, the attacker would need one of two things: Root access on a server, or physical access to the infrastructure. Physical access can be escalated to root access if you know what you're doing. Once someone has either of those things, encrypted traffic won't matter, they will be able to learn anything they need or do any damage they desire, without even needing to sniff the traffic. Thanks, Shawn --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
