I've a question about indexing/searching techniques in relation to document level security. In planning a system that has, let's say, about 1million search documents with about 100 search fields each. Most of them unstored to keep the index size low, because some of them can contain some kilobytes and some of them several hundred kilobytes. Two of these search fields are for permission checking, where i keep the explicitely allowed and explicitely disallowed users and usergroups. (usergroups can be in a hierarchical structure with permission inheritance)
So when a user searches in the system, his user id, and ids of usergroup memberships are added as a filter query in my application logic before the query is sent to solr. So far so good for the searching part. But the problem is, that the permissions can be changed by administrators of that system, requiring to re-index the two permission search fields. first idea: Partial updates of index entries is not possible, so i need to fetch all the 1million documents from a database to do a re-indexing just because some permissions changed. The fetching process is rather expensive and requires more then 14hours. I am sure that this can be optimized of course, but i would rather try to avoid a whole re-indexing of all content. second idea: Another idea would be to store just the permissions in one small and fast to update index and all the other stuff in the other huge and not so often updated index. But i didn't find any possibilities to combine these two indices in one query. Is that even possible? Does somebody have experience with these topics or give advice how to solve that case properly? Thanks in advance. -- View this message in context: http://lucene.472066.n3.nabble.com/document-level-security-indexing-searching-techniques-tp946528p946528.html Sent from the Solr - User mailing list archive at Nabble.com.
