You could implement a good solution with the underlying Lucene ParallelReader http://lucene.apache.org/java/3_0_2/api/core/org/apache/lucene/index/ParallelReader.html Keep the 100 search fields - 'static' info - in one index, the permissions info in another index that gets updated when the permissions change.
Does SOLR expose this kind of functionality? -Glen Newton http://zzzoot.blogspot.com/ http://zzzoot.blogspot.com/2009/07/project-torngat-building-large-scale.html On 7 July 2010 00:38, RL <rl.subscri...@gmail.com> wrote: > > I've a question about indexing/searching techniques in relation to document > level security. > In planning a system that has, let's say, about 1million search documents > with about 100 search fields each. Most of them unstored to keep the index > size low, because some of them can contain some kilobytes and some of them > several hundred kilobytes. Two of these search fields are for permission > checking, where i keep the explicitely allowed and explicitely disallowed > users and usergroups. (usergroups can be in a hierarchical structure with > permission inheritance) > > So when a user searches in the system, his user id, and ids of usergroup > memberships are added as a filter query in my application logic before the > query is sent to solr. So far so good for the searching part. > > But the problem is, that the permissions can be changed by administrators of > that system, requiring to re-index the two permission search fields. > > first idea: > Partial updates of index entries is not possible, so i need to fetch all the > 1million documents from a database to do a re-indexing just because some > permissions changed. The fetching process is rather expensive and requires > more then 14hours. I am sure that this can be optimized of course, but i > would rather try to avoid a whole re-indexing of all content. > > second idea: > Another idea would be to store just the permissions in one small and fast to > update index and all the other stuff in the other huge and not so often > updated index. But i didn't find any possibilities to combine these two > indices in one query. Is that even possible? > > > Does somebody have experience with these topics or give advice how to solve > that case properly? > Thanks in advance. > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/document-level-security-indexing-searching-techniques-tp946528p946528.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -
