If you COULD solve your problem by indexing 'public', or other tokens from a limited vocabulary of document roles, in a field -- then I'd definitely suggest you look into doing that, rather than doing odd things with Solr instead. If the only barrier is not currently having sufficient logic at the indexing stage to do that, then it is going to end up being a lot less of a headache in the long term to simply add a layer at the indexing stage to add that in, then trying to get Solr to do things outside of it's, well, 'comfort zone'.
Of course, depending on your requirements, it might not be possible to do that, maybe you can't express the semantics in terms of a limited set of roles applied to documents. And then maybe your best option really is sending an up to 2k element list (not exactly the same list every time, presumably) of acceptable documents to Solr with every query, and maybe you can get that to work reasonably. Depending on how many different complete lists of documents you have, maybe there's a way to use Solr caches effectively in that situation, or maybe that's not even neccesary since lookup by unique id should be pretty quick anyway, not really sure. But if the semantics are possible, much better to work with Solr rather than against it, it's going to take a lot less tinkering to get Solr to perform well if you can just send an fq=role:public or something, instead of a list of document IDs. You won't need to worry about it, it'll just work, because you know you're having Solr do what it's built to do. Totally worth a bit of work to add a logic layer at the indexing stage. IMO. ________________________________________ From: Erick Erickson [erickerick...@gmail.com] Sent: Saturday, January 22, 2011 4:50 PM To: solr-user@lucene.apache.org Subject: Re: api key filtering 1024 is the default number, it can be increased. See MaxBooleanClauses in solrconfig.xml This shouldn't be a problem with 2K clauses, but expanding it to tens of thousands is probably a mistake (but test to be sure). Best Erick On Sat, Jan 22, 2011 at 3:50 PM, Matt Mitchell <goodie...@gmail.com> wrote: > Hey thanks I'll definitely have a read. The only problem with this though, > is that our api is a thin layer of app-code, with solr only (no db), we > index data from our sql db into solr, and push the index off for > consumption. > > The only other idea I had was to send a list of the allowed document ids > along with every solr query, but then I'm sure I'd run into a filter query > limit. Each key could be associated with up to 2k documents, so that's 2k > values in an fq which would probably be too many for lucene (I think its > limit 1024). > > Matt > > On Sat, Jan 22, 2011 at 3:40 PM, Dennis Gearon <gear...@sbcglobal.net > >wrote: > > > The only way that you would have that many api keys per record, is if one > > of > > them represented 'public', right? 'public' is a ROLE. Your answer is to > use > > RBAC > > style techniques. > > > > > > Here are some links that I have on the subject. What I'm thinking of > doing > > is: > > Sorry for formatting, Firefox is freaking out. I cut and pasted these > from > > an > > email from my sent box. I hope the links came out. > > > > > > Part 1 > > > > > > > http://www.xaprb.com/blog/2006/08/16/how-to-build-role-based-access-control-in-sql/ > > > > > > Part2 > > Role-based access control in SQL, part 2 at Xaprb > > > > > > > > > > > > ACL/RBAC Bookmarks ALL > > > > UserRbac - symfony - Trac > > A Role-Based Access Control (RBAC) system for PHP > > Appendix C: Task-Field Access > > Role-based access control in SQL, part 2 at Xaprb > > PHP Access Control - PHP5 CMS Framework Development | PHP Zone > > Linux file and directory permissions > > MySQL :: MySQL 5.0 Reference Manual :: C.5.4.1 How to Reset the Root > > Password > > per RECORD/Entity permissions? - symfony users | Google Groups > > Special Topics: Authentication and Authorization | The Definitive Guide > to > > Yii | > > Yii Framework > > > > att.net Mail (gear...@sbcglobal.net) > > Solr - User - Modelling Access Control > > PHP Generic Access Control Lists > > Row-level Model Access Control for CakePHP « some flot, some jet > > Row-level Model Access Control for CakePHP « some flot, some jet > > Yahoo! GeoCities: Get a web site with easy-to-use site building tools. > > Class that acts as a client to a JSON service : JSON « GWT « Java > > Juozas Kaziukėnas devBlog > > Re: [symfony-users] Implementing an existing ACL API in symfony > > php - CakePHP ACL Database Setup: ARO / ACO structure? - Stack Overflow > > W3C ACL System > > makeAclTables.sql > > SchemaWeb - Classes And Properties - ACL Schema > > Reardon's Ruminations: Spring Security ACL Schema for Oracle > > trunk/modules/auth/libraries/Khacl.php | Source/SVN | Assembla > > Acl.php - kohana-mptt - Project Hosting on Google Code > > Asynchronous JavaScript Technology and XML (Ajax) With the Java Platform > > The page cannot be found > > > > > > Dennis Gearon > > > > > > Signature Warning > > ---------------- > > It is always a good idea to learn from your own mistakes. It is usually a > > better > > idea to learn from others’ mistakes, so you do not have to make them > > yourself. > > from 'http://blogs.techrepublic.com.com/security/?p=4501&tag=nl.e036' > > > > > > EARTH has a Right To Life, > > otherwise we all die. > > > > > > > > ----- Original Message ---- > > From: Matt Mitchell <goodie...@gmail.com> > > To: solr-user@lucene.apache.org > > Sent: Sat, January 22, 2011 11:48:22 AM > > Subject: api key filtering > > > > Just wanted to see if others are handling this in some special way, but I > > think this is pretty simple. > > > > We have a database of api keys that map to "allowed" db records. I'm > > planning on indexing the db records into solr, along with their api keys > in > > an indexed, non-stored, multi-valued field. Then, to query for docs that > > belong to a particular api key, they'll be queried using a filter query > on > > api_key. > > > > The only concern of mine is that, what if we end up with 100k api_keys? > > Would it be a problem to have 100k non-stored keys in each document? We > > have > > about 500k documents total. > > > > Matt > > > > >
