Re: hybrid document routing
Cool! ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Tue, Aug 11, 2020 at 9:55 AM Joel Bernstein wrote: > SOLR-14728 supports sub-second performance on joins with more than 1 > million values from the from index. Nice for access control. > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Tue, Aug 11, 2020 at 9:49 AM Joel Bernstein wrote: > >> This ticket will shed some light: >> >> https://issues.apache.org/jira/browse/SOLR-14728 >> >> >> I think I'm planning using a different approach to distribute tha ACL's >> to all shards. >> >> >> >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> >> On Tue, Aug 11, 2020 at 1:18 AM Gus Heck wrote: >> >>> Sounds like complex ACLs based on group memberships that use graph >>> queries ? that would require local ACL's... >>> >>> On Mon, Aug 10, 2020 at 5:56 PM Ishan Chattopadhyaya < >>> ichattopadhy...@gmail.com> wrote: >>> This seems like an XY problem. Would it be possible to describe the original problem that led you to this solution (in the prototype)? Also, do you think folks at solr-users@ list would have more ideas related to this usecase and cross posting there would help? On Tue, 11 Aug, 2020, 1:43 am David Smiley, wrote: > Are you sure you need the docs in the same shard when maybe you could > assume a core exists on each node and then do a query-time join? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein > wrote: > >> I have a situation where I'd like to have the standard compositeId >> router in place for a collection. But, I'd like certain documents (ACL >> documents) to be duplicated on each shard in the collection. To achieve >> the >> level of access control performance and scalability I'm looking for I >> need >> the ACL records to be in the same core as the main documents. >> >> I put together a prototype where the compositeId router accepted >> implicit routing parameters and it worked in my testing. Before I open a >> ticket suggesting this approach I wonder what other people thought the >> best >> approach would be to accomplish this goal. >> >> >> >>> >>> -- >>> http://www.needhamsoftware.com (work) >>> http://www.the111shift.com (play) >>> >>
Re: hybrid document routing
SOLR-14728 supports sub-second performance on joins with more than 1 million values from the from index. Nice for access control. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Aug 11, 2020 at 9:49 AM Joel Bernstein wrote: > This ticket will shed some light: > > https://issues.apache.org/jira/browse/SOLR-14728 > > > I think I'm planning using a different approach to distribute tha ACL's to > all shards. > > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > > On Tue, Aug 11, 2020 at 1:18 AM Gus Heck wrote: > >> Sounds like complex ACLs based on group memberships that use graph >> queries ? that would require local ACL's... >> >> On Mon, Aug 10, 2020 at 5:56 PM Ishan Chattopadhyaya < >> ichattopadhy...@gmail.com> wrote: >> >>> This seems like an XY problem. Would it be possible to describe the >>> original problem that led you to this solution (in the prototype)? Also, do >>> you think folks at solr-users@ list would have more ideas related to >>> this usecase and cross posting there would help? >>> >>> On Tue, 11 Aug, 2020, 1:43 am David Smiley, wrote: >>> Are you sure you need the docs in the same shard when maybe you could assume a core exists on each node and then do a query-time join? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein wrote: > I have a situation where I'd like to have the standard compositeId > router in place for a collection. But, I'd like certain documents (ACL > documents) to be duplicated on each shard in the collection. To achieve > the > level of access control performance and scalability I'm looking for I need > the ACL records to be in the same core as the main documents. > > I put together a prototype where the compositeId router accepted > implicit routing parameters and it worked in my testing. Before I open a > ticket suggesting this approach I wonder what other people thought the > best > approach would be to accomplish this goal. > > > >> >> -- >> http://www.needhamsoftware.com (work) >> http://www.the111shift.com (play) >> >
Re: hybrid document routing
This ticket will shed some light: https://issues.apache.org/jira/browse/SOLR-14728 I think I'm planning using a different approach to distribute tha ACL's to all shards. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Aug 11, 2020 at 1:18 AM Gus Heck wrote: > Sounds like complex ACLs based on group memberships that use graph queries > ? that would require local ACL's... > > On Mon, Aug 10, 2020 at 5:56 PM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> This seems like an XY problem. Would it be possible to describe the >> original problem that led you to this solution (in the prototype)? Also, do >> you think folks at solr-users@ list would have more ideas related to >> this usecase and cross posting there would help? >> >> On Tue, 11 Aug, 2020, 1:43 am David Smiley, wrote: >> >>> Are you sure you need the docs in the same shard when maybe you could >>> assume a core exists on each node and then do a query-time join? >>> >>> ~ David Smiley >>> Apache Lucene/Solr Search Developer >>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein >>> wrote: >>> I have a situation where I'd like to have the standard compositeId router in place for a collection. But, I'd like certain documents (ACL documents) to be duplicated on each shard in the collection. To achieve the level of access control performance and scalability I'm looking for I need the ACL records to be in the same core as the main documents. I put together a prototype where the compositeId router accepted implicit routing parameters and it worked in my testing. Before I open a ticket suggesting this approach I wonder what other people thought the best approach would be to accomplish this goal. > > -- > http://www.needhamsoftware.com (work) > http://www.the111shift.com (play) >
Re: hybrid document routing
Sounds like complex ACLs based on group memberships that use graph queries ? that would require local ACL's... On Mon, Aug 10, 2020 at 5:56 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > This seems like an XY problem. Would it be possible to describe the > original problem that led you to this solution (in the prototype)? Also, do > you think folks at solr-users@ list would have more ideas related to this > usecase and cross posting there would help? > > On Tue, 11 Aug, 2020, 1:43 am David Smiley, wrote: > >> Are you sure you need the docs in the same shard when maybe you could >> assume a core exists on each node and then do a query-time join? >> >> ~ David Smiley >> Apache Lucene/Solr Search Developer >> http://www.linkedin.com/in/davidwsmiley >> >> >> On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein >> wrote: >> >>> I have a situation where I'd like to have the standard compositeId >>> router in place for a collection. But, I'd like certain documents (ACL >>> documents) to be duplicated on each shard in the collection. To achieve the >>> level of access control performance and scalability I'm looking for I need >>> the ACL records to be in the same core as the main documents. >>> >>> I put together a prototype where the compositeId router accepted >>> implicit routing parameters and it worked in my testing. Before I open a >>> ticket suggesting this approach I wonder what other people thought the best >>> approach would be to accomplish this goal. >>> >>> >>> -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
Re: hybrid document routing
This seems like an XY problem. Would it be possible to describe the original problem that led you to this solution (in the prototype)? Also, do you think folks at solr-users@ list would have more ideas related to this usecase and cross posting there would help? On Tue, 11 Aug, 2020, 1:43 am David Smiley, wrote: > Are you sure you need the docs in the same shard when maybe you could > assume a core exists on each node and then do a query-time join? > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein wrote: > >> I have a situation where I'd like to have the standard compositeId router >> in place for a collection. But, I'd like certain documents (ACL documents) >> to be duplicated on each shard in the collection. To achieve the level of >> access control performance and scalability I'm looking for I need the ACL >> records to be in the same core as the main documents. >> >> I put together a prototype where the compositeId router accepted implicit >> routing parameters and it worked in my testing. Before I open a ticket >> suggesting this approach I wonder what other people thought the best >> approach would be to accomplish this goal. >> >> >>
Re: hybrid document routing
Are you sure you need the docs in the same shard when maybe you could assume a core exists on each node and then do a query-time join? ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Mon, Aug 10, 2020 at 2:34 PM Joel Bernstein wrote: > I have a situation where I'd like to have the standard compositeId router > in place for a collection. But, I'd like certain documents (ACL documents) > to be duplicated on each shard in the collection. To achieve the level of > access control performance and scalability I'm looking for I need the ACL > records to be in the same core as the main documents. > > I put together a prototype where the compositeId router accepted implicit > routing parameters and it worked in my testing. Before I open a ticket > suggesting this approach I wonder what other people thought the best > approach would be to accomplish this goal. > > >
hybrid document routing
I have a situation where I'd like to have the standard compositeId router in place for a collection. But, I'd like certain documents (ACL documents) to be duplicated on each shard in the collection. To achieve the level of access control performance and scalability I'm looking for I need the ACL records to be in the same core as the main documents. I put together a prototype where the compositeId router accepted implicit routing parameters and it worked in my testing. Before I open a ticket suggesting this approach I wonder what other people thought the best approach would be to accomplish this goal.
