[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859277#comment-16859277 ] ASF subversion and git services commented on SOLR-13131: Commit 3d57a323a900bf2b80c27ba7a04387103ce516d2 in lucene-solr's branch refs/heads/master from Gus Heck [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3d57a32 ] SOLR-13131 Fix CHANGES.txt entry > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-13131) Category Routed Aliases
Ah yes I had meant to fix this. It's from when I was misunderstanding how changes was meant to be updated... will do On Sat, Jun 8, 2019 at 12:18 AM Cassandra Targett (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859100#comment-16859100 > ] > > Cassandra Targett commented on SOLR-13131: > -- > > [~gus_heck] - CHANGES.txt includes this issue under 9.0, but this issue > says it's fixed in 8.1 (and the docs are in 8.1). Was the CHANGES entry > perhaps put in the wrong place? > > See https://github.com/apache/lucene-solr/blob/master/solr/CHANGES.txt#L56 > > > Category Routed Aliases > > --- > > > > Key: SOLR-13131 > > URL: https://issues.apache.org/jira/browse/SOLR-13131 > > Project: Solr > > Issue Type: Improvement > > Security Level: Public(Default Security Level. Issues are Public) > > Components: SolrCloud > >Affects Versions: master (9.0) > >Reporter: Gus Heck > >Assignee: Gus Heck > >Priority: Major > > Fix For: 8.1, master (9.0) > > > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the > collections to also be searched as a whole. > > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > > This ticket also looks forward to (but does not include) the creation of > a Dimensionally Routed Alias which would allow organizing time routed data > also segregated by device > > Further design details to be added in comments. > > > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005) > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859100#comment-16859100 ] Cassandra Targett commented on SOLR-13131: -- [~gus_heck] - CHANGES.txt includes this issue under 9.0, but this issue says it's fixed in 8.1 (and the docs are in 8.1). Was the CHANGES entry perhaps put in the wrong place? See https://github.com/apache/lucene-solr/blob/master/solr/CHANGES.txt#L56 > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834876#comment-16834876 ] Ishan Chattopadhyaya commented on SOLR-13131: - bq. We perhaps should open a test-fixing ticket instead? Sure. Shall we resolve this issue and open a new issue? > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834872#comment-16834872 ] Ishan Chattopadhyaya commented on SOLR-13131: - Ah, sorry. I was trying to move all Open/ReOpened issues marked from 8.1 to 8.2. I missed that this is already in 8.1. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: master (9.0), 8.2 > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834865#comment-16834865 ] Gus Heck commented on SOLR-13131: - This was reopened on test instability questions, but those are non-functional intermittent test OOM type problems. The feature works and the code is in so I think the fix version should reflect that. I do think some further refinement of the tests is likely necessary but it's going to take a deeper investigation than I have cycles for in the near future (but I will get back to it). We perhaps should open a test-fixing ticket instead? > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: master (9.0), 8.2 > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[
https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810212#comment-16810212
]
Gus Heck commented on SOLR-13131:
-
Also FWIW I just saw a build that appears to have hit this known issue... which
perhaps is occurring more frequently with this test for some reason than it did
while beasting the similar TRA test. ( SOLR-13059 ). The problem is it's very
hard to find code that is 100% reliable to wait for a collection to really
fully be created. (I seem to recall seeing [~markrmil...@gmail.com] commenting
about or creating an issue for that but I can't find it) I know that in
SOLR-13051 I adopted some code from his test fixing efforts in this area but it
didn't 100% fix the problem.
{code:java}
// TODO: fix SOLR-13059, a where this wait isn't working ~0.3% of the time.
waitCol(1,configName);
// manipulate the config...
checkNoError(getSolrClient().request(new V2Request.Builder("/collections/" +
configName + "/config")
{code}
http://jenkins.sarowe.net/view/main-branches/job/Lucene-Solr-tests-master/lastBuild/testReport/org.apache.solr.update.processor/CategoryRoutedAliasUpdateProcessorTest/testNonEnglish/
> Category Routed Aliases
> ---
>
> Key: SOLR-13131
> URL: https://issues.apache.org/jira/browse/SOLR-13131
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Major
> Fix For: 8.1, master (9.0)
>
> Attachments: indexingWithCRA.png, indexingwithoutCRA.png,
> indexintWithoutCRA2.png
>
>
> This ticket is to add a second type of routed alias in addition to the
> current time routed aliases. The new type of alias will allow data driven
> creation of collections based on the values of a field and automated
> organization of these collections under an alias that allows the collections
> to also be searched as a whole.
> The use case in mind at present is an IOT device type segregation, but I
> could also see this leading to the ability to direct updates to tenant
> specific hardware (in cooperation with autoscaling).
> This ticket also looks forward to (but does not include) the creation of a
> Dimensionally Routed Alias which would allow organizing time routed data also
> segregated by device
> Further design details to be added in comments.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809400#comment-16809400 ] Gus Heck commented on SOLR-13131: - In opening 13 tar.gz files, I've found 8 of them have out of memory errors. I may have missed a prior OOM in the early files having seen later exceptions first when scrolling back up. At least 50% of the problem is tests running out of memory. So this might mean there's some need to work on skinnying up the test, but I'm not worried about the functionality from anything I've seen. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809361#comment-16809361 ] Gus Heck commented on SOLR-13131: - The first half a dozen I've looked out included 2 zk session expirations, 1 OOM Heap, one OOM GC limit exceeded and 2 Conection timeouts... so far nothing that's an actual logic failure still looking. Also, not sure why this would be rolled back instead of BadApple or AwaitsFix. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809141#comment-16809141 ] Gus Heck commented on SOLR-13131: - Hmm, I didn't see this on the bad apples reports? I'll take a look. I'd been looking at the bad-apple reports and assuming that if it didn't show there, then there wasn't a problem > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794041#comment-16794041 ] Gus Heck commented on SOLR-13131: - Ah I had noticed that, and that adding 1. 2. 3. Made it go away... I wasn't aware that . Without a number was an option. Will try to fix soon, but pretty busy this weekend. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[
https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793967#comment-16793967
]
Hoss Man commented on SOLR-13131:
-
[~gus_heck] - i really appreciate you adding aliases.adoc as part of this
issue, but it currently triggers several asciidoc warnings due to the use of
{{1.}} multiple times in an ordered list (this is supported to make it easier
for migrating docs from other markup syntaxes, but not recomended - hence the
warnigs: https://asciidoctor.org/docs/user-manual/#ordered-lists ) ...
{noformat}
[exec] asciidoctor: WARNING: aliases.adoc: line 26: list item index:
expected 2, got 1
[exec] asciidoctor: WARNING: aliases.adoc: line 27: list item index:
expected 3, got 1
[exec] asciidoctor: WARNING: aliases.adoc: line 225: list item index:
expected 2, got 1
[exec] asciidoctor: WARNING: aliases.adoc: line 226: list item index:
expected 3, got 1
{noformat}
..could you please update these to use the recommended {{. }} syntax instead?
> Category Routed Aliases
> ---
>
> Key: SOLR-13131
> URL: https://issues.apache.org/jira/browse/SOLR-13131
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Major
> Fix For: 8.1, master (9.0)
>
> Attachments: indexingWithCRA.png, indexingwithoutCRA.png,
> indexintWithoutCRA2.png
>
>
> This ticket is to add a second type of routed alias in addition to the
> current time routed aliases. The new type of alias will allow data driven
> creation of collections based on the values of a field and automated
> organization of these collections under an alias that allows the collections
> to also be searched as a whole.
> The use case in mind at present is an IOT device type segregation, but I
> could also see this leading to the ability to direct updates to tenant
> specific hardware (in cooperation with autoscaling).
> This ticket also looks forward to (but does not include) the creation of a
> Dimensionally Routed Alias which would allow organizing time routed data also
> segregated by device
> Further design details to be added in comments.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791731#comment-16791731 ] ASF subversion and git services commented on SOLR-13131: Commit 1d85f13eeb00430b0319b8e0cb5b27f98dd43edb in lucene-solr's branch refs/heads/branch_8x from Gus Heck [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1d85f13 ] SOLR-13131 Category Routed Aliases (cherry picked from commit d8f2a02fdb11a484425f9fddfa7061711d2f0034) > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791320#comment-16791320 ] Gus Heck commented on SOLR-13131: - Pushed to master, but it's getting late here will do 8x in the morning assuming tests pass > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791314#comment-16791314 ] ASF subversion and git services commented on SOLR-13131: Commit d8f2a02fdb11a484425f9fddfa7061711d2f0034 in lucene-solr's branch refs/heads/master from Gus Heck [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d8f2a02 ] SOLR-13131 Category Routed Aliases > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784699#comment-16784699 ] Gus Heck commented on SOLR-13131: - Work on this feature seems is more or less complete, feedback/review from community welcome. If no objection I will merge to master (9.0) and branch_8x for (8.1) early next week. Docs are updated too, with the Time Routed Aliases page changed to cover Aliases in general. I'm terrible at spotting my own typos and mis-edits, so a read by someone there would be appreciated. All changes are on a branch named solr-13131. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759287#comment-16759287 ] David Smiley commented on SOLR-13131: - Okay. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758855#comment-16758855 ] Gus Heck commented on SOLR-13131: - Attached a series of images showing how indexing such a use case is simplified. With CRA's one could use a single generic send to so solr step, whereas without you're writing custom batching code or configuring a myriad of generic senders. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758853#comment-16758853 ] Gus Heck commented on SOLR-13131: - This feature starts from the position that you have a use case where you want accept a heterogeneous stream of data and segregate it into various collections. If you don't have a reason to separate the data into distinct collections, or the data flows generating documents are separate and not easily merged, there would be little or no call for using a CRA. The key benefit is that it's data driven, and doesn't require human intervention or down time for configuration/devops/programming/etc to begin accepting a new type. This could be important if one is feeding a continuous stream of IoT sensor data (for example) and new sensor types/brands/locations/etc may come on line and be added without notice. Autmated collection creation from outside solr based on data values in the documents doesn't have a smooth, easy solution that I can see. One obviously can't run a check for the existence of a collection for every document via collections api. That would be insanely slow. Parsing exception messages to know when you need to create a new collection also seems very ugly. A workable solution likely involves tracking solr's list of collections separately, but that will have obvious concurrency pitfalls. One could possibly build indexing infrastructure that monitored zookeeper directly similar to what Solr does, but that's complex and requires skill with zookeeper. Also, I'm not sure I like that idea since it turns zookeeper's organization and details into a public API. By way of contrast, Solr is already well positioned to know it's own state, handle concurrency and react to document values. Another benefit is sheer convenience and reduction of client side (indexing) complexity when segregating based on a field value. One doesn't have to build and maintain infrastructure to map categories to your collections, which would be required when building URL's to send the data to specific collections or setting collections on each client... and if you're handling a mixed stream then you have to batch each type independently because they will be headed for different URL's or handled by separate SolrJ clients... I can also imagine CRA's greatly easing construction of systems with a collection per tenant pattern. The indexing infrastructure would always stamp the tenant's data with their customer_id and so long as that happens you can be sure that solr will route to separate collections on customer_id. The front end can build it's queries knowing the customer id and setting the appropriate collection. Leaks between customers become impossible, and there is absolutely no need to change infrastructure to add a customer (other than adding nodes for capacity every N customers of course). There also would be no need or write code that has to run admin level commands. Admin command access could possibly be removed from the application entirely. Running reports across tenants (querying via the alias in a back end application) would "just work" again with no special programming. Moving big or noisy tenants to preferred hardware would not require software/config changes either, just admin commands, or auto-scaling labels, and wouldn't disrupt any of the foregoing. Much like TRA's there are ways to do any/all of this with custom code, or alternate infrastructure, the goal is to make it easier and more hands off. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758655#comment-16758655 ] David Smiley commented on SOLR-13131: - Can you help me understand a use-case where a client system can't easily manage this itself (and thus is best done here in SolrCloud internally)? You loosely mentioned a use-case but maybe I'm unimaginative but it doesn't seem like a big deal for a client to detect it needs to create a collection first. With time routed data, there is interesting stuff that SolrCloud can do that's a pain for a client but I'm not appreciating what this is for a simple "category" case. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758421#comment-16758421 ] Gus Heck commented on SOLR-13131: - While working on this it has occurred to me that case-insensitive categories might be desirable. Such a feature would also imply a need to define the locale for such comparisons. Not yet sure if that should be a sub task or a follow on enhancement. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16754439#comment-16754439 ] Gus Heck commented on SOLR-13131: - Started a feature branch for this group of tickets: https://gitbox.apache.org/repos/asf?p=lucene-solr.git;a=shortlog;h=refs/heads/solr-13131 > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[
https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739731#comment-16739731
]
Gus Heck commented on SOLR-13131:
-
h1. Functionality
h2. New Parameter Value
*router.name* would gain a new valid value of "category"
h2. New Params
This feature would need some safety valves on it to avoid collection creation
(similar in spirit to router.maxFutureMs for TRAs). To that end I suggest:
# *router.maxCardinality* to place a limit on the total number of collections
that can be created (maybe required?)
# *router.mustMatch* to provide pattern matching for valid data and reject
requests that would create an undesired collection (optional)
# {color:#707070}*router.dictionary*{color} might also be added to provide a
set of acceptable values (optional) - This may or may not be implemented as
part of this ticket.
With respect to router.dictionary, I could imagine there being a desire to have
that dictionary used as a spell checker for segments of the values. One could
break the value on _ (or something else) and make sure all the parts are
spelled properly. One could also imagine the dictionary being applied to
specific matching groups from router.mustMatch, but all of this dictionary
based checking would be a future enhancement. I'm mentioning it here to get the
idea out there for future reference.
h2. Routed Field Constraints
The data in the field to be routed will need to be constrained in a couple ways
to make this work
# The routed field would need to be single valued, and encountering multiple
values should throw an error.
# The value in the routed field must be convertible to a valid collection
name. This conversion will likely be done by replacing any invalid characters
with '_' and it is the user's responsibility to ensure that the resulting names
are unique and do not interfere with other collections in the system. Values
that resolve to an existing collection that is not part of the alias will cause
an error to be returned, the existing collection will remain unaffected and
will not become added to the alias.
h2. Validations
In addition to constraints on the values, the following validations will be
enforced at the time the CategoryRoutedAlias is created
# The *collections* attribute is not set (applies only to non-routed aliases)
# None of the TimeRoutedAlias attributes are present
# TimeRoutedAliases will also be modified to validate that
*router.maxCardinality* and *router.mustMatch* are not set
h1. Implementation
The intention here is to first convert TimeRoutedAliasUpdateProcessor to
RoutedAliasUpdateProcessor and move as much time related functionality to
TimeRoutedAlias class as possible. If necessary TimeRoutedAliasUpdateProcessor
may still remain as a (hopefully skinny) subclass of RoutedUpdateProcessor. I
also hope to extract a RoutedAlias interface from TimeRoutedAlias and that will
implemented on a new CategoryRoutedAlias class. Ideally I'd like to end up with
a RoutedAliasUpdateProcessor and two concrete RoutedAlias implementations,
though I'm not sure if that will really be possible. I'll break things down and
make individual tickets for sub parts after I play with the code a little.
Both v1 api and v2 api will be supported
h1. Documentation
# The TimeRoutedAliases page will be converted to a RoutedAliases page with
sections for TimeRoutedAliases and CategoryRoutedAliases
# The CreateAliasCommand Documentation will be updated
# The v2 api will return documentation for the new and modified attributes via
that api.
> Category Routed Aliases
> ---
>
> Key: SOLR-13131
> URL: https://issues.apache.org/jira/browse/SOLR-13131
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: master (9.0)
>Reporter: Gus Heck
>Assignee: Gus Heck
>Priority: Major
>
> This ticket is to add a second type of routed alias in addition to the
> current time routed aliases. The new type of alias will allow data driven
> creation of collections based on the values of a field and automated
> organization of these collections under an alias that allows the collections
> to also be searched as a whole.
> The use case in mind at present is an IOT device type segregation, but I
> could also see this leading to the ability to direct updates to tenant
> specific hardware (in cooperation with autoscaling).
> This ticket also looks forward to (but does not include) the creation of a
> Dimensionally Routed Alias which would allow organizing time routed data also
> segregated by device
> Further design details to be added in comments.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
