[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392318#comment-16392318
 ] 

Shalin Shekhar Mangar commented on SOLR-11722:
--

bq. No the "async" command was not removed; perhaps it appeared it was by 
looking at the diff which would show edits between the two commands, both of 
which had async, one gets removed which you'd see, but you wouldn't necessarily 
see in the diff that the existing "async" is still there (line 186).

Ah, yes, that was it. Sorry for the noise.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391476#comment-16391476
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 567f253447eb6487f5eae6431066c215505638aa in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=567f253 ]

SOLR-11722: Minor improvements to documentation in collections.Commands.json

(cherry picked from commit 582c828)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391475#comment-16391475
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 582c828727b306844c550bbeb6390f52a9bda092 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=582c828 ]

SOLR-11722: Minor improvements to documentation in collections.Commands.json


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391472#comment-16391472
 ] 

David Smiley commented on SOLR-11722:
-

Thanks for reviewing Shalin.  No the "async" command was not removed; perhaps 
it _appeared_ it was by looking at the diff which would show edits between the 
two commands, both of which had async, one gets removed which you'd see, but 
you wouldn't necessarily see in the diff that the existing "async" is still 
there (line 186).

While I was looking at this, I started tweaking some of the description 
language so I'll commit that.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391147#comment-16391147
 ] 

Shalin Shekhar Mangar commented on SOLR-11722:
--

Did you really mean to remove "async" param from the collections.Commands.json 
for create-alias API?

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390140#comment-16390140
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit ae97500ec2198280e4dc96bd8b5d64b65034cd5c in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ae97500 ]

SOLR-11722: Merge CREATEDROUTEDALIAS command into CREATEALIAS.  Only
an API & docs change; the implementation was already there.

(cherry picked from commit ee1c742)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390137#comment-16390137
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit ee1c742c199c47894396c2e614a076d971e2d1df in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee1c742 ]

SOLR-11722: Merge CREATEDROUTEDALIAS command into CREATEALIAS.  Only
an API & docs change; the implementation was already there.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386702#comment-16386702
 ] 

David Smiley commented on SOLR-11722:
-

Thanks for the updated patch [~gus_heck] on converting this functionality into 
CREATEALIAS.  I see the new GitHub link auto-attached and the info in the 
Worklog here in Jira so it's nice to see we can create additional PRs for the 
same issue.  It looks good to me, albeit a typo in 
{{testCollectionNmaesMustBeAbsent}} which I can correct.  I'll commit in a bit.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358616#comment-16358616
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 8bb135aeda07d75773bac52c89d448af6a90d541 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8bb135a ]

SOLR-11722: Rename max-future-ms to maxFutureMs

(cherry picked from commit 5a26381)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358614#comment-16358614
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 5a263814241bf2db21107ded9ab84282bbda667e in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5a26381 ]

SOLR-11722: Rename max-future-ms to maxFutureMs


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353345#comment-16353345
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 3251679abd3f4cb8325423ab0d96ff71bcb198e2 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3251679 ]

SOLR-11722 SOLR-11617: Alias tests: Ensure zkStateReader's view is up to date 
before acting

(cherry picked from commit 812d400)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353343#comment-16353343
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 812d400807bcebc782f85dcf3bba5619421880cb in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=812d400 ]

SOLR-11722 SOLR-11617: Alias tests: Ensure zkStateReader's view is up to date 
before acting


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348917#comment-16348917
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 88bdfbb5298ef3c79ce5e00f47f5d0ad733b4562 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=88bdfbb ]

SOLR-11722: Dial down threads in ConcurrentCreateRoutedAliasTest

(cherry picked from commit f0854ce)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348914#comment-16348914
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit f0854ce11f1e6ccaa3190c82362c0c49ec043df8 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f0854ce ]

SOLR-11722: Dial down threads in ConcurrentCreateRoutedAliasTest


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342493#comment-16342493
 ] 

Gus Heck commented on SOLR-11722:
-

Ah I hadn't noticed it, but there is a mixed convention in the v2 api... top 
level commands seem to be hyphen separated, but lower level properties are 
camelCased... I didn't note that dichotomy because I had collapsed the commands 
I wasn't working with in my ide most of the time... I agree that we should fix 
that before this gets released.  I did contemplate Date math there for max 
future as well, but my initial reaction was that in some ways (with tz 
considerations) that might be subtly more complicated for something that maybe 
rarely needs that complexity... on the other hand yeah it's more consistent API 
wise, and maybe there's a case where someone does want to account for timezone 
complexities. I could go either way on it. 

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342492#comment-16342492
 ] 

Gus Heck commented on SOLR-11722:
-

Ah I hadn't noticed it, but there is a mixed convention in the v2 api... top 
level commands seem to be hyphen separated, but lower level properties are 
camelCased... I didn't note that dichotomy because I had collapsed the commands 
I wasn't working with in my ide most of the time... I agree that we should fix 
that before this gets released.  I did contemplate Date math there for max 
future as well, but my initial reaction was that in some ways (with tz 
considerations) that might be subtly more complicated for something that maybe 
rarely needs that complexity... on the other hand yeah it's more consistent API 
wise, and maybe there's a case where someone does want to account for timezone 
complexities. I could go either way on it. 

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342401#comment-16342401
 ] 

David Smiley commented on SOLR-11722:
-

There should definitely be a section (of an existing page) in the ref guide on 
time routed aliases.  There isn't yet; I think it could wait until the 7.3 
release time as it'll then be clear what 7.3 holds (and doesn't) with respect 
to TRAs.

Another "in retrosect":  The param "max-future-ms" ought to be "maxFutureMs".  
And after looking at SOLR-11066, perhaps ought to be a duration (date math) to 
be nicer.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342348#comment-16342348
 ] 

Gus Heck commented on SOLR-11722:
-

Possibly. On the flip side we somewhat loose the advertising of the feature in 
the list of commands. Maybe if there were a full separate section on this 
feature in the ref guide so that it shows up in the tree/toc stuff, that would 
compensate. Wouldn't want this to become a "hidden, little known" feature. 

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341708#comment-16341708
 ] 

David Smiley commented on SOLR-11722:
-

In retrospect... I wonder if it would make the overall API less busy if 
create-alias had these options instead of creating a new API command?  We 
already have it combined at the CreateAliasCmd level. When it comes time to 
delete the whole thing, again, we just add logic to delete underlying 
collections to the existing alias deletion.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341292#comment-16341292
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 9cf9b8cc496568262fbc3bafeb14280986d7bce4 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9cf9b8c ]

SOLR-11722: Improve the v2/v1 API mapping, including a bug.
Wrapped getParameterNamesIterator failed to consider attrToParams.
(committing this separately from rest)

(cherry picked from commit d8e9ab8)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341293#comment-16341293
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 8ecbf8197042fdab6c5fd86bfc7373c545efd054 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8ecbf81 ]

SOLR-11722: New CREATEROUTEDALIAS cluster command for time routed aliases.
(a refactoring of some of the related parts will follow in next commit)

(cherry picked from commit a1828a5)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341295#comment-16341295
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit ae1f380056e726904defd8c58b8ce26b8cfde338 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ae1f380 ]

SOLR-11722: CHANGES.txt

(cherry picked from commit 3c9829e)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341294#comment-16341294
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 5fd8c21c56bbb934ecfe6a33dd2971f278f365b3 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5fd8c21 ]

SOLR-11722: Refactor out a TimeRoutedAlias class from various parts.
Also allowed TRA's to be tolerant of pre-existing collections.

(cherry picked from commit b0d244f)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341267#comment-16341267
 ] 

David Smiley commented on SOLR-11722:
-

The 3rd commit refactored out a TimeRoutedAlias class from some commonly needed 
code (including parameter validation).  This functionality is all a bit more 
tidy now.
Another change of note is that I made the the TRA tolerant of the possibility 
that the collection may already exist.  It's not likely but if so it's adopted.

[~noble.paul] FYI you may find the commit 
[https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d8e9ab8] 
interesting as it relates to the API v2/v1 mapping, and fixed a bug which we 
discovered while working on the feature here.

 

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341262#comment-16341262
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit 3c9829efad47d92497afa60ce557ee12fab0934a in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3c9829e ]

SOLR-11722: CHANGES.txt


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341248#comment-16341248
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit d8e9ab8785e638ecf07eed43055aa02332eb7862 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d8e9ab8 ]

SOLR-11722: Improve the v2/v1 API mapping, including a bug.
Wrapped getParameterNamesIterator failed to consider attrToParams.
(committing this separately from rest)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341250#comment-16341250
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit b0d244f656b5f0030bfda97aaf5e6e7ad085325c in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b0d244f ]

SOLR-11722: Refactor out a TimeRoutedAlias class from various parts.
Also allowed TRA's to be tolerant of pre-existing collections.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341249#comment-16341249
 ] 

ASF subversion and git services commented on SOLR-11722:


Commit a1828a5664983b8e26177537d233a78d2c0c33cd in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a1828a5 ]

SOLR-11722: New CREATEROUTEDALIAS cluster command for time routed aliases.
(a refactoring of some of the related parts will follow in next commit)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333919#comment-16333919
 ] 

David Smiley commented on SOLR-11722:
-

BTW there was no bug in createNodeSet; I was confused.

I don't think I can add commits to your PR so I forked your branch and filed a 
PR back to you:

https://github.com/nsoft/lucene-solr/pull/1

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332395#comment-16332395
 ] 

Gus Heck commented on SOLR-11722:
-

I think that's one of the ones that differs in v1/v2 naming, check that the 
conversion got handled? I'm not very familiar with that feature, so it might 
not have been tested properly.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331747#comment-16331747
 ] 

David Smiley commented on SOLR-11722:
-

I've been working on moving this further along.  There seems to be a bug in 
honoring createNodeSet (and that option might be quite useful with this 
feature) so I'm working on testing it and making it pass.  I'll have something 
tomorrow.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Priority: Major
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324348#comment-16324348
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user nsoft commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161294218
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  SolrParams v1Params = convertToV1WhenRequired(req, collectionParams);
+
+  // We need to add this temporary name just to pass validation.
--- End diff --

ah that's actually checked here: 
https://github.com/apache/lucene-solr/pull/304/files#diff-3fe6a8aeb14a57e63507fa17f8346771R207,
 but It could be moved to this class (or done both places)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323976#comment-16323976
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161226045
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  SolrParams v1Params = convertToV1WhenRequired(req, collectionParams);
+
+  // We need to add this temporary name just to pass validation.
--- End diff --

I wasn't clear; I don't have any issue with this part of the patch. I'm 
suggesting _adding_ a check that we require the configSet.  I needed to click 
on some line to insert my comment; maybe I should have put my review comment on 
the blank line.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323968#comment-16323968
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161223342
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -30,13 +44,101 @@
 import org.apache.solr.common.cloud.ZkStateReader;
 import org.apache.solr.common.util.NamedList;
 import org.apache.solr.common.util.StrUtils;
+import org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor;
+import org.apache.solr.util.DateMathParser;
+import org.apache.solr.util.TimeZoneUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
+import static java.time.format.DateTimeFormatter.ISO_INSTANT;
+import static 
org.apache.solr.cloud.OverseerCollectionMessageHandler.COLL_CONF;
+import static org.apache.solr.common.SolrException.ErrorCode.BAD_REQUEST;
 import static org.apache.solr.common.params.CommonParams.NAME;
+import static org.apache.solr.common.params.CommonParams.TZ;
+import static 
org.apache.solr.handler.admin.CollectionsHandler.ROUTED_ALIAS_COLLECTION_PROP_PFX;
+import static 
org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor.DATE_TIME_FORMATTER;
 
 
 public class CreateAliasCmd implements Cmd {
+
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static final String ROUTING_TYPE = "router.name";
+  public static final String ROUTING_FIELD = "router.field";
+  public static final String ROUTING_INCREMENT = "router.interval";
+  public static final String ROUTING_MAX_FUTURE = "router.max-future-ms";
+  public static final String START = "start";
+  // Collection constants should all reflect names in the v2 structured 
input for this command, not v1
+  // names used for CREATE
+  public static final String CREATE_COLLECTION_CONFIG = 
"create-collection.config";
--- End diff --

My concern isn't just about the duplication of the Strings/names, it's 
about the "set" of them here.  Even if we could refer to a constant in some 
other class accessible to SolrJ, it would still be a maintenance burden to 
refer to the set of those that exist since it's redundant with the collection 
creation code.  So can we eliminate this entirely (I'd love that) or failing 
that have a single place where the set of these strings are defined so we don't 
even need to list them in CreateAliasCmd.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323657#comment-16323657
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155289
  
--- Diff: 
solr/core/src/java/org/apache/solr/update/processor/TimeRoutedAliasUpdateProcessor.java
 ---
@@ -99,6 +101,8 @@
   .parseDefaulting(ChronoField.HOUR_OF_DAY, 0)
   .parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
   .parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
+  // Setting a timezone here is fine as a default, but generally need 
to clone with user's timezone, so that
+  // truncation of milliseconds is consistent.
--- End diff --

removed in update


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323659#comment-16323659
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155141
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
+&& !"stateFormat".equals(stringObjectEntry.getKey())
+&& !"name".equals(stringObjectEntry.getKey()))
+.collect(Collectors.toMap((e) -> "collection-create." + 
e.getKey(), e -> String.valueOf(e.getValue(;
+cleanMap.put(ROUTING_FIELD, routedField);
+cleanMap.put(ROUTING_TYPE, routingType);
+cleanMap.put(ROUTING_INCREMENT, increment);
+cleanMap.put(ROUTING_MAX_FUTURE, maxFutureMs);
+cleanMap.put(TZ, tz);
+return cleanMap;
+  }
+
+  private Instant validateStart(TimeZone zone, DateTimeFormatter fmt, 
String start) {
--- End diff --

checking the ms was the main point, the format and reparse was meant to 
trap any timezone/formatter craziness, which mostly goes away with the 
collection creation. Simplified, in update


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323656#comment-16323656
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155263
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -962,6 +982,76 @@ public static CollectionOperation get(CollectionAction 
action) {
 }
   }
 
+  private static SolrParams convertToV1WhenRequired(SolrQueryRequest req, 
ModifiableSolrParams params) {
+SolrParams v1Params = params; // (maybe...)
+
+// in the v2 world we get a data map based on the json request, from 
the CommandOperation associated
+// with the request, so locate that if we can.. if we find it we have 
to translate the v2 request
+// properties to v1 params, otherwise we're already good to go.
+List cmds = req.getCommands(true);
+if (cmds.size() > 1) {
+  // todo: not sure if this is the right thing to do here, but also 
not sure what to do if there is more than one...
+  throw new SolrException(BAD_REQUEST, "Only one command is allowed 
when creating a routed alias");
+}
+CommandOperation c = cmds.size() == 0 ? null : cmds.get(0);
+if (c != null) {  // v2 api, do conversion to v1
+  v1Params = new 
BaseHandlerApiSupport.V2ToV1SolrParams(CollectionApiMapping.Meta.CREATE_COLLECTION,
+  req.getPathTemplateValues(), true, params,
+  new CommandOperation("create", 
c.getDataMap().get("create-collection")));
+}
+return v1Params;
+  }
+
+  private static Map 
parseColletionCreationProps(CollectionsHandler h, SolrParams params, String 
prefix)
--- End diff --

fixed in update


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323658#comment-16323658
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155153
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
+&& !"stateFormat".equals(stringObjectEntry.getKey())
+&& !"name".equals(stringObjectEntry.getKey()))
+.collect(Collectors.toMap((e) -> "collection-create." + 
e.getKey(), e -> String.valueOf(e.getValue(;
+cleanMap.put(ROUTING_FIELD, routedField);
+cleanMap.put(ROUTING_TYPE, routingType);
+cleanMap.put(ROUTING_INCREMENT, increment);
+cleanMap.put(ROUTING_MAX_FUTURE, maxFutureMs);
+cleanMap.put(TZ, tz);
+return cleanMap;
+  }
+
+  private Instant validateStart(TimeZone zone, DateTimeFormatter fmt, 
String start) {
+// This is the normal/easy case, if we can get away with this great!
+TemporalAccessor startTime = attemptTimeStampParsing(start, 
zone.toZoneId());
+if (startTime == null) {
+  // No luck, they gave us either date math, or garbage, so we have to 
do more work to figure out which and
+  // to make sure it's valid date math and that it doesn't encode any 
millisecond precision.
+  ZonedDateTime now = 
ZonedDateTime.now(zone.toZoneId()).truncatedTo(ChronoUnit.MILLIS);
+  try {
+Date date = DateMathParser.parseMath(Date.from(now.toInstant()), 
start);
+String reformatted = 
fmt.format(date.toInstant().truncatedTo(ChronoUnit.MILLIS));
+Date reparse = 
Date.from(Instant.from(DATE_TIME_FORMATTER.parse(reformatted)));
+if (!reparse.equals(date)) {
+  throw new SolrException(BAD_REQUEST,
+  "Formatted time did not have the same milliseconds as 
original: " + date.getTime() + " vs. " +
+  reparse.getTime() + " This indicates that you used date 
math that includes milliseconds. " +
+  "(Hint: 'NOW' used without rounding always has this 
problem)" );
+}
+return date.toInstant();
+  } catch (SolrException e) {
+throw new SolrException(BAD_REQUEST,
+"Start Time for the first collection must be a timestamp of 
the format -MM-dd_HH_mm_ss, " +
+"or a valid date math expression not containing specific 
milliseconds", e);
+  }
+}
+return Instant.from(startTime);
+  }
+
+  private TemporalAccessor attemptTimeStampParsing(String start, ZoneId 
zone) {
+try {
+  DATE_TIME_FORMATTER.withZone(zone);
+  return DATE_TIME_FORMATTER.parse(start);
+} catch (DateTimeParseException e) {
+  return null;
+}
+  }
+
+  private boolean anyRoutingParams(ZkNodeProps message) {
+
+return message.containsKey(ROUTING_FIELD) || 
message.containsKey(ROUTING_TYPE) || message.containsKey(START)
+|| message.containsKey(ROUTING_INCREMENT) || 
message.containsKey(TZ);
+  }
+
   private void validateAllCollectionsExistAndNoDups(List 
collectionList, ZkStateReader zkStateReader) {
 final String collectionStr = StrUtils.join(collectionList, ',');
 
 if (new HashSet<>(collectionList).size() != collectionList.size()) {
-  throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+  throw new SolrException(BAD_REQUEST,
   String.format(Locale.ROOT,  "Can't create collection alias for 
collections='%s', since it contains duplicates", collectionStr));
 }
 ClusterState clusterState = zkStateReader.getClusterState();
 Set aliasNames = 
zkStateReader.getAliases().getCollectionAliasListMap().keySet();
 for (String collection : collectionList) {
   if (clusterState.getCollectionOrNull(collection) == null && 
!aliasNames.contains(collection)) {
-throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+throw new SolrException(BAD_REQUEST,
 String.format(Locale.ROOT,  "Can't create collection alias for 
collections='%s', '%s' 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323649#comment-16323649
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155221
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
--- End diff --

I think I was finding that in running the tests it wasn't always showing 
up. I can take it out. Removed here and in cmd class in update


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323653#comment-16323653
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155184
  
--- Diff: 
solr/core/src/java/org/apache/solr/cloud/OverseerCollectionMessageHandler.java 
---
@@ -218,6 +218,7 @@ public OverseerCollectionMessageHandler(ZkStateReader 
zkStateReader, String myId
 .put(RELOAD, this::reloadCollection)
 .put(DELETE, new DeleteCollectionCmd(this))
 .put(CREATEALIAS, new CreateAliasCmd(this))
+.put(CREATEROUTEDALIAS, new CreateAliasCmd(this))
--- End diff --

Seemed like a good idea at the time :) As you like it.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323650#comment-16323650
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155243
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  SolrParams v1Params = convertToV1WhenRequired(req, collectionParams);
+
+  // We need to add this temporary name just to pass validation.
--- End diff --

The name on this line that I am TMP-ing isn't the collection config name, 
it's the name of the collection. We don't want them to pass it but the 
collection param parsing code I'm reusing will barf (rightly) if there's no 
name for a collection. We'll fix that up later when we calculate the name for 
the initial collection in the Cmd class.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323654#comment-16323654
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161155122
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
--- End diff --

Made my own constants & made set existing constants were too 
scattered/disorganized/nonexistant ... fixed in update


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323531#comment-16323531
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161138197
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

An offline chat reveals that I misunderstood the intent some time back. 
Names for collections ARE meant to be in UTC, that will make things simpler I 
think.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323502#comment-16323502
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161134845
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

> you want TZ to adjust any dates that don't include a timezone, and not 
adjust any that do (where the only acceptable date included TZ is utc via the Z 
suffix)?

Not really because Solr doesn't accept ambiguous dates (dates without a 
'Z', == UTC) so that's kind of a non-issue.  _The TimeZone is needed to 
correctly interpret date math (if present)_.  Oh yeah, DateRangeField can 
accept very dates without 'Z' but lets ignore that :-)  Nonetheless DRF is 
strictly UTC.

The format of the collection name is an internal detail so it almost 
doesn't matter.  I'd rather it be permanently UTC interpreted to avoid problems 
in changing TZ somehow.  But I can sympathize that there's an argument to be 
made it would look nicer if it honored the TZ.  But again it's internal.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
>   

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323386#comment-16323386
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161122011
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

In reading the issue referenced in the above referenced comment again, it 
occurs to me that what I hadn't thought about is that by having TZ parameter at 
the top level it does sort of imply that it applies to the whole request, and 
we need to handle all dates supplied either as the specified UTC or if the Z is 
omitted handle them as the timezone specified. I had mostly been thinking in 
terms of the effect of TZ on updates and users trying to optimize by querying 
directly to the correct collection and the calculation of "next" collections. 
In theory, there could be 2 TZ's one for the routing configuration router.tz 
and one that applies to the dates in this request (the top level TZ)... but 
that is probably be overkill. So now I think I understand what you've been 
trying to tell me... you want TZ to adjust any dates that don't include a 
timezone, and not adjust any that do (where the only acceptable date included 
TZ is utc via the Z 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323256#comment-16323256
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161110384
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

(and that format has no timzeone in it so in theory we could treat it as 
local time based on TZ but at the moment I treat it as utc).


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323235#comment-16323235
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161107544
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
--- End diff --

yeah, this is stuff that the collection create code adds. I'll look if I 
can use constants from there and make a set out of them.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323186#comment-16323186
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161103620
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

actually amending that we do accept the format used for the collection 
names, but since a time stamp is valid date math we need to handle this too.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323171#comment-16323171
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161101824
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

So if someone specifies start as an iso instant, we don't want to adjust 
that to match the specified zone? This would lead to 
start=2018-01-14T20:00:00:00.Z=EST=+1DAY it would seem 
that the first bucket should be 2018-01-15 but this way we would be creating a 
first bucket for 2018-01-14 instead. That seems confusing. Solr fairly 
consistently takes the position that times you speak to it must be in UTC, 
changing that here seems awkward. We could allow more general time format and 
take the timezone from the entered time, so that they can enter 
2018-01-15T00:00:00:00.EST which is unlike the rest of how solr handles 
time AFAIK but would make the correspondence between the collection name and 
the time value entered intuitive (and maybe complicates combining this with the 
datemath) 

I had originally proposed that collections names for this be UTC and making 
them full time stamps to avoid all this TZ stuff, but the response 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323124#comment-16323124
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161096191
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
--- End diff --

in this area of the code I implement checks for the following:
name
start
router.field
router.name
router.interval
router.max-future-ms
TZ
collection-create.config

Only that last one could possibly be left to the collection-create 
infrastructure. The rest are ours (note that the router ones are not the 
collection level routing but the alias level routing).


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323095#comment-16323095
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user fsparv commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161093191
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -30,13 +44,101 @@
 import org.apache.solr.common.cloud.ZkStateReader;
 import org.apache.solr.common.util.NamedList;
 import org.apache.solr.common.util.StrUtils;
+import org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor;
+import org.apache.solr.util.DateMathParser;
+import org.apache.solr.util.TimeZoneUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
+import static java.time.format.DateTimeFormatter.ISO_INSTANT;
+import static 
org.apache.solr.cloud.OverseerCollectionMessageHandler.COLL_CONF;
+import static org.apache.solr.common.SolrException.ErrorCode.BAD_REQUEST;
 import static org.apache.solr.common.params.CommonParams.NAME;
+import static org.apache.solr.common.params.CommonParams.TZ;
+import static 
org.apache.solr.handler.admin.CollectionsHandler.ROUTED_ALIAS_COLLECTION_PROP_PFX;
+import static 
org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor.DATE_TIME_FORMATTER;
 
 
 public class CreateAliasCmd implements Cmd {
+
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static final String ROUTING_TYPE = "router.name";
+  public static final String ROUTING_FIELD = "router.field";
+  public static final String ROUTING_INCREMENT = "router.interval";
+  public static final String ROUTING_MAX_FUTURE = "router.max-future-ms";
+  public static final String START = "start";
+  // Collection constants should all reflect names in the v2 structured 
input for this command, not v1
+  // names used for CREATE
+  public static final String CREATE_COLLECTION_CONFIG = 
"create-collection.config";
--- End diff --

I agree, I don't like having them here and in the collection admin request  
but solrj can't see core. Is there a constants class I can move them to that 
solrj can see?


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322995#comment-16322995
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161076165
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
+
+  // check that the increment is valid date math
+  String checkIncrement = ISO_INSTANT.format(Instant.now()) + 
increment;
+  DateMathParser.parseMath(new Date(), checkIncrement); // exception 
if invalid increment
--- End diff --

Maybe I wasn't clear before.  I mean create a new DateMathParser using the 
constructor that accept the timezone (and we can supply ours; null is okay).  
Then call the instance method `parseMath(str)` on it (not the static with 
'now').  the arg should be the start metadata.  The "now" will default to the 
current instant in time which is what we want.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322993#comment-16322993
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161071581
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -962,6 +982,76 @@ public static CollectionOperation get(CollectionAction 
action) {
 }
   }
 
+  private static SolrParams convertToV1WhenRequired(SolrQueryRequest req, 
ModifiableSolrParams params) {
+SolrParams v1Params = params; // (maybe...)
+
+// in the v2 world we get a data map based on the json request, from 
the CommandOperation associated
+// with the request, so locate that if we can.. if we find it we have 
to translate the v2 request
+// properties to v1 params, otherwise we're already good to go.
+List cmds = req.getCommands(true);
+if (cmds.size() > 1) {
+  // todo: not sure if this is the right thing to do here, but also 
not sure what to do if there is more than one...
+  throw new SolrException(BAD_REQUEST, "Only one command is allowed 
when creating a routed alias");
+}
+CommandOperation c = cmds.size() == 0 ? null : cmds.get(0);
+if (c != null) {  // v2 api, do conversion to v1
+  v1Params = new 
BaseHandlerApiSupport.V2ToV1SolrParams(CollectionApiMapping.Meta.CREATE_COLLECTION,
+  req.getPathTemplateValues(), true, params,
+  new CommandOperation("create", 
c.getDataMap().get("create-collection")));
+}
+return v1Params;
+  }
+
+  private static Map 
parseColletionCreationProps(CollectionsHandler h, SolrParams params, String 
prefix)
--- End diff --

typo


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322997#comment-16322997
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161074349
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  SolrParams v1Params = convertToV1WhenRequired(req, collectionParams);
+
+  // We need to add this temporary name just to pass validation.
--- End diff --

We need to insist that the collection config name is passed here.  It's 
very problematic to depend on automatic creation.  On a related note, when it's 
time to document TRA's, we may want to warn against data driven schema.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322994#comment-16322994
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161077363
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
--- End diff --

I think much of these param validation checks are supposed to be handled at 
the CollectionsHandler level; no?  I'm not 100% sure.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323000#comment-16323000
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161078867
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
+&& !"stateFormat".equals(stringObjectEntry.getKey())
+&& !"name".equals(stringObjectEntry.getKey()))
+.collect(Collectors.toMap((e) -> "collection-create." + 
e.getKey(), e -> String.valueOf(e.getValue(;
+cleanMap.put(ROUTING_FIELD, routedField);
+cleanMap.put(ROUTING_TYPE, routingType);
+cleanMap.put(ROUTING_INCREMENT, increment);
+cleanMap.put(ROUTING_MAX_FUTURE, maxFutureMs);
+cleanMap.put(TZ, tz);
+return cleanMap;
+  }
+
+  private Instant validateStart(TimeZone zone, DateTimeFormatter fmt, 
String start) {
--- End diff --

This appears redundant with using DateMathParser earlier and I'm sure we 
don't need so much code for this matter (ignore millisecond truncation; in 
practice the user is going to do something like `/MONTH` or `/DAY` for start.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322999#comment-16322999
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161075027
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"Invalid timezone:" + tz);
+log.error("Could not create routed alias",solrException);
+throw solrException;
+
+  }
+  TimeZone zone;
+  if (tz != null) {
+zone = TimeZoneUtils.getTimeZone(tz);
+  } else {
+zone = TimeZoneUtils.getTimeZone("UTC");
+  }
+  DateTimeFormatter fmt = 
DATE_TIME_FORMATTER.withZone(zone.toZoneId());
--- End diff --

No we don't want to use the TZ to format the collection name


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322996#comment-16322996
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161079147
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
+&& !"stateFormat".equals(stringObjectEntry.getKey())
+&& !"name".equals(stringObjectEntry.getKey()))
+.collect(Collectors.toMap((e) -> "collection-create." + 
e.getKey(), e -> String.valueOf(e.getValue(;
+cleanMap.put(ROUTING_FIELD, routedField);
+cleanMap.put(ROUTING_TYPE, routingType);
+cleanMap.put(ROUTING_INCREMENT, increment);
+cleanMap.put(ROUTING_MAX_FUTURE, maxFutureMs);
+cleanMap.put(TZ, tz);
+return cleanMap;
+  }
+
+  private Instant validateStart(TimeZone zone, DateTimeFormatter fmt, 
String start) {
+// This is the normal/easy case, if we can get away with this great!
+TemporalAccessor startTime = attemptTimeStampParsing(start, 
zone.toZoneId());
+if (startTime == null) {
+  // No luck, they gave us either date math, or garbage, so we have to 
do more work to figure out which and
+  // to make sure it's valid date math and that it doesn't encode any 
millisecond precision.
+  ZonedDateTime now = 
ZonedDateTime.now(zone.toZoneId()).truncatedTo(ChronoUnit.MILLIS);
+  try {
+Date date = DateMathParser.parseMath(Date.from(now.toInstant()), 
start);
+String reformatted = 
fmt.format(date.toInstant().truncatedTo(ChronoUnit.MILLIS));
+Date reparse = 
Date.from(Instant.from(DATE_TIME_FORMATTER.parse(reformatted)));
+if (!reparse.equals(date)) {
+  throw new SolrException(BAD_REQUEST,
+  "Formatted time did not have the same milliseconds as 
original: " + date.getTime() + " vs. " +
+  reparse.getTime() + " This indicates that you used date 
math that includes milliseconds. " +
+  "(Hint: 'NOW' used without rounding always has this 
problem)" );
+}
+return date.toInstant();
+  } catch (SolrException e) {
+throw new SolrException(BAD_REQUEST,
+"Start Time for the first collection must be a timestamp of 
the format -MM-dd_HH_mm_ss, " +
+"or a valid date math expression not containing specific 
milliseconds", e);
+  }
+}
+return Instant.from(startTime);
+  }
+
+  private TemporalAccessor attemptTimeStampParsing(String start, ZoneId 
zone) {
+try {
+  DATE_TIME_FORMATTER.withZone(zone);
+  return DATE_TIME_FORMATTER.parse(start);
+} catch (DateTimeParseException e) {
+  return null;
+}
+  }
+
+  private boolean anyRoutingParams(ZkNodeProps message) {
+
+return message.containsKey(ROUTING_FIELD) || 
message.containsKey(ROUTING_TYPE) || message.containsKey(START)
+|| message.containsKey(ROUTING_INCREMENT) || 
message.containsKey(TZ);
+  }
+
   private void validateAllCollectionsExistAndNoDups(List 
collectionList, ZkStateReader zkStateReader) {
 final String collectionStr = StrUtils.join(collectionList, ',');
 
 if (new HashSet<>(collectionList).size() != collectionList.size()) {
-  throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+  throw new SolrException(BAD_REQUEST,
   String.format(Locale.ROOT,  "Can't create collection alias for 
collections='%s', since it contains duplicates", collectionStr));
 }
 ClusterState clusterState = zkStateReader.getClusterState();
 Set aliasNames = 
zkStateReader.getAliases().getCollectionAliasListMap().keySet();
 for (String collection : collectionList) {
   if (clusterState.getCollectionOrNull(collection) == null && 
!aliasNames.contains(collection)) {
-throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
+throw new SolrException(BAD_REQUEST,
 String.format(Locale.ROOT,  "Can't create collection alias for 
collections='%s', '%s' 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322992#comment-16322992
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161074912
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
+  throw new NumberFormatException("Negative value not allowed 
here");
+}
+  } catch (NumberFormatException e) {
+throw new SolrException(BAD_REQUEST, ROUTING_MAX_FUTURE + " must 
be a valid long integer representing a number " +
+"of milliseconds greater than or equal to zero");
+  }
 
-zkStateReader.aliasesHolder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+  // Validate we got everything we need
+  if (routedField == null || routingType == null || start == null || 
increment == null) {
+SolrException solrException = new SolrException(BAD_REQUEST, "If 
any of " + CREATE_ROUTED_ALIAS_PARAMS +
+" are supplied, then all of " + REQUIRED_ROUTING_PARAMS + " 
must be present.");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+
+  if (!"time".equals(routingType)) {
+SolrException solrException = new SolrException(BAD_REQUEST, "Only 
time based routing is supported at this time");
+log.error("Could not create routed alias",solrException);
+throw solrException;
+  }
+  // Check for invalid timezone
+  if(tz != null && !TimeZoneUtils.KNOWN_TIMEZONE_IDS.contains(tz)) {
--- End diff --

You needn't explicitly validate TZ here; call TimeZoneUtils.parseTimezone 
instead which does so (which I added recently).


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322998#comment-16322998
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161079495
  
--- Diff: 
solr/core/src/test/org/apache/solr/cloud/CreateRoutedAliasTest.java ---
@@ -0,0 +1,351 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.cloud;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.time.temporal.ChronoUnit;
+import java.util.Date;
+import java.util.Locale;
+import java.util.Map;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.http.HttpEntity;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.InputStreamEntity;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.lucene.util.IOUtils;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.impl.CloudSolrClient;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.common.cloud.Aliases;
+import org.apache.solr.common.cloud.ZkStateReader;
+import org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor;
+import org.apache.solr.util.DateMathParser;
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+/**
+ * Direct http tests of the CreateRoutedAlias functionality.
+ */
+public class CreateRoutedAliasTest extends SolrCloudTestCase {
--- End diff --

Hmm; it'd be nice to combine this with the existing test to just have one 
TimeRoutedAliases test.  It can wait.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322990#comment-16322990
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161074624
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -45,13 +147,92 @@ public CreateAliasCmd(OverseerCollectionMessageHandler 
ocmh) {
   public void call(ClusterState state, ZkNodeProps message, NamedList 
results)
   throws Exception {
 final String aliasName = message.getStr(NAME);
-final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
-final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
-
 ZkStateReader zkStateReader = ocmh.zkStateReader;
-validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+ZkStateReader.AliasesManager holder = zkStateReader.aliasesHolder;
+if (!anyRoutingParams(message)) {
+  final List canonicalCollectionList = 
parseCollectionsParameter(message.get("collections"));
+  final String canonicalCollectionsString = 
StrUtils.join(canonicalCollectionList, ',');
+  validateAllCollectionsExistAndNoDups(canonicalCollectionList, 
zkStateReader);
+  holder.applyModificationAndExportToZk(aliases -> 
aliases.cloneWithCollectionAlias(aliasName, canonicalCollectionsString));
+} else {
+  final String routedField = message.getStr(ROUTING_FIELD);
+  final String routingType = message.getStr(ROUTING_TYPE);
+  final String tz = message.getStr(TZ);
+  final String start = message.getStr(START);
+  final String increment = message.getStr(ROUTING_INCREMENT);
+  final String maxFutureMs = message.getStr(ROUTING_MAX_FUTURE);
+
+  try {
+if (0 > Long.valueOf(maxFutureMs)) {
--- End diff --

use parseLong


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322991#comment-16322991
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161078056
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -68,34 +249,100 @@ public void call(ClusterState state, ZkNodeProps 
message, NamedList results)
 Thread.sleep(100);
   }
 
+  private Map buildAliasMap(String routedField, String 
routingType, String tz, String increment, String maxFutureMs, ZkNodeProps 
collectionProps) {
+Map properties = collectionProps.getProperties();
+Map cleanMap = properties.entrySet().stream()
+.filter(stringObjectEntry ->
+!"fromApi".equals(stringObjectEntry.getKey())
--- End diff --

These 3 constants here are unfortunate maybe they could be in a 
constant Set and checked here?


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322989#comment-16322989
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161071261
  
--- Diff: 
solr/core/src/java/org/apache/solr/cloud/OverseerCollectionMessageHandler.java 
---
@@ -218,6 +218,7 @@ public OverseerCollectionMessageHandler(ZkStateReader 
zkStateReader, String myId
 .put(RELOAD, this::reloadCollection)
 .put(DELETE, new DeleteCollectionCmd(this))
 .put(CREATEALIAS, new CreateAliasCmd(this))
+.put(CREATEROUTEDALIAS, new CreateAliasCmd(this))
--- End diff --

Hmm; is there a point to this?  I guess it's debatable.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322988#comment-16322988
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161071815
  
--- Diff: solr/core/src/java/org/apache/solr/cloud/CreateAliasCmd.java ---
@@ -30,13 +44,101 @@
 import org.apache.solr.common.cloud.ZkStateReader;
 import org.apache.solr.common.util.NamedList;
 import org.apache.solr.common.util.StrUtils;
+import org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor;
+import org.apache.solr.util.DateMathParser;
+import org.apache.solr.util.TimeZoneUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
+import static java.time.format.DateTimeFormatter.ISO_INSTANT;
+import static 
org.apache.solr.cloud.OverseerCollectionMessageHandler.COLL_CONF;
+import static org.apache.solr.common.SolrException.ErrorCode.BAD_REQUEST;
 import static org.apache.solr.common.params.CommonParams.NAME;
+import static org.apache.solr.common.params.CommonParams.TZ;
+import static 
org.apache.solr.handler.admin.CollectionsHandler.ROUTED_ALIAS_COLLECTION_PROP_PFX;
+import static 
org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessor.DATE_TIME_FORMATTER;
 
 
 public class CreateAliasCmd implements Cmd {
+
+  private static final Logger log = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+  public static final String ROUTING_TYPE = "router.name";
+  public static final String ROUTING_FIELD = "router.field";
+  public static final String ROUTING_INCREMENT = "router.interval";
+  public static final String ROUTING_MAX_FUTURE = "router.max-future-ms";
+  public static final String START = "start";
+  // Collection constants should all reflect names in the v2 structured 
input for this command, not v1
+  // names used for CREATE
+  public static final String CREATE_COLLECTION_CONFIG = 
"create-collection.config";
--- End diff --

Ugh, all those CREATE_COLLECTION_* constants are a maintenance burden; 
surely we can avoid them?  I really want to avoid double-specificity of the 
collection-creation stuff in any shape or form (not in docs/constants/etc.)


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322987#comment-16322987
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161070349
  
--- Diff: 
solr/core/src/java/org/apache/solr/update/processor/TimeRoutedAliasUpdateProcessor.java
 ---
@@ -99,6 +101,8 @@
   .parseDefaulting(ChronoField.HOUR_OF_DAY, 0)
   .parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
   .parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
+  // Setting a timezone here is fine as a default, but generally need 
to clone with user's timezone, so that
+  // truncation of milliseconds is consistent.
--- End diff --

As I indicated in a previous comment on JIRA, this comment isn't true.  We 
deliberately make the formatted collection name *insensitive* to the timezone.  
Heck we ought to say that here.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322985#comment-16322985
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161068781
  
--- Diff: 
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java ---
@@ -476,6 +451,31 @@ private static void 
addStatusToResponse(NamedList results, RequestStatus
   SolrIdentifierValidator.validateAliasName(req.getParams().get(NAME));
   return req.getParams().required().getAll(null, NAME, "collections");
 }),
+CREATEROUTEDALIAS_OP(CREATEROUTEDALIAS, (req, rsp, h) -> {
+  String alias = req.getParams().get(NAME);
+  SolrIdentifierValidator.validateAliasName(alias);
+  Map params = req.getParams().required()
+  .getAll(null, REQUIRED_ROUTING_PARAMS.toArray(new 
String[REQUIRED_ROUTING_PARAMS.size()]));
+  req.getParams().getAll(params, NONREQUIRED_ROUTING_PARAMS);
+  // subset the params to reuse the collection creation/parsing code
+  ModifiableSolrParams collectionParams = 
extractPrefixedParams("create-collection.", req.getParams());
+  if (collectionParams.get(NAME) != null) {
+SolrException solrException = new SolrException(BAD_REQUEST, 
"routed aliases calculate names for their " +
+"dependent collections, you cannot specify the name.");
+log.error("Could not create routed alias",solrException);
--- End diff --

I believe the general practice for BAD_REQUEST param validation is for Solr 
to not log it.  Besides, it's redundant with throwing the exception, and any 
exception thrown could be chosen to log or not centrally at some entrypoint 
juncture or by a thread handler for uncaught exceptions.  So lets not litter 
log statements that are superfluous unless there's some interesting/useful 
context.



> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322986#comment-16322986
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

Github user dsmiley commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/304#discussion_r161070053
  
--- Diff: 
solr/core/src/java/org/apache/solr/update/processor/TimeRoutedAliasUpdateProcessor.java
 ---
@@ -252,6 +256,7 @@ public void processAdd(AddUpdateCommand cmd) throws 
IOException {
   /** Computes the timestamp of the next collection given the timestamp of 
the one before. */
   public static Instant computeNextCollTimestamp(Instant fromTimestamp, 
String intervalDateMath, TimeZone intervalTimeZone) {
 //TODO overload DateMathParser.parseMath to take tz and "now"
+// GH: I don't think that's necessary, you can set the TZ on now when 
you pass it in?
--- End diff --

True that it's not necessary; it's just a proposed convenience.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322880#comment-16322880
 ] 

ASF GitHub Bot commented on SOLR-11722:
---

GitHub user nsoft opened a pull request:

https://github.com/apache/lucene-solr/pull/304

SOLR-11722

patch #3 including docs, test SSL fix and precommit fixes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nsoft/lucene-solr SOLR-11722

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/304.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #304


commit aadae0c0e03bff019eac819919430b7c6eb7aabd
Author: Gus Heck 
Date:   2018-01-11T20:09:25Z

SOLR-11722 patch #3 including docs, test SSL fix and precommit fixes




> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322448#comment-16322448
 ] 

Gus Heck commented on SOLR-11722:
-

Looks like the common thing  for this is to to add @SolrTestCaseJ4.SuppressSSL 
which seems to show up on about 30 tests... 

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321433#comment-16321433
 ] 

Gus Heck commented on SOLR-11722:
-

Hmm I seem to have a reproducable intermitent failure in my test, I had seen 
this earlier, but it didn't look related and I didn't see it after updating, 
but now it's back... 

ant test  -Dtestcase=CreateRoutedAliasTest -Dtests.method=testV1 
-Dtests.seed=25994AC1D6CA0163 -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH 
-Dtests.timezone=Pacific/Johnston -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

leads to:
{code}
   [junit4] ERROR   0.16s | CreateRoutedAliasTest.testV1 <<<
   [junit4]> Throwable #1: javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
valid certification path to requested target
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([25994AC1D6CA0163:8A578DB19727ADF3]:0)
   [junit4]>at 
sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
   [junit4]>at 
sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
   [junit4]>at 
sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
   [junit4]>at 
sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
   [junit4]>at 
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
   [junit4]>at 
sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
   [junit4]>at 
sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
   [junit4]>at 
sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
   [junit4]>at 
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
   [junit4]>at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
   [junit4]>at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
   [junit4]>at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
   [junit4]>at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
   [junit4]>at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
   [junit4]>at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
   [junit4]>at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
   [junit4]>at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
   [junit4]>at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
   [junit4]>at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
   [junit4]>at 
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   [junit4]>at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
   [junit4]>at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   [junit4]>at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   [junit4]>at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
   [junit4]>at 
org.apache.solr.cloud.CreateRoutedAliasTest.testV1(CreateRoutedAliasTest.java:164)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
   [junit4]> Caused by: sun.security.validator.ValidatorException: PKIX 
path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
valid certification path to requested target
   [junit4]>at 
sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
   [junit4]>at 
sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
   [junit4]>at 
sun.security.validator.Validator.validate(Validator.java:260)
   [junit4]>at 
sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
   [junit4]>at 
sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
   [junit4]>at 
sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
   [junit4]>at 
sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
   [junit4]>... 59 more
   [junit4]> Caused by: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
valid certification path to requested target
   [junit4]>at 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321275#comment-16321275
 ] 

Gus Heck commented on SOLR-11722:
-

Attached a patch, I believe I addressed the above comments I did switch to 
using LocalSolrRequest, but that did not alleviate the need for the v2-v1 and 
parseCollectionCreationProps changes because they get utilized in the _OP enum 
to masage the incomming parameter before requests can be made. rather than in 
the Cmd class where delegation to the Command infrastructure via 
LocalSolrRequest happens. 

Names should be mostly in sync with your changes in SOLR-11653, and I extracted 
your collection creation code into createCollectionAndWait() and reused that 
(which is how I picked up LocalSolrRequest :) ).

What's not in this patch is documentation, but I want to get this out where you 
and others can look at it. Working on docs now, which could be added as an 
independent patch if you want to commit this one.


> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch, SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307495#comment-16307495
 ] 

David Smiley commented on SOLR-11722:
-

RE V2/V1, the "properties" mechanism looks promising.  I think it's totally 
fine to have _introspect not expose the collection creation particulars of time 
routed alias. We can add docs to help users knows some do's and don'ts.

Yeah naming stuff is hard.  I'm good with "collection-create"; it's not bad.

RE alias routing, I think it's a dead-end to try to invent names similar to 
"router" but not use "router" simply because shards have a router using the 
name "router". This is the same concept, but we are applying it to this new 
"routed alias" thing as opposed to shards.  And we have a new type/name 
expressly for time routing.  If some new thing comes along in Solr that might 
be routed (which I admit seems unlikely), it might then also try to avoid using 
the "route" word... hence it feels wrong to try to have them use separate 
words.  It's the same concept.  But since the location of use is different that 
gives context, I think it's fine to use the same parameters as there isn't 
ambiguity.  Here we're adding alias metadata parameters, and so it applies to 
the alias.  In collection creation, the router is name-spaced in the context of 
_that_.  Hence I propose we use these metadata fields: router.name=time, 
router.field=myTimestampField, router.interval=+1DAY, and it's debatable if TZ 
is router.TZ or just TZ.  "start" isn't metadata, it's a param of this command 
that isn't used after.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307276#comment-16307276
 ] 

Gus Heck commented on SOLR-11722:
-

*Routing Field:* I'm quite happy if we don't need to check the routing field 
for existence. That would be very difficult, I just felt it slightly weird that 
I was able to validate all the other inputs and not that.

*GregorianCalendar:*  I'll certainly move to the new API, old habits... thx.

*parseCollectionCreationProps:* I'll look at your way with LocalSolrRequest. I 
was never fond of having to touch the create command, but I really really 
didn't want to duplicate it 

*duplication in collections.Commands.json:* I definitely also hate this, but I 
think the v2 api is going to reject anything it doesn't recognize and also if 
there's nothing there, then there's no "self documenting" for v2 for all the 
collection creation bits. Even worse, there are some differences in what we 
accept vs the create command. Most notably we can't accept a name for the 
collection (although we possibly could turn that into a prefix, we would need 
to document that, meaning). Also async and waitForState need to be trimmed off 
as they don't make sense as part of a sub-command. Also the description needs 
to differ...

I don't think the v2 API design anticipated command reuse that was user 
configurable. If we worked from an example collection, this issue would go 
away, but your points on that front with respect to extra steps and replica 
allocation remain valid. If we could rename the example collection, that second 
issue would get a lot better, but alas there is no collection renaming aside 
from aliases AFAICT, and I don't think we want to go there... 

So solve this duplication and keep things pretty and normalize in the JSON, it 
would need to move to JSOG or some other strategy for cross linking in JSON, 
and furthermore facilitate masking/overlay to edit out and replace the parts 
that need to differ in the create-routed-alias context. That's obviously a 
huge, unreasonable change to include within this ticket.

However as I write this, I recall noticing during my testing of this that 
properties is of type object, and has no sub-documentation. I suspect we could 
just omit things in the same fashion. So maybe V2 is more tolerant than I 
thought. Initially it obviously complained about not having a collection-create 
element... It looks like if we set additional properties to true, we could 
leave properties blank, Not sure if that's going to work at that level, or with 
the nested stuff though. If it does work, we could omit this *at the expense of 
not completely documenting what we provide in _introspect*... 

*V2 to V1:* The v2 to v1 translation stuff was trapped in a private method 
where it modified the request object by wrapping the params in an anonymous 
subclass of SolrParams that returns V1 params instead of V2 params. This didn't 
suit us because our request object is for the create-routed-alias command and 
we don't want it modified. It might be that moving to a LocalSolrRequest as you 
suggest avoids the need to extract this. 

That said I wonder about this code in general... there's lots of usages of 
setParams out there, someone down stream of one of those may be expecting a 
particular subclass of SolrParams

*"collection-create":* yeah, not the greatest name. Threw it in there to stop 
my minutes of name pondering and get on with things. Maybe "create-with" ? or 
"create-options" or "create-params"? ... "cparams" or "copts" is nice and 
concise, but cryptic. 

Hoping others have an opinion here, naming stuff is hard :)

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to 

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307068#comment-16307068
 ] 

David Smiley commented on SOLR-11722:
-

Correcting myself... calculateStart should factor in the TZ parameter because 
of the possible presence of date math.  I think the easiest way to do this 
would be to modify DateMathParser.parseMath to be overloaded with a variant 
that accepts a TZ parameter.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307065#comment-16307065
 ] 

David Smiley commented on SOLR-11722:
-

Another Q: in the collections.Commands.json file was it necessary to repeat all 
the create-collection stuff?  I think the maintenance cost of repeating this is 
way too high.

I was thinking of the "collection-create." prefix to collection creation 
params.  I guess it's okay.  My idealist self says it should just be 
"collection." in spite of the awkwardness of having a 
"collection.collection.configName", and then in some other issue work to fix 
that poor name.  But I admit I don't have the energy to bother with the 
thankless drudgery of renaming such a commonly used parameter like that.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307062#comment-16307062
 ] 

David Smiley commented on SOLR-11722:
-

Hey, excellent progress!  My review follows...

The TZ is only to be used as an argument to DateMathParser's constructor which 
we use when applying the increment.  It is not to be used to format the 
collection names.  If it were, we could have nasty bugs if the TZ were changed 
since it would change the interpretation of preexisting collections.

I disagree that we should check that the routing field exists in the schema.  
Shard routing doesn't, FWIW.

calculateStart() should do little more than 
DateMathParser.parseMath(null,start).  See its javadocs.  It's used all over 
Solr to parse a date (that may but needn't have date math).  To check that 
there are no milliseconds, simply check that the result of 
instant.truncatedTo(ChronoUnit.SECONDS) is equal to the non-truncated version.  
Please avoid GregorianCalendar like the plauge (old API).

In CollectionsHandler, I see you refactored out a parseCollectionCreationProps 
method.  Okay... I can see the point of that as it makes it more concise to 
call this but probably only slightly.  The alternative (which I do in 
SOLR-11653) is to call CollectionOperation.CREATE_OP.execute using a 
LocalSolrRequest wrapper.  What do you think?

I don't quite grok the v2/v1 translation stuff yet; I'll need to look at that 
closer some other day.  But it seems unfortunate that you have to go out of 
your way to make this work just to add some new command, right?  Maybe there is 
an easier way; I don't know.  I guess we can look closer once the patch is more 
ready (i.e. tests).

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
> Attachments: SOLR-11722.patch
>
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294390#comment-16294390
 ] 

Gus Heck commented on SOLR-11722:
-

Sorry it's been so long to update here, life got "interesting". I see your 
point with respect to replica allocation. We can make the template collection 
something added later if folks seem to want it. I've gone partway into 
implementing, the timezone thing is somewhat complicated by the truncation of 
the milliseconds in the pattern for the collection name, which probably 
requires that the formatter doing the truncation to have a matching timezone 
(timezones are political entities, not necessarily bound by sensibilities or 
mathematical convenience, so I'm not confident about assuming that none of them 
ever messed with (or will mess with) milliseconds) 

I'm looking into the way the router property works, I think we can do something 
similar, though I'm not sure that the code for that is directly reusable. I'm 
working on a patch which will hopefully be presentable soon.

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278751#comment-16278751
 ] 

David Smiley commented on SOLR-11722:
-

I can see the value in an "example collection" but I think this requires the 
user to do more in most cases.  They have to create this dummy collection 1st, 
which obviates some of the convenience we're trying to offer.  This dummy 
collection might take up resources (e.g. cores on some nodes) that will prevent 
choosing some nodes for future replica allocations.  If you want to add this as 
a convenience option then it seems fine to me, but not mandatory.  If you want 
to keep it, I suggest the name "template collection" rather than "example".

RE timezones:  It'd be nice to call it "TZ" to be consistent with use of it 
elsewhere.  But not a big deal.  I think the choice of how collection names are 
to be formatted here took place in SOLR-11542 to be released in Solr 7.2 
shortly.  I agree we don't want the user to read the code to know such things; 
so at the very least we should document.  The docs on this timezone param 
should spell this out.

By the way, notice how the create-collection command has a "router" _property_ 
that itself has sub-properites of "name" and "field".  That's interesting.  
[~noble.paul] how does one express that as an API command?  It's not clear 
reading 
https://builds.apache.org/job/Solr-reference-guide-master/javadoc/v2-api.html 
which is what I've been using as the docs on this V2 API.  [~noble.paul] are we 
required to document each parameter or can we say that properties starting with 
"createCollection." are mapped to collection creation parameters?

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11722) API to create a Time Routed Alias and first collection

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277892#comment-16277892
 ] 

Gus Heck commented on SOLR-11722:
-

I think we can handle the creation of the properly formatted collection name 
and maybe we can pull the creation metadata from an example collection? The 
example collection can be retained for future use, modified or deleted as the 
user sees fit as we will still record any parameters we care about as alias 
metadata. This should be nice and convenient for folks initially investigating 
this feature, and it also avoids duplicating configuration parameters already 
defined elsewhere here. Someone checking it out for the first time can just 
throw the name of an existing collection with a time valued field in it and 
start playing. 

Also, in thinking about usability I suspect we may want to allow a full time 
stamp including timezone specifier in the names for collections so that users 
don't have to have dig into zookeeper or the collections api to determine what 
timezone is in play. (they could even perhaps find out using {{=\[shard]}}). 
Such a thing would however require adjustment of the regular expressions in 
TimeRoutedAliasUpdateProcessor.

How about a set of options like this? (expressed here as an addition to the 
v2api _introspect for create-alias)
{code}
"collection-routing": {
  "type": "string",
  "description": "The type of routing to perform. Currently only 'time' 
is supported. This property is mutually exclusive with 'collections', and 
required with any route-* property. Collections created will be named for the 
alias and the first value allowed for the partition represented by each 
collection separated by an underscore. Time based routing will express the 
first value as a time stamp including the applicable timezone specifier (only 
time based routing is supported at this time)"
},
"route-field": {
  "type": "string",
  "description": "The field in incoming documents that is consulted to 
decide which collection the document should be routed to."
},
"route-start": {
  "type": "string",
  "description": "The earliest/lowest value for routeField that may be 
indexed into this alias. Documents with values less than this will return an 
error. For time based routing this may be a date math expression."
},
"route-timezone": {
  "type": "string",
  "description": "Optional timezone for time based routing. If omitted, 
UTC timezone will be used"
},
"route-increment": {
  "type": "string",
  "description": "A specification of the width of the interval for each 
partition collection. For time based routing this should be a date math 
expression fragment starting with the + character. When appended to a standard 
iso date the result should be a valid date math expression."
},
"route-example-collection": {
  "type": "string",
  "description": "A collection to use as an initial example for the 
creation of the initial partition of a collection routing alias. Subsequent 
partitions will ignore this value and will take configuration values (config, 
numShards, etc) from the first partition."
},
{code}

> API to create a Time Routed Alias and first collection
> --
>
> Key: SOLR-11722
> URL: https://issues.apache.org/jira/browse/SOLR-11722
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>
> This issue is about creating a single API command to create a "Time Routed 
> Alias" along with its first collection.  Need to decide what endpoint URL it 
> is and parameters.
> Perhaps in v2 it'd be {{/api/collections?command=create-routed-alias}} or 
> alternatively piggy-back off of command=create-alias but we add more options, 
> perhaps with a prefix like "router"?
> Inputs:
> * alias name
> * misc collection creation metadata (e.g. config, numShards, ...) perhaps in 
> this context with a prefix like "collection."
> * metadata for TimeRoutedAliasUpdateProcessor, currently: router.field
> * date specifier for first collection; can include "date math".
> We'll certainly add more options as future features unfold.
> I believe the collection needs to be created first (referring to the alias 
> name via a core property), and then the alias pointing to it which demands 
> collections exist first.  When figuring the collection name, you'll need to 
> reference the format in TimeRoutedAliasUpdateProcessor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: