[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750028#comment-16750028
]
ASF subversion and git services commented on SOLR-9735:
---
Commit a3fe51ffac903d674cf38c0c1ea78a63f677fdc6 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a3fe51f ]
SOLR-9735: Update CHANGES and the ref guide.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750026#comment-16750026
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 70b300f539778243a3fb3e41d10a070fb18c0a91 in lucene-solr's branch
refs/heads/branch_8x from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=70b300f ]
SOLR-9735: Update CHANGES and the ref guide.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749948#comment-16749948
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 6a4ef4ea3dc67a21508f9b001d088e61e188a407 in lucene-solr's branch
refs/heads/branch_7x from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6a4ef4e ]
SOLR-9735: Fix v2 API for AutoscalingHistoryHandler.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749949#comment-16749949
]
ASF subversion and git services commented on SOLR-9735:
---
Commit d6c0e9a8714e21532f6b64ac3756918517723c11 in lucene-solr's branch
refs/heads/branch_7x from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d6c0e9a ]
SOLR-9735: Update CHANGES and the ref guide.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220536#comment-16220536
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 99f5760142330ea4c00de9d1a17807f913aab6be in lucene-solr's branch
refs/heads/master from [~ab]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=99f5760 ]
SOLR-9735: Fix v2 API for AutoscalingHistoryHandler.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220368#comment-16220368
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
A note for people following this issue -- we intend to do all further
development for the sub-tasks in this issue on the master branch itself.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220252#comment-16220252
]
ASF subversion and git services commented on SOLR-9735:
---
Commit eb230e13f0ae400b4e8d108fed85c75a7600947d in lucene-solr's branch
refs/heads/branch_7_1 from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=eb230e1 ]
SOLR-9735: Fix autoscaling.Commands.json structure
(cherry picked from commit b5d3d24)
(cherry picked from commit 7a3491d)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220253#comment-16220253
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 50a99fcd91b8d1fd07a4287e99c7eb5b2aca61fb in lucene-solr's branch
refs/heads/branch_7_1 from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=50a99fc ]
SOLR-9735: Enable randomized testing between v1 and v2 APIs
(cherry picked from commit 0fad80b)
(cherry picked from commit 0dbfcdb)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220250#comment-16220250
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 0dbfcdbd7c1231779a42258838384ef65db2fda1 in lucene-solr's branch
refs/heads/branch_7x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0dbfcdb ]
SOLR-9735: Enable randomized testing between v1 and v2 APIs
(cherry picked from commit 0fad80b)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220249#comment-16220249
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 7a3491dc9360b6c6a0d172e3c15598501330a2f9 in lucene-solr's branch
refs/heads/branch_7x from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7a3491d ]
SOLR-9735: Fix autoscaling.Commands.json structure
(cherry picked from commit b5d3d24)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220247#comment-16220247
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 0fad80b4094fa7e06123839f5f2735065c0bc329 in lucene-solr's branch
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0fad80b ]
SOLR-9735: Enable randomized testing between v1 and v2 APIs
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220235#comment-16220235
]
ASF subversion and git services commented on SOLR-9735:
---
Commit b5d3d242ef909cb80d6ccf2a187f06cbcb642bc8 in lucene-solr's branch
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b5d3d24 ]
SOLR-9735: Fix autoscaling.Commands.json structure
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197150#comment-16197150
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 2f1d5174bfed5e7cac398c1f88f161b17deffc5d in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2f1d517 ]
SOLR-9735: Fixed precommit errors
(cherry picked from commit 15d17ed)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197149#comment-16197149
]
ASF subversion and git services commented on SOLR-9735:
---
Commit e695acc28b2cb9ad1010336b97ee074529e47dbf in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e695acc ]
SOLR-9735: Fixed precommit errors
(cherry picked from commit 3012239)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197151#comment-16197151
]
ASF subversion and git services commented on SOLR-9735:
---
Commit ecfb2abdabf79c644019b031dc495638e49b2000 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ecfb2ab ]
SOLR-9735: fix precommit errors after merging autoscaling branch into master
(cherry picked from commit fe38dca)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197148#comment-16197148
]
ASF subversion and git services commented on SOLR-9735:
---
Commit c2633b85576b75568a56e71bd1df422639b4fc7e in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c2633b8 ]
SOLR-9735: license header missing
(cherry picked from commit b09c99f)
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196859#comment-16196859
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
Noble merged the autoscaling branch to master last week. The tests look stable
and we haven't seen any new failures. I am going to merge these changes to
branch_7x now.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170464#comment-16170464
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
There are a good chunk of features that have accumulated in the
feature/autoscaling branch. I am going to review the remaining nocommits and
tests so that we can merge this branch to master. I'll hold off merging to
branch_7x until we can finish SOLR-11085.
We have another branch named feature/autoscaling_72 which contains features
that we plan to release in 7.2. Once autoscaling branch is merged to master, we
can get rid of autoscaling_72 and continue further development on the
autoscaling branch itself.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124773#comment-16124773
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
Yes, that is because autoscaling features haven't been released yet. Once the
ref guide for 7.0 is out then that link will go to the actual page describing
the APIs.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124668#comment-16124668
]
David Smiley commented on SOLR-9735:
NP Shalin. BTW the URL you added:
https://lucene.apache.org/solr/guide/solrcloud-autoscaling-api.html does a
redirect to:
https://lucene.apache.org/solr/guide/6_6/solrcloud-autoscaling-api.html which
404's.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124481#comment-16124481
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
Thanks [~dsmiley] for catching this. I have fixed the nocommit as well as
enabled validation checks for JSON files.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123405#comment-16123405
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 67de1a3f70ad04a8f040dddfa80ccd6ac1277f15 in lucene-solr's branch
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=67de1a3 ]
SOLR-9735: Replace nocommit with link to ref guide document
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122110#comment-16122110
]
David Smiley commented on SOLR-9735:
It appears that the precommit detection of "nocommit" string does not look in
.json documents. I see it in
solr/solrj/src/resources/apispec/autoscaling.Commands.json for documentation
that is nonexistent. build.xml line 137 (ant target
"-validate-source-patterns") references file extensions but not .json. Can you
fix this and the actual nocommit [~shalinmangar]?
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076096#comment-16076096
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
Linking to SOLR-11005
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061149#comment-16061149
]
ASF subversion and git services commented on SOLR-9735:
---
Commit b3c3f596d9242bffed6e177bb4ef609e2b0388a6 in lucene-solr's branch
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b3c3f59 ]
SOLR-9735: fix Recovered WARNINGs in autoscaling/Policy.applyRules()
('Internal inconsistency detected during lambda shape analysis') is the
(Recovered) warning.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047144#comment-16047144
]
Varun Thacker commented on SOLR-9735:
-
Here was the initial commit which didn't get caught by the Jira bot for
reference:
https://github.com/apache/lucene-solr/commit/e5d8ed397ab8db3268e1de86ca5ee5fe53dc04cc
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038164#comment-16038164
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 2a625dfb1ff6d71cb2435e3efeefafe80746cd72 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2a625df ]
SOLR-9735: validation of all input values to either string or long
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038165#comment-16038165
]
ASF subversion and git services commented on SOLR-9735:
---
Commit e36d251751143ba36e06defbca8de4ff228483a0 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e36d251 ]
SOLR-9735: sysLoadAverage and headusage are now double
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038162#comment-16038162
]
ASF subversion and git services commented on SOLR-9735:
---
Commit e2ae9992c5fdb3e729c987467ab6e121021f3319 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e2ae999 ]
SOLR-9735: validation for specific tags
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038163#comment-16038163
]
ASF subversion and git services commented on SOLR-9735:
---
Commit 4e691f9e4ae4269d4fbf8d947cc82bb7123085e7 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4e691f9 ]
SOLR-9735: validation of all input values to either string or long
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038076#comment-16038076
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
The feature/autoscaling_solr7 is ready for merge to master. We've mostly been
adding tests and fixing the occasional bug the last week. I'm going to merge it
now so that Jenkins can start chewing on it.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028583#comment-16028583
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
It has been a couple of months since we shared the initial design document. So
a quick update is in order. The policy and preference engine is mostly done so
that is going to be the first thing that we will merge to master. The trigger
support and rest of the autoscaling features are still being worked on and it
is likely that they won't have enough testing before Solr 7. So our plan is to
create a new branch off of master with just the bits from autoscaling branch
that we feel are ready for merge/release in Solr 7. These include:
# The policy and preference engine
# The associated set-policy, set-cluster-policy and set-cluster-preferences API
# The autoscaling read and diagnostics API
# MOVEREPLICA API (which is already present in master)
# SOLR-10419 -- which changes all collection APIs to use the policy engine to
place replicas
I'll shortly be creating a feature/autoscaling_solr7 branch for these bits and
we hope to merge it to master this week. I'll add a comment here once the
branch is ready for review.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936267#comment-15936267
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
Please access the [first iteration of the design
here|https://docs.google.com/document/d/1QX6AyOkceQs-PlQUW_TE4jb9N2ZgL7GFL7Ao7yl7fBM/edit?usp=sharing].
Anyone with the link can comment on the google document without signing-in.
It outlines the basic concepts and the vocabulary we will use to discuss and
implement this feature. It also has examples of various new APIs that will be
written to support this feature.
A good part of the design and terminology was conceived by the open source team
at Lucidworks during our in-person meetings at San Francisco in February. Noble
Paul and I are the primary authors of the document but it includes valuable
ideas and feedback from all members of the open source team viz. Andrzej,
Cassandra, Dat, Erick, Ishan, Hoss, Steve and Trey. This contribution is being
sponsored by Lucidworks Inc.
This is very much a living document and will continue to be refined as we think
through various use-cases and edge cases. Please feel free to share your
feedback and comments here.
I have already created a features/autoscaling branch for this work on the
public git repo and we've begun prototyping key pieces of the design for which
we'll open sub-issues.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9735) Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893910#comment-15893910
]
Shalin Shekhar Mangar commented on SOLR-9735:
-
We (Lucidworks) would like to push this forward. We are working on a tentative
design that we will share soon for feedback. I'll create a feature/autoscaling
branch off of master for collaboration.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> --
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Anshum Gupta
>Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
