Re: Yet another repair solution
A small update here. There is now a 1.0.0 release available of our repair scheduler solution. https://github.com/ericsson/ecchronos There is a binary distribution available at https://search.maven.org/se arch?q=g:%22com.ericsson.bss.cassandra.ecchronos%22%20AND%20a:%22ecchro nos-binary%22 if you want to try it out. Best Regards Marcus Olsson On tis, 2018-09-11 at 13:35 +, Marcus Olsson wrote: > Sure thing! > > Up until now it has been running in an OSGi environment, so among > other > things I'm working towards both OSGi and a standalone application. > > It's designed to be tightly coupled with a single instance, where it > keeps track of the repair state and performs repair of tables for > that > node only. > The current features include alarms, "pausing repairs", metrics, > dynamic scheduling and "pluggability" for each of them (as well as > some > other components like connection management, lease management, etc). > > The design is based on CASSANDRA-10070 with Cassandra (and LWT) as a > default backend for the lease management. It utilizes the repair > history from Cassandra to determine repair state of tables in order > to > prioritize and schedule them. This also means that a manual "nodetool > repair" would be counted towards the repair state of the tables. > > Best Regards > Marcus Olsson > > On tor, 2018-08-30 at 07:55 -0700, Dinesh Joshi wrote: > > > > In the meanwhile, do you think you could highlight the features of > > your repair solution / sidecar? > > > > Dinesh > > > > > > > > > > > On Aug 30, 2018, at 4:57 AM, Marcus Olsson > > n. > > > com> wrote: > > > > > > Great to see that there is an interest! As there currently are > > > some > > > internal dependencies etc. in place there is still some work to > > > be > > > done before we can publish it. I would expect this to take at > > > least > > > a few weeks, to try set the correct expectations. > > > > > > Best Regards > > > Marcus Olsson > > > > > > On tis, 2018-08-28 at 23:18 -0700, Vinay Chella wrote: > > > I am excited to see that the community is working on solving the > > > critical > > > problems in C* operations (e.g., repair, backups etc.,) with > > > different > > > solutions. Of course, learnings from these systems are key to > > > designing the > > > robust solution which works for everyone. > > > > > > > > > Thanks, > > > Vinay Chella > > > > > > > > > On Tue, Aug 28, 2018 at 1:23 PM Roopa > > al > > > id<mailto:rtangir...@netflix.com.invalid>> > > > wrote: > > > > > > > > > +1 interested in seeing and understanding another repair > > > solution. > > > > > > > > > On Aug 28, 2018, at 1:03 PM, Joseph Lynch > > ma > > > ilto:joe.e.ly...@gmail.com>> wrote: > > > > > > I'm pretty interested in seeing and understanding your solution! > > > When we > > > started on CASSANDRA-14346 reading your design documents and plan > > > you > > > sketched out in CASSANDRA-10070 were really helpful in improving > > > our > > > design. I'm particularly interested in how the Scheduler/Job/Task > > > APIs > > > turned out (we're working on something similar internally and > > > would > > > love > > > to > > > > > > compare notes and figure out the best way to implement that kind > > > of > > > abstraction)? > > > > > > -Joey > > > > > > > > > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson < > > > marcus.ols...@ericsson.com<mailto:marcus.ols...@ericsson.com>> > > > > > > wrote: > > > > > > > > > Hi, > > > > > > With the risk of stirring the repair/side-car topic even further > > > I'd > > > just > > > > > > > > > like to mention that we have recently gotten approval to > > > contribute > > > our > > > repair management side-car solution. > > > It's based on the proposal in > > > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a > > > standalone > > > application sitting next to each instance. > > > With the recent discussions in mind I'd just like to hear the > > > thoughts > > > from the community on this before we put in the effort of > > > bringing > > > our > > > solution into open source. > > > > > > Would there be an interest of having yet another repair solution > > > in > > > the > > > discussion? > > > > > > Best Regards > > > Marcus Olsson > > > > > > - > > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > to > > > :dev-unsubscr...@cassandra.apache.org> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > il > > > to:dev-h...@cassandra.apache.org> > > > > > > > > - > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Yet another repair solution
Sure thing! Up until now it has been running in an OSGi environment, so among other things I'm working towards both OSGi and a standalone application. It's designed to be tightly coupled with a single instance, where it keeps track of the repair state and performs repair of tables for that node only. The current features include alarms, "pausing repairs", metrics, dynamic scheduling and "pluggability" for each of them (as well as some other components like connection management, lease management, etc). The design is based on CASSANDRA-10070 with Cassandra (and LWT) as a default backend for the lease management. It utilizes the repair history from Cassandra to determine repair state of tables in order to prioritize and schedule them. This also means that a manual "nodetool repair" would be counted towards the repair state of the tables. Best Regards Marcus Olsson On tor, 2018-08-30 at 07:55 -0700, Dinesh Joshi wrote: > In the meanwhile, do you think you could highlight the features of > your repair solution / sidecar? > > Dinesh > > > > > On Aug 30, 2018, at 4:57 AM, Marcus Olsson > com> wrote: > > > > Great to see that there is an interest! As there currently are some > > internal dependencies etc. in place there is still some work to be > > done before we can publish it. I would expect this to take at least > > a few weeks, to try set the correct expectations. > > > > Best Regards > > Marcus Olsson > > > > On tis, 2018-08-28 at 23:18 -0700, Vinay Chella wrote: > > I am excited to see that the community is working on solving the > > critical > > problems in C* operations (e.g., repair, backups etc.,) with > > different > > solutions. Of course, learnings from these systems are key to > > designing the > > robust solution which works for everyone. > > > > > > Thanks, > > Vinay Chella > > > > > > On Tue, Aug 28, 2018 at 1:23 PM Roopa > id<mailto:rtangir...@netflix.com.invalid>> > > wrote: > > > > > > +1 interested in seeing and understanding another repair solution. > > > > > > On Aug 28, 2018, at 1:03 PM, Joseph Lynch > ilto:joe.e.ly...@gmail.com>> wrote: > > > > I'm pretty interested in seeing and understanding your solution! > > When we > > started on CASSANDRA-14346 reading your design documents and plan > > you > > sketched out in CASSANDRA-10070 were really helpful in improving > > our > > design. I'm particularly interested in how the Scheduler/Job/Task > > APIs > > turned out (we're working on something similar internally and would > > love > > to > > > > compare notes and figure out the best way to implement that kind of > > abstraction)? > > > > -Joey > > > > > > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson < > > marcus.ols...@ericsson.com<mailto:marcus.ols...@ericsson.com>> > > > > wrote: > > > > > > Hi, > > > > With the risk of stirring the repair/side-car topic even further > > I'd > > just > > > > > > like to mention that we have recently gotten approval to contribute > > our > > repair management side-car solution. > > It's based on the proposal in > > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a > > standalone > > application sitting next to each instance. > > With the recent discussions in mind I'd just like to hear the > > thoughts > > from the community on this before we put in the effort of bringing > > our > > solution into open source. > > > > Would there be an interest of having yet another repair solution in > > the > > discussion? > > > > Best Regards > > Marcus Olsson > > > > - > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > :dev-unsubscr...@cassandra.apache.org> > > For additional commands, e-mail: dev-h...@cassandra.apache.org > to:dev-h...@cassandra.apache.org> > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org >
Re: Yet another repair solution
In the meanwhile, do you think you could highlight the features of your repair solution / sidecar? Dinesh > On Aug 30, 2018, at 4:57 AM, Marcus Olsson wrote: > > Great to see that there is an interest! As there currently are some internal > dependencies etc. in place there is still some work to be done before we can > publish it. I would expect this to take at least a few weeks, to try set the > correct expectations. > > Best Regards > Marcus Olsson > > On tis, 2018-08-28 at 23:18 -0700, Vinay Chella wrote: > I am excited to see that the community is working on solving the critical > problems in C* operations (e.g., repair, backups etc.,) with different > solutions. Of course, learnings from these systems are key to designing the > robust solution which works for everyone. > > > Thanks, > Vinay Chella > > > On Tue, Aug 28, 2018 at 1:23 PM Roopa > mailto:rtangir...@netflix.com.invalid>> > wrote: > > > +1 interested in seeing and understanding another repair solution. > > > On Aug 28, 2018, at 1:03 PM, Joseph Lynch > mailto:joe.e.ly...@gmail.com>> wrote: > > I'm pretty interested in seeing and understanding your solution! When we > started on CASSANDRA-14346 reading your design documents and plan you > sketched out in CASSANDRA-10070 were really helpful in improving our > design. I'm particularly interested in how the Scheduler/Job/Task APIs > turned out (we're working on something similar internally and would love > to > > compare notes and figure out the best way to implement that kind of > abstraction)? > > -Joey > > > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson < > marcus.ols...@ericsson.com<mailto:marcus.ols...@ericsson.com>> > > wrote: > > > Hi, > > With the risk of stirring the repair/side-car topic even further I'd > just > > > like to mention that we have recently gotten approval to contribute our > repair management side-car solution. > It's based on the proposal in > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone > application sitting next to each instance. > With the recent discussions in mind I'd just like to hear the thoughts > from the community on this before we put in the effort of bringing our > solution into open source. > > Would there be an interest of having yet another repair solution in the > discussion? > > Best Regards > Marcus Olsson > > - > To unsubscribe, e-mail: > dev-unsubscr...@cassandra.apache.org<mailto:dev-unsubscr...@cassandra.apache.org> > For additional commands, e-mail: > dev-h...@cassandra.apache.org<mailto:dev-h...@cassandra.apache.org> > > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Yet another repair solution
Great to see that there is an interest! As there currently are some internal dependencies etc. in place there is still some work to be done before we can publish it. I would expect this to take at least a few weeks, to try set the correct expectations. Best Regards Marcus Olsson On tis, 2018-08-28 at 23:18 -0700, Vinay Chella wrote: I am excited to see that the community is working on solving the critical problems in C* operations (e.g., repair, backups etc.,) with different solutions. Of course, learnings from these systems are key to designing the robust solution which works for everyone. Thanks, Vinay Chella On Tue, Aug 28, 2018 at 1:23 PM Roopa mailto:rtangir...@netflix.com.invalid>> wrote: +1 interested in seeing and understanding another repair solution. On Aug 28, 2018, at 1:03 PM, Joseph Lynch mailto:joe.e.ly...@gmail.com>> wrote: I'm pretty interested in seeing and understanding your solution! When we started on CASSANDRA-14346 reading your design documents and plan you sketched out in CASSANDRA-10070 were really helpful in improving our design. I'm particularly interested in how the Scheduler/Job/Task APIs turned out (we're working on something similar internally and would love to compare notes and figure out the best way to implement that kind of abstraction)? -Joey On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson < marcus.ols...@ericsson.com<mailto:marcus.ols...@ericsson.com>> wrote: Hi, With the risk of stirring the repair/side-car topic even further I'd just like to mention that we have recently gotten approval to contribute our repair management side-car solution. It's based on the proposal in https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone application sitting next to each instance. With the recent discussions in mind I'd just like to hear the thoughts from the community on this before we put in the effort of bringing our solution into open source. Would there be an interest of having yet another repair solution in the discussion? Best Regards Marcus Olsson - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org<mailto:dev-unsubscr...@cassandra.apache.org> For additional commands, e-mail: dev-h...@cassandra.apache.org<mailto:dev-h...@cassandra.apache.org>
Re: Yet another repair solution
I am excited to see that the community is working on solving the critical problems in C* operations (e.g., repair, backups etc.,) with different solutions. Of course, learnings from these systems are key to designing the robust solution which works for everyone. Thanks, Vinay Chella On Tue, Aug 28, 2018 at 1:23 PM Roopa wrote: > +1 interested in seeing and understanding another repair solution. > > > On Aug 28, 2018, at 1:03 PM, Joseph Lynch wrote: > > > > I'm pretty interested in seeing and understanding your solution! When we > > started on CASSANDRA-14346 reading your design documents and plan you > > sketched out in CASSANDRA-10070 were really helpful in improving our > > design. I'm particularly interested in how the Scheduler/Job/Task APIs > > turned out (we're working on something similar internally and would love > to > > compare notes and figure out the best way to implement that kind of > > abstraction)? > > > > -Joey > > > > > > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson < > marcus.ols...@ericsson.com> > > wrote: > > > >> Hi, > >> > >> With the risk of stirring the repair/side-car topic even further I'd > just > >> like to mention that we have recently gotten approval to contribute our > >> repair management side-car solution. > >> It's based on the proposal in > >> https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone > >> application sitting next to each instance. > >> With the recent discussions in mind I'd just like to hear the thoughts > >> from the community on this before we put in the effort of bringing our > >> solution into open source. > >> > >> Would there be an interest of having yet another repair solution in the > >> discussion? > >> > >> Best Regards > >> Marcus Olsson > >> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: Yet another repair solution
+1 interested in seeing and understanding another repair solution. > On Aug 28, 2018, at 1:03 PM, Joseph Lynch wrote: > > I'm pretty interested in seeing and understanding your solution! When we > started on CASSANDRA-14346 reading your design documents and plan you > sketched out in CASSANDRA-10070 were really helpful in improving our > design. I'm particularly interested in how the Scheduler/Job/Task APIs > turned out (we're working on something similar internally and would love to > compare notes and figure out the best way to implement that kind of > abstraction)? > > -Joey > > > On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson > wrote: > >> Hi, >> >> With the risk of stirring the repair/side-car topic even further I'd just >> like to mention that we have recently gotten approval to contribute our >> repair management side-car solution. >> It's based on the proposal in >> https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone >> application sitting next to each instance. >> With the recent discussions in mind I'd just like to hear the thoughts >> from the community on this before we put in the effort of bringing our >> solution into open source. >> >> Would there be an interest of having yet another repair solution in the >> discussion? >> >> Best Regards >> Marcus Olsson >> - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Yet another repair solution
I'm pretty interested in seeing and understanding your solution! When we started on CASSANDRA-14346 reading your design documents and plan you sketched out in CASSANDRA-10070 were really helpful in improving our design. I'm particularly interested in how the Scheduler/Job/Task APIs turned out (we're working on something similar internally and would love to compare notes and figure out the best way to implement that kind of abstraction)? -Joey On Tue, Aug 28, 2018 at 6:34 AM Marcus Olsson wrote: > Hi, > > With the risk of stirring the repair/side-car topic even further I'd just > like to mention that we have recently gotten approval to contribute our > repair management side-car solution. > It's based on the proposal in > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone > application sitting next to each instance. > With the recent discussions in mind I'd just like to hear the thoughts > from the community on this before we put in the effort of bringing our > solution into open source. > > Would there be an interest of having yet another repair solution in the > discussion? > > Best Regards > Marcus Olsson >
Re: Yet another repair solution
I'm also very interested. On Tue, Aug 28, 2018 at 8:47 AM Dinesh Joshi wrote: > > On Aug 28, 2018, at 6:33 AM, Marcus Olsson > wrote: > > > > Hi, > > > > With the risk of stirring the repair/side-car topic even further I'd > just like to mention that we have recently gotten approval to contribute > our repair management side-car solution. > > It's based on the proposal in > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone > application sitting next to each instance. > > With the recent discussions in mind I'd just like to hear the thoughts > from the community on this before we put in the effort of bringing our > solution into open source. > > > > Would there be an interest of having yet another repair solution in the > discussion? > > I personally think looking at multiple options is important. So yes, it > would be interesting to see your solution. > > Dinesh > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Jon Haddad http://www.rustyrazorblade.com twitter: rustyrazorblade
Re: Yet another repair solution
> On Aug 28, 2018, at 6:33 AM, Marcus Olsson wrote: > > Hi, > > With the risk of stirring the repair/side-car topic even further I'd just > like to mention that we have recently gotten approval to contribute our > repair management side-car solution. > It's based on the proposal in > https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone > application sitting next to each instance. > With the recent discussions in mind I'd just like to hear the thoughts from > the community on this before we put in the effort of bringing our solution > into open source. > > Would there be an interest of having yet another repair solution in the > discussion? I personally think looking at multiple options is important. So yes, it would be interesting to see your solution. Dinesh - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Yet another repair solution
Hi, With the risk of stirring the repair/side-car topic even further I'd just like to mention that we have recently gotten approval to contribute our repair management side-car solution. It's based on the proposal in https://issues.apache.org/jira/browse/CASSANDRA-10070 as a standalone application sitting next to each instance. With the recent discussions in mind I'd just like to hear the thoughts from the community on this before we put in the effort of bringing our solution into open source. Would there be an interest of having yet another repair solution in the discussion? Best Regards Marcus Olsson
