>From my point of view CEP-1 is overly broad in terms it wants to achieve. My understanding is that CEPs have to have a well-defined scope.
I agree with Joey that we should close CEP-1 with the features I have proposed earlier in this thread. And have any future Sidecar work captured in subsequent CEPs. Best, - Francisco On 2024/10/14 21:58:55 Joseph Lynch wrote: > Hi everyone! > > I am curious what Dinesh's perspective is but I think the specific > enumerated scope in CEP-1 isn't super critical to be honest. That CEP > successfully (imo) built consensus that the community wants a separate > management process, and that sidecar both exists today and has useful > functionality (which is great!). I agree with Jon and others in these > threads that the functionality isn't super accessible until some of the > tickets Francisco mentioned are worked on and a release is made. I also > agree with Francisco that getting to a release, even if it is an alpha, > should be the goal - let's get to a release. > > I am personally fine closing CEP-1 and saying "This CEP has been superseded > by subsequent CEPs - future work in the sidecar requiring community > consensus will have separate CEPs as needed". That doesn't mean that the > scope in CEP-1 isn't useful, and I hope some of the gaps are added in the > future, but I think we can release without them and the focus should be on > the gaps for release not the gaps in functionality with CEP-1. > > Also I should mention I am extremely excited to see all the great features > that have landed and that there is a place outside the main DB for those > kinds of innovations to be tried out that doesn't conflict with the main > process. In my mind, having that place was the purpose of CEP-1, which is > done - the specific enumerated features are less important in my opinion. > > -Joey > > On Mon, Oct 14, 2024 at 4:09 PM Patrick McFadin <pmcfa...@gmail.com> wrote: > > > What are we going to do with CEP-1? Cut and rescope in a new CEP or > > rewrite CEP-1? > > > > On Mon, Oct 14, 2024 at 11:18 AM Francisco Guerrero <fran...@apache.org> > > wrote: > > > >> Hi folks, > >> > >> It was great meeting some of you in person at Community over Code where > >> we had a chance to discuss the Cassandra Sidecar project. A lot of > >> you expressed interest in having a release of Sidecar to start using the > >> existing features in the project: > >> > >> - C* adapters for versions 4.0, 4.1, 5.0 and trunk > >> - Cassandra Analytics support > >> - Restoring SSTables from S3-compatible object storage > >> - Mutual TLS authentication > >> - Role base access control > >> - Cluster health checks > >> - Observability > >> - Sidecar Client > >> > >> Some of the call-outs in this thread of things we need for a release: > >> > >> - Documentation > >> - Bug fixes [1][2][3] > >> > >> These are other things mentioned in the thread, where we would like help > >> from the community: > >> > >> - vNode support [4] > >> - Backup support [5] > >> - Bulk APIs [6] > >> > >> Once we have documentation and the bug fixes mentioned above, I will > >> start a thread vote for a release of Cassandra Sidecar 0.1 alpha. > >> > >> We want the community to benefit from the features already present in > >> Sidecar. The more community engagement we have, the more feature-rich > >> will become. > >> > >> Additionally, we can leverage easy-cass-stress to have Cassandra Sidecar > >> included in your Cassandra clusters, so we can easily start trying it out. > >> > >> Please let me know your thoughts on this. > >> > >> Best, > >> - Francisco > >> > >> [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 > >> [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 > >> [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 > >> [4] https://issues.apache.org/jira/browse/CASSANDRASC-149 > >> [5] https://issues.apache.org/jira/browse/CASSANDRASC-148 > >> [6] https://issues.apache.org/jira/browse/CASSANDRASC-3 > >> > >> On 2024/10/03 12:05:14 Josh McKenzie wrote: > >> > I'm tentatively in favor of the idea of us cutting releases for all our > >> ecosystem dependencies as a blocker to cutting a major with Cassandra > >> proper. I say tentatively since we've had trouble getting majors across the > >> line for awhile so adding more dependencies to that feels risky, however I > >> think the improvement to user experience justifies it. > >> > > >> > Also - I suspect the effort required to get subprojects across the line > >> will be quite a bit less than the mainline DB. > >> > > >> > If this is something there's agreement / consensus on, I'd be happy to > >> take on the role of driving that coordination in the future. > >> > > >> > On Wed, Oct 2, 2024, at 3:16 PM, Jon Haddad wrote: > >> > > > Mostly Analytics, which should not be a blocker for Sidecar. > >> > > > >> > > Yes, agreed. I'm just trying to understand the context of the vnode > >> statement and how we're framing 1.0 sidecar. > >> > > > >> > > > We definitely need to support 5.0 for the Analytics release, but > >> that's orthogonal to Sidecar. > >> > > > >> > > It is, unless we're endorsing the analytics library as a primary > >> reason to use sidecar. Then it becomes a dependency people rely on, and I > >> don't want it blocking people from upgrading. > >> > > > >> > > > Yeah, definitely. I agree that we need to support the latest > >> released version of Cassandra in ecosystem projects. However, without > >> official releases there won't be adoption and without adoption there won't > >> be feedback from the community on what features / improvements are needed. > >> > > > >> > > I don't expect that our first release will be feature complete, but > >> it should be at least compelling. I'm still not aware of what > >> functionality exists that would meet that requirement. > >> > > > >> > > Think about this from the perspective of reading a blog post. We're > >> excited to announce sidecar 1.0! Here's what you can do: > >> > > > >> > > 1. Backup / restore? > >> > > 2. ? > >> > > 3. ? > >> > > > >> > > All I'm asking for are 3 reasons why people should care. If one of > >> them is backups, do we provide more flexible backup targets than S3, and if > >> we provide incremental / continuous backup options? Is there a scheduler > >> or do people need to roll their own? Is it coordinated, or is the intent > >> that people handle it on their own? > >> > > > >> > > I work with a lot of end users - this is the type of thing that > >> affects people and can either help or harm the project image. > >> > > > >> > > Jon > >> > > > >> > > > >> > > On Wed, Oct 2, 2024 at 2:40 PM Francisco Guerrero <fran...@apache.org> > >> wrote: > >> > >> > By vnode support, do you mean the analytics library? Or do other > >> features > >> > >> > in sidecar not work with vnodes? > >> > >> > >> > >> Mostly Analytics, which should not be a blocker for Sidecar. > >> However, I do > >> > >> feel there should be more testing around vnodes in Sidecar. > >> > >> > >> > >> > If we're talking about analytics, that gets a little complicated. > >> Are we > >> > >> > also then talking about 1.0'ing analytics? If so, I think we need > >> support > >> > >> > for 5.0 and BTI there. > >> > >> > >> > >> We need to release Analytics at some point as well, we should have a > >> similar > >> > >> discuss thread regarding Analytics. We definitely need to support > >> 5.0 for the > >> > >> Analytics release, but that's orthogonal to Sidecar. > >> > >> > >> > >> > Increasing the size of the ecosystem is challenging... > >> > >> > >> > >> Yeah, definitely. I agree that we need to support the latest > >> released version > >> > >> of Cassandra in ecosystem projects. However, without official > >> releases > >> > >> there won't be adoption and without adoption there won't be feedback > >> from > >> > >> the community on what features / improvements are needed. > >> > >> > >> > >> On 2024/10/02 18:05:42 Jon Haddad wrote: > >> > >> > By vnode support, do you mean the analytics library? Or do other > >> features > >> > >> > in sidecar not work with vnodes? > >> > >> > > >> > >> > If we're talking about analytics, that gets a little complicated. > >> Are we > >> > >> > also then talking about 1.0'ing analytics? If so, I think we need > >> support > >> > >> > for 5.0 and BTI there. > >> > >> > > >> > >> > In my opinion, if something core to the project has official > >> releases, such > >> > >> > as drivers or operational tooling that people depend on, it should > >> also > >> > >> > support the latest version of Cassandra, on release. It doesn't > >> look good > >> > >> > if we release C* without usable drivers or tooling to operate it. > >> It is > >> > >> > massively deflating to announce we just released 5.0 (exciting!) > >> but you > >> > >> > can't use it for 6 months because you're using analytics lib and > >> the people > >> > >> > that contribute to it has better things to do with their time. I > >> think > >> > >> > part of the obligation of maintaining these projects needs to be > >> keeping up > >> > >> > with latest releases. If that can't be done, we should continue > >> to treat > >> > >> > it as a use-as-your-own-risk thing without official releases. > >> > >> > > >> > >> > Increasing the size of the ecosystem is challenging... > >> > >> > > >> > >> > Jon > >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero < > >> fran...@apache.org> > >> > >> > wrote: > >> > >> > > >> > >> > > Hi folks, > >> > >> > > > >> > >> > > Thanks for all the input. I'm trying to gather all the comments, > >> and from > >> > >> > > what I > >> > >> > > can gather it seems that most of the opinions are converging > >> towards > >> > >> > > scoping > >> > >> > > a Sidecar release. These are the items that were called out that > >> we will > >> > >> > > need > >> > >> > > for a release: > >> > >> > > - Documentation > >> > >> > > - Authorization / Authentication > >> > >> > > - vnode support > >> > >> > > > >> > >> > > There are some smaller bug fixes that need to be included that > >> we can label > >> > >> > > as part of the 1.0 release.[1][2][3] > >> > >> > > > >> > >> > > If there are any other tasks we need to completed, I encourage > >> the > >> > >> > > community > >> > >> > > to create JIRAs that can be in the release milestone for the > >> Sidecar. > >> > >> > > > >> > >> > > I agree with Stefan that OpenAPI is desirable. OpenAPI is > >> something I've > >> > >> > > been > >> > >> > > looking into as well. I'm also glad to see Abhijeet can help > >> with the > >> > >> > > documentation effort, I can also help on that front. > >> > >> > > > >> > >> > > Hopefully, I've captured your comments as truly as possible. > >> > >> > > > >> > >> > > Thanks again for all the feedback. > >> > >> > > > >> > >> > > Best, > >> > >> > > - Francisco > >> > >> > > > >> > >> > > [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 > >> > >> > > [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 > >> > >> > > [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 > >> > >> > > > >> > >> > > On 2024/10/02 15:29:20 Jon Haddad wrote: > >> > >> > > > When I developed some of the original sidecar code, it was > >> based on REST > >> > >> > > > Easy, which would have allowed us to generate the spec > >> automatically. I > >> > >> > > > did this in a similar project. > >> > >> > > > > >> > >> > > > That was removed here: > >> > >> > > > https://issues.apache.org/jira/browse/CASSANDRASC-57 > >> > >> > > > > >> > >> > > > But unfortunately it looks like you can't generate the spec > >> from the > >> > >> > > code. > >> > >> > > > > >> > >> > > > Disappointing, that functionality was really useful. I wish > >> someone had > >> > >> > > > asked me before gutting it. > >> > >> > > > > >> > >> > > > Jon > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič < > >> > >> > > smikloso...@apache.org> > >> > >> > > > wrote: > >> > >> > > > > >> > >> > > > > Something like this: > >> > >> > > > > > >> > >> > > > > https://instaclustr.github.io/instaclustr-icarus-go-client/ > >> > >> > > > > > >> > >> > > > > On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič < > >> > >> > > smikloso...@apache.org> > >> > >> > > > > wrote: > >> > >> > > > > > >> > >> > > > >> While documenting endpoints please use something like > >> OpenAPI > >> > >> > > > >> specification. The sidecar should expose this document > >> itself so when > >> > >> > > I go > >> > >> > > > >> to this and that URL, I see all endpoints, I put the > >> payloads / > >> > >> > > parameters > >> > >> > > > >> for them and I can just directly call that, no messing with > >> curl / > >> > >> > > wget or > >> > >> > > > >> programmatically or whatever like that. The barrier to > >> exercise the > >> > >> > > basic > >> > >> > > > >> functionality has to virtually not be there. > >> > >> > > > >> > >> > >> > > > >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey < > >> > >> > > dubey.abhijee...@gmail.com> > >> > >> > > > >> wrote: > >> > >> > > > >> > >> > >> > > > >>> Hi folks, > >> > >> > > > >>> > >> > >> > > > >>> I have been using Sidecar recently and have found some of > >> its > >> > >> > > > >>> functionalities to be quite useful. Hari and I are also > >> working on > >> > >> > > CEP-40 > >> > >> > > > >>> which aims to introduce live migration features in Sidecar > >> in the > >> > >> > > > >>> near future. > >> > >> > > > >>> > >> > >> > > > >>> However, as others have mentioned, I agree that it > >> currently lacks > >> > >> > > > >>> proper documentation. > >> > >> > > > >>> > >> > >> > > > >>> Since this is an official Apache project, I believe that > >> creating > >> > >> > > > >>> comprehensive documentation would be beneficial. This > >> documentation > >> > >> > > should > >> > >> > > > >>> include an overview, architecture, a list and description > >> of various > >> > >> > > > >>> endpoints, and some examples or tutorials on how to use > >> Sidecar's > >> > >> > > features. > >> > >> > > > >>> > >> > >> > > > >>> This documentation would help people get started with > >> Sidecar and > >> > >> > > lower > >> > >> > > > >>> the entry barrier for many. We can update the documentation > >> > >> > > incrementally > >> > >> > > > >>> as needed, along with future enhancements and new > >> features. However, > >> > >> > > > >>> creating some form of formal documentation would be very > >> helpful. > >> > >> > > > >>> > >> > >> > > > >>> To this end I'm willing and highly interested in writing > >> some form of > >> > >> > > > >>> formal documentation for the Sidecar project. Please let > >> me know your > >> > >> > > > >>> thoughts/opinions on this proposal. > >> > >> > > > >>> > >> > >> > > > >>> > >> > >> > > > >>> > >> > >> > > > >>> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič < > >> > >> > > smikloso...@apache.org> > >> > >> > > > >>> wrote: > >> > >> > > > >>> > >> > >> > > > >>>> Totally agree with Jon here basically on all fronts. > >> Apache > >> > >> > > Cassandra > >> > >> > > > >>>> Sidecar was always a hard nut to crack for me, that is > >> probably why > >> > >> > > I have > >> > >> > > > >>>> not been involved with that a lot even that is a great > >> tool to have > >> > >> > > and be > >> > >> > > > >>>> invested in as I was writing my own sidecar and I found a > >> lot of > >> > >> > > > >>>> similarities and problems Apache's sidecar tries to fix. > >> There was > >> > >> > > some > >> > >> > > > >>>> invisible barrier I have never managed to jump over. I > >> was looking > >> > >> > > around > >> > >> > > > >>>> and I am very sorry if I just have not found it yet but > >> there is > >> > >> > > not a list > >> > >> > > > >>>> of endpoints a sidecar has, is there? In readme and dev > >> docs there > >> > >> > > is just > >> > >> > > > >>>> nothing. Taking it at a face value I just don't know what > >> Sidecar is > >> > >> > > > >>>> capable of and how to use it. I see in the commit history > >> there is > >> > >> > > a bunch > >> > >> > > > >>>> of commits mentioning S3 but it is a total blackbox for > >> me as a > >> > >> > > potential > >> > >> > > > >>>> user. > >> > >> > > > >>>> > >> > >> > > > >>>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad < > >> j...@rustyrazorblade.com> > >> > >> > > > >>>> wrote: > >> > >> > > > >>>> > >> > >> > > > >>>>> I don't think we should release sidecar 1.0 without any > >> docs. > >> > >> > > > >>>>> > >> > >> > > > >>>>> I took a look through the closed JIRAs to see what's > >> there. Here's > >> > >> > > > >>>>> what I found, please correct me if there's more: > >> > >> > > > >>>>> > >> > >> > > > >>>>> - Lots of stuff related to analytics. > >> > >> > > > >>>>> > >> > >> > > > >>>>> I would be pretty excited for this, but the analytics > >> library only > >> > >> > > > >>>>> works with single token clusters. Most folks don't run > >> Cassandra > >> > >> > > this > >> > >> > > > >>>>> way. I realize there's some element of everyone needs > >> to scratch > >> > >> > > their own > >> > >> > > > >>>>> itch, but I don't think we can really call this a useful > >> feature > >> > >> > > if the > >> > >> > > > >>>>> overwhelming majority of folks can't use it. I've > >> worked with a > >> > >> > > couple > >> > >> > > > >>>>> hundred teams over the years and can only think of 1 org > >> outside > >> > >> > > of Apple > >> > >> > > > >>>>> and Netflix that used 1 token, and It was a cluster that > >> predated > >> > >> > > v-nodes. > >> > >> > > > >>>>> > >> > >> > > > >>>>> The analytics repo says it's compatible with Cassandra > >> 4, but not > >> > >> > > 5. > >> > >> > > > >>>>> > >> > >> > > > >>>>> - Backup & Restore from S3 > >> > >> > > > >>>>> > >> > >> > > > >>>>> Is this compatible with other cloud providers or object > >> stores? It > >> > >> > > > >>>>> specifically lists S3 in JIRA. I haven't looked at the > >> source > >> > >> > > yet. Am I > >> > >> > > > >>>>> correct in reading it supports backing up snapshots, no > >> continuous > >> > >> > > > >>>>> backups? Seems like we should have at least feature > >> parity with > >> > >> > > Medusa if > >> > >> > > > >>>>> we're going to release something here. > >> > >> > > > >>>>> > >> > >> > > > >>>>> All the other closed JIRAs look related to these two > >> items. So the > >> > >> > > > >>>>> question is, are we releasing 1.0 as an limited S3 > >> backup and > >> > >> > > restore > >> > >> > > > >>>>> tool? One that prevents you from upgrading to Cassandra > >> 5 if you > >> > >> > > happen to > >> > >> > > > >>>>> use single token clusters? > >> > >> > > > >>>>> > >> > >> > > > >>>>> Who is the target audience? > >> > >> > > > >>>>> > >> > >> > > > >>>>> Jon > >> > >> > > > >>>>> > >> > >> > > > >>>>> > >> > >> > > > >>>>> > >> > >> > > > >>>>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi < > >> djo...@apache.org> > >> > >> > > wrote: > >> > >> > > > >>>>> > >> > >> > > > >>>>>> Currently the Sidecar has a lot of functionality that is > >> > >> > > immediately > >> > >> > > > >>>>>> usable by the community. Apart from minor fixes, the > >> AuthN/Z > >> > >> > > story would be > >> > >> > > > >>>>>> wrapped up soon. Post this, I would propose moving > >> forward with > >> > >> > > cutting a > >> > >> > > > >>>>>> release with the existing feature set so we can get > >> this in the > >> > >> > > hands of > >> > >> > > > >>>>>> our community. > >> > >> > > > >>>>>> > >> > >> > > > >>>>>> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell < > >> cclive1...@gmail.com> > >> > >> > > > >>>>>> wrote: > >> > >> > > > >>>>>> > >> > >> > > > >>>>>>> Have the same question ： what ‘s the plan ？ > >> > >> > > > >>>>>>> > >> > >> > > > >>>>>>> Jeff Jirsa <jji...@gmail.com>于2024年10月2日 周三上午10:43写道： > >> > >> > > > >>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie < > >> jmcken...@apache.org > >> > >> > > > > >> > >> > > > >>>>>>>> wrote: > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> However it is used by a number of other features as a > >> dependency > >> > >> > > > >>>>>>>> such as analytics, backup/restore, repair, metrics, > >> and CDC > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> It seems like a natural pressure relief valve for > >> moving > >> > >> > > operations > >> > >> > > > >>>>>>>> out of a core C* node that are well served out of > >> process. > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> Yea, but the point of the foundation is to RELEASE > >> software for > >> > >> > > the > >> > >> > > > >>>>>>>> public good, and the link asserting consensus was > >> dec2018, so > >> > >> > > its’ 5.5 > >> > >> > > > >>>>>>>> years and no releases. > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> What’s the plan here? > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>>>>>>> > >> > >> > > > >>> > >> > >> > > > >>> -- > >> > >> > > > >>> *Abhijeet* > >> > >> > > > >>> > >> > >> > > > >> > >> > >> > > > > >> > >> > > > >> > >> > > >> > > >> > > >
