Hi everyone! I am curious what Dinesh's perspective is but I think the specific enumerated scope in CEP-1 isn't super critical to be honest. That CEP successfully (imo) built consensus that the community wants a separate management process, and that sidecar both exists today and has useful functionality (which is great!). I agree with Jon and others in these threads that the functionality isn't super accessible until some of the tickets Francisco mentioned are worked on and a release is made. I also agree with Francisco that getting to a release, even if it is an alpha, should be the goal - let's get to a release.
I am personally fine closing CEP-1 and saying "This CEP has been superseded by subsequent CEPs - future work in the sidecar requiring community consensus will have separate CEPs as needed". That doesn't mean that the scope in CEP-1 isn't useful, and I hope some of the gaps are added in the future, but I think we can release without them and the focus should be on the gaps for release not the gaps in functionality with CEP-1. Also I should mention I am extremely excited to see all the great features that have landed and that there is a place outside the main DB for those kinds of innovations to be tried out that doesn't conflict with the main process. In my mind, having that place was the purpose of CEP-1, which is done - the specific enumerated features are less important in my opinion. -Joey On Mon, Oct 14, 2024 at 4:09 PM Patrick McFadin <pmcfa...@gmail.com> wrote: > What are we going to do with CEP-1? Cut and rescope in a new CEP or > rewrite CEP-1? > > On Mon, Oct 14, 2024 at 11:18 AM Francisco Guerrero <fran...@apache.org> > wrote: > >> Hi folks, >> >> It was great meeting some of you in person at Community over Code where >> we had a chance to discuss the Cassandra Sidecar project. A lot of >> you expressed interest in having a release of Sidecar to start using the >> existing features in the project: >> >> - C* adapters for versions 4.0, 4.1, 5.0 and trunk >> - Cassandra Analytics support >> - Restoring SSTables from S3-compatible object storage >> - Mutual TLS authentication >> - Role base access control >> - Cluster health checks >> - Observability >> - Sidecar Client >> >> Some of the call-outs in this thread of things we need for a release: >> >> - Documentation >> - Bug fixes [1][2][3] >> >> These are other things mentioned in the thread, where we would like help >> from the community: >> >> - vNode support [4] >> - Backup support [5] >> - Bulk APIs [6] >> >> Once we have documentation and the bug fixes mentioned above, I will >> start a thread vote for a release of Cassandra Sidecar 0.1 alpha. >> >> We want the community to benefit from the features already present in >> Sidecar. The more community engagement we have, the more feature-rich >> will become. >> >> Additionally, we can leverage easy-cass-stress to have Cassandra Sidecar >> included in your Cassandra clusters, so we can easily start trying it out. >> >> Please let me know your thoughts on this. >> >> Best, >> - Francisco >> >> [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 >> [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 >> [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 >> [4] https://issues.apache.org/jira/browse/CASSANDRASC-149 >> [5] https://issues.apache.org/jira/browse/CASSANDRASC-148 >> [6] https://issues.apache.org/jira/browse/CASSANDRASC-3 >> >> On 2024/10/03 12:05:14 Josh McKenzie wrote: >> > I'm tentatively in favor of the idea of us cutting releases for all our >> ecosystem dependencies as a blocker to cutting a major with Cassandra >> proper. I say tentatively since we've had trouble getting majors across the >> line for awhile so adding more dependencies to that feels risky, however I >> think the improvement to user experience justifies it. >> > >> > Also - I suspect the effort required to get subprojects across the line >> will be quite a bit less than the mainline DB. >> > >> > If this is something there's agreement / consensus on, I'd be happy to >> take on the role of driving that coordination in the future. >> > >> > On Wed, Oct 2, 2024, at 3:16 PM, Jon Haddad wrote: >> > > > Mostly Analytics, which should not be a blocker for Sidecar. >> > > >> > > Yes, agreed. I'm just trying to understand the context of the vnode >> statement and how we're framing 1.0 sidecar. >> > > >> > > > We definitely need to support 5.0 for the Analytics release, but >> that's orthogonal to Sidecar. >> > > >> > > It is, unless we're endorsing the analytics library as a primary >> reason to use sidecar. Then it becomes a dependency people rely on, and I >> don't want it blocking people from upgrading. >> > > >> > > > Yeah, definitely. I agree that we need to support the latest >> released version of Cassandra in ecosystem projects. However, without >> official releases there won't be adoption and without adoption there won't >> be feedback from the community on what features / improvements are needed. >> > > >> > > I don't expect that our first release will be feature complete, but >> it should be at least compelling. I'm still not aware of what >> functionality exists that would meet that requirement. >> > > >> > > Think about this from the perspective of reading a blog post. We're >> excited to announce sidecar 1.0! Here's what you can do: >> > > >> > > 1. Backup / restore? >> > > 2. ? >> > > 3. ? >> > > >> > > All I'm asking for are 3 reasons why people should care. If one of >> them is backups, do we provide more flexible backup targets than S3, and if >> we provide incremental / continuous backup options? Is there a scheduler >> or do people need to roll their own? Is it coordinated, or is the intent >> that people handle it on their own? >> > > >> > > I work with a lot of end users - this is the type of thing that >> affects people and can either help or harm the project image. >> > > >> > > Jon >> > > >> > > >> > > On Wed, Oct 2, 2024 at 2:40 PM Francisco Guerrero <fran...@apache.org> >> wrote: >> > >> > By vnode support, do you mean the analytics library? Or do other >> features >> > >> > in sidecar not work with vnodes? >> > >> >> > >> Mostly Analytics, which should not be a blocker for Sidecar. >> However, I do >> > >> feel there should be more testing around vnodes in Sidecar. >> > >> >> > >> > If we're talking about analytics, that gets a little complicated. >> Are we >> > >> > also then talking about 1.0'ing analytics? If so, I think we need >> support >> > >> > for 5.0 and BTI there. >> > >> >> > >> We need to release Analytics at some point as well, we should have a >> similar >> > >> discuss thread regarding Analytics. We definitely need to support >> 5.0 for the >> > >> Analytics release, but that's orthogonal to Sidecar. >> > >> >> > >> > Increasing the size of the ecosystem is challenging... >> > >> >> > >> Yeah, definitely. I agree that we need to support the latest >> released version >> > >> of Cassandra in ecosystem projects. However, without official >> releases >> > >> there won't be adoption and without adoption there won't be feedback >> from >> > >> the community on what features / improvements are needed. >> > >> >> > >> On 2024/10/02 18:05:42 Jon Haddad wrote: >> > >> > By vnode support, do you mean the analytics library? Or do other >> features >> > >> > in sidecar not work with vnodes? >> > >> > >> > >> > If we're talking about analytics, that gets a little complicated. >> Are we >> > >> > also then talking about 1.0'ing analytics? If so, I think we need >> support >> > >> > for 5.0 and BTI there. >> > >> > >> > >> > In my opinion, if something core to the project has official >> releases, such >> > >> > as drivers or operational tooling that people depend on, it should >> also >> > >> > support the latest version of Cassandra, on release. It doesn't >> look good >> > >> > if we release C* without usable drivers or tooling to operate it. >> It is >> > >> > massively deflating to announce we just released 5.0 (exciting!) >> but you >> > >> > can't use it for 6 months because you're using analytics lib and >> the people >> > >> > that contribute to it has better things to do with their time. I >> think >> > >> > part of the obligation of maintaining these projects needs to be >> keeping up >> > >> > with latest releases. If that can't be done, we should continue >> to treat >> > >> > it as a use-as-your-own-risk thing without official releases. >> > >> > >> > >> > Increasing the size of the ecosystem is challenging... >> > >> > >> > >> > Jon >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero < >> fran...@apache.org> >> > >> > wrote: >> > >> > >> > >> > > Hi folks, >> > >> > > >> > >> > > Thanks for all the input. I'm trying to gather all the comments, >> and from >> > >> > > what I >> > >> > > can gather it seems that most of the opinions are converging >> towards >> > >> > > scoping >> > >> > > a Sidecar release. These are the items that were called out that >> we will >> > >> > > need >> > >> > > for a release: >> > >> > > - Documentation >> > >> > > - Authorization / Authentication >> > >> > > - vnode support >> > >> > > >> > >> > > There are some smaller bug fixes that need to be included that >> we can label >> > >> > > as part of the 1.0 release.[1][2][3] >> > >> > > >> > >> > > If there are any other tasks we need to completed, I encourage >> the >> > >> > > community >> > >> > > to create JIRAs that can be in the release milestone for the >> Sidecar. >> > >> > > >> > >> > > I agree with Stefan that OpenAPI is desirable. OpenAPI is >> something I've >> > >> > > been >> > >> > > looking into as well. I'm also glad to see Abhijeet can help >> with the >> > >> > > documentation effort, I can also help on that front. >> > >> > > >> > >> > > Hopefully, I've captured your comments as truly as possible. >> > >> > > >> > >> > > Thanks again for all the feedback. >> > >> > > >> > >> > > Best, >> > >> > > - Francisco >> > >> > > >> > >> > > [1] https://issues.apache.org/jira/browse/CASSANDRASC-120 >> > >> > > [2] https://issues.apache.org/jira/browse/CASSANDRASC-121 >> > >> > > [3] https://issues.apache.org/jira/browse/CASSANDRASC-122 >> > >> > > >> > >> > > On 2024/10/02 15:29:20 Jon Haddad wrote: >> > >> > > > When I developed some of the original sidecar code, it was >> based on REST >> > >> > > > Easy, which would have allowed us to generate the spec >> automatically. I >> > >> > > > did this in a similar project. >> > >> > > > >> > >> > > > That was removed here: >> > >> > > > https://issues.apache.org/jira/browse/CASSANDRASC-57 >> > >> > > > >> > >> > > > But unfortunately it looks like you can't generate the spec >> from the >> > >> > > code. >> > >> > > > >> > >> > > > Disappointing, that functionality was really useful. I wish >> someone had >> > >> > > > asked me before gutting it. >> > >> > > > >> > >> > > > Jon >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič < >> > >> > > smikloso...@apache.org> >> > >> > > > wrote: >> > >> > > > >> > >> > > > > Something like this: >> > >> > > > > >> > >> > > > > https://instaclustr.github.io/instaclustr-icarus-go-client/ >> > >> > > > > >> > >> > > > > On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič < >> > >> > > smikloso...@apache.org> >> > >> > > > > wrote: >> > >> > > > > >> > >> > > > >> While documenting endpoints please use something like >> OpenAPI >> > >> > > > >> specification. The sidecar should expose this document >> itself so when >> > >> > > I go >> > >> > > > >> to this and that URL, I see all endpoints, I put the >> payloads / >> > >> > > parameters >> > >> > > > >> for them and I can just directly call that, no messing with >> curl / >> > >> > > wget or >> > >> > > > >> programmatically or whatever like that. The barrier to >> exercise the >> > >> > > basic >> > >> > > > >> functionality has to virtually not be there. >> > >> > > > >> >> > >> > > > >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey < >> > >> > > dubey.abhijee...@gmail.com> >> > >> > > > >> wrote: >> > >> > > > >> >> > >> > > > >>> Hi folks, >> > >> > > > >>> >> > >> > > > >>> I have been using Sidecar recently and have found some of >> its >> > >> > > > >>> functionalities to be quite useful. Hari and I are also >> working on >> > >> > > CEP-40 >> > >> > > > >>> which aims to introduce live migration features in Sidecar >> in the >> > >> > > > >>> near future. >> > >> > > > >>> >> > >> > > > >>> However, as others have mentioned, I agree that it >> currently lacks >> > >> > > > >>> proper documentation. >> > >> > > > >>> >> > >> > > > >>> Since this is an official Apache project, I believe that >> creating >> > >> > > > >>> comprehensive documentation would be beneficial. This >> documentation >> > >> > > should >> > >> > > > >>> include an overview, architecture, a list and description >> of various >> > >> > > > >>> endpoints, and some examples or tutorials on how to use >> Sidecar's >> > >> > > features. >> > >> > > > >>> >> > >> > > > >>> This documentation would help people get started with >> Sidecar and >> > >> > > lower >> > >> > > > >>> the entry barrier for many. We can update the documentation >> > >> > > incrementally >> > >> > > > >>> as needed, along with future enhancements and new >> features. However, >> > >> > > > >>> creating some form of formal documentation would be very >> helpful. >> > >> > > > >>> >> > >> > > > >>> To this end I'm willing and highly interested in writing >> some form of >> > >> > > > >>> formal documentation for the Sidecar project. Please let >> me know your >> > >> > > > >>> thoughts/opinions on this proposal. >> > >> > > > >>> >> > >> > > > >>> >> > >> > > > >>> >> > >> > > > >>> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič < >> > >> > > smikloso...@apache.org> >> > >> > > > >>> wrote: >> > >> > > > >>> >> > >> > > > >>>> Totally agree with Jon here basically on all fronts. >> Apache >> > >> > > Cassandra >> > >> > > > >>>> Sidecar was always a hard nut to crack for me, that is >> probably why >> > >> > > I have >> > >> > > > >>>> not been involved with that a lot even that is a great >> tool to have >> > >> > > and be >> > >> > > > >>>> invested in as I was writing my own sidecar and I found a >> lot of >> > >> > > > >>>> similarities and problems Apache's sidecar tries to fix. >> There was >> > >> > > some >> > >> > > > >>>> invisible barrier I have never managed to jump over. I >> was looking >> > >> > > around >> > >> > > > >>>> and I am very sorry if I just have not found it yet but >> there is >> > >> > > not a list >> > >> > > > >>>> of endpoints a sidecar has, is there? In readme and dev >> docs there >> > >> > > is just >> > >> > > > >>>> nothing. Taking it at a face value I just don't know what >> Sidecar is >> > >> > > > >>>> capable of and how to use it. I see in the commit history >> there is >> > >> > > a bunch >> > >> > > > >>>> of commits mentioning S3 but it is a total blackbox for >> me as a >> > >> > > potential >> > >> > > > >>>> user. >> > >> > > > >>>> >> > >> > > > >>>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad < >> j...@rustyrazorblade.com> >> > >> > > > >>>> wrote: >> > >> > > > >>>> >> > >> > > > >>>>> I don't think we should release sidecar 1.0 without any >> docs. >> > >> > > > >>>>> >> > >> > > > >>>>> I took a look through the closed JIRAs to see what's >> there. Here's >> > >> > > > >>>>> what I found, please correct me if there's more: >> > >> > > > >>>>> >> > >> > > > >>>>> - Lots of stuff related to analytics. >> > >> > > > >>>>> >> > >> > > > >>>>> I would be pretty excited for this, but the analytics >> library only >> > >> > > > >>>>> works with single token clusters. Most folks don't run >> Cassandra >> > >> > > this >> > >> > > > >>>>> way. I realize there's some element of everyone needs >> to scratch >> > >> > > their own >> > >> > > > >>>>> itch, but I don't think we can really call this a useful >> feature >> > >> > > if the >> > >> > > > >>>>> overwhelming majority of folks can't use it. I've >> worked with a >> > >> > > couple >> > >> > > > >>>>> hundred teams over the years and can only think of 1 org >> outside >> > >> > > of Apple >> > >> > > > >>>>> and Netflix that used 1 token, and It was a cluster that >> predated >> > >> > > v-nodes. >> > >> > > > >>>>> >> > >> > > > >>>>> The analytics repo says it's compatible with Cassandra >> 4, but not >> > >> > > 5. >> > >> > > > >>>>> >> > >> > > > >>>>> - Backup & Restore from S3 >> > >> > > > >>>>> >> > >> > > > >>>>> Is this compatible with other cloud providers or object >> stores? It >> > >> > > > >>>>> specifically lists S3 in JIRA. I haven't looked at the >> source >> > >> > > yet. Am I >> > >> > > > >>>>> correct in reading it supports backing up snapshots, no >> continuous >> > >> > > > >>>>> backups? Seems like we should have at least feature >> parity with >> > >> > > Medusa if >> > >> > > > >>>>> we're going to release something here. >> > >> > > > >>>>> >> > >> > > > >>>>> All the other closed JIRAs look related to these two >> items. So the >> > >> > > > >>>>> question is, are we releasing 1.0 as an limited S3 >> backup and >> > >> > > restore >> > >> > > > >>>>> tool? One that prevents you from upgrading to Cassandra >> 5 if you >> > >> > > happen to >> > >> > > > >>>>> use single token clusters? >> > >> > > > >>>>> >> > >> > > > >>>>> Who is the target audience? >> > >> > > > >>>>> >> > >> > > > >>>>> Jon >> > >> > > > >>>>> >> > >> > > > >>>>> >> > >> > > > >>>>> >> > >> > > > >>>>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi < >> djo...@apache.org> >> > >> > > wrote: >> > >> > > > >>>>> >> > >> > > > >>>>>> Currently the Sidecar has a lot of functionality that is >> > >> > > immediately >> > >> > > > >>>>>> usable by the community. Apart from minor fixes, the >> AuthN/Z >> > >> > > story would be >> > >> > > > >>>>>> wrapped up soon. Post this, I would propose moving >> forward with >> > >> > > cutting a >> > >> > > > >>>>>> release with the existing feature set so we can get >> this in the >> > >> > > hands of >> > >> > > > >>>>>> our community. >> > >> > > > >>>>>> >> > >> > > > >>>>>> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell < >> cclive1...@gmail.com> >> > >> > > > >>>>>> wrote: >> > >> > > > >>>>>> >> > >> > > > >>>>>>> Have the same question ： what ‘s the plan ？ >> > >> > > > >>>>>>> >> > >> > > > >>>>>>> Jeff Jirsa <jji...@gmail.com>于2024年10月2日 周三上午10:43写道： >> > >> > > > >>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie < >> jmcken...@apache.org >> > >> > > > >> > >> > > > >>>>>>>> wrote: >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> However it is used by a number of other features as a >> dependency >> > >> > > > >>>>>>>> such as analytics, backup/restore, repair, metrics, >> and CDC >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> It seems like a natural pressure relief valve for >> moving >> > >> > > operations >> > >> > > > >>>>>>>> out of a core C* node that are well served out of >> process. >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> Yea, but the point of the foundation is to RELEASE >> software for >> > >> > > the >> > >> > > > >>>>>>>> public good, and the link asserting consensus was >> dec2018, so >> > >> > > its’ 5.5 >> > >> > > > >>>>>>>> years and no releases. >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> What’s the plan here? >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>>>>>>> >> > >> > > > >>> >> > >> > > > >>> -- >> > >> > > > >>> *Abhijeet* >> > >> > > > >>> >> > >> > > > >> >> > >> > > > >> > >> > > >> > >> > >> > >> >
