If it were that simple, they would probably have just gone with that solution. That said, the API is functionally vendor agnostic at this point at the Java API level. So I see no need to add abstraction above that. I've got probably 2/3 of nifi-cassandra-bundle converted. Hitting a few pain points where I'm having to dig deep into the docs to make progress, but so far, so good.
On Wed, Mar 20, 2024 at 2:38 PM Matt Burgess <mattyb...@apache.org> wrote: > It would be interesting to see if you exclude the Scylla API JAR from the > Scylla implementation and instead include DataStax's, if that works. > However I'm still leaning towards a vendor-agnostic API. > > On Wed, Mar 20, 2024 at 11:26 AM Mike Thomsen <mikerthom...@gmail.com> > wrote: > > > At first glance, the package names look identical to me: > > > > https://java-driver.docs.scylladb.com/scylla-4.15.0.x/api/index.html > > > > So I see no reason to not take them at their word that it's drop-in > > > > On Wed, Mar 20, 2024 at 11:04 AM David Handermann < > > exceptionfact...@apache.org> wrote: > > > > > Mike, > > > > > > One important thing to mention about the DataStax vs ScyllaDB driver > > > is that the Maven coordinates are different, and managing the > > > dependencies correctly will make or break the implementation. > > > > > > In other words, if it is possible to use the DataStax 4 core JAR in > > > the Controller Service API, but use the ScyllaDB 3 query JAR in the > > > ScyllaDB implementation, then that could avoid the need for additional > > > abstraction. Without taking a closer look, however, I would be > > > surprised if this worked. > > > > > > Although ScyllaDB highlights their forked driver as a drop-in > > > replacement for the DataStax version, and maintains the same Java > > > package names, there is a difference between a complete replacement > > > and a shared API JAR. Without a common API JAR, that both > > > implementations can use, it will be necessary to provide an > > > abstraction in NiFi that avoids depending on either library at the > > > Controller Service API level. > > > > > > Regards, > > > David Handermann > > > > > > On Wed, Mar 20, 2024 at 8:25 AM Mike Thomsen <mikerthom...@gmail.com> > > > wrote: > > > > > > > > Matt/David, > > > > > > > > I got some time this morning to take a crack at directly migrating it > > > over > > > > to the DataStax 4.17 driver. Definitely got a lot of work to do, but > so > > > far > > > > I haven't hit any real snags. This is a branch that reverts the > commit > > to > > > > remove the cassandra bundle and reuses the existing features as a > > > > foundation. From what I'm seeing so far (feels like I'm about 25% > done) > > > it > > > > should be doable to reuse the existing bundle, but rename it to the > > "CQL > > > > Bundle" and just add a second controller service for Scylla that is > > > > otherwise 100% the same codewise. > > > > > > > > On Tue, Mar 19, 2024 at 6:41 PM Mike Thomsen <mikerthom...@gmail.com > > > > > wrote: > > > > > > > > > A cursory look at the Cassandra 5 stuff didn’t indicate any > > > > > incompatibility. So yeah, I think we are likely pretty safe to use > > the > > > 4.17 > > > > > driver > > > > > Sent from my iPhone > > > > > > > > > > > On Mar 19, 2024, at 3:35 PM, Matt Burgess <mattyb...@apache.org> > > > wrote: > > > > > > > > > > > > Is it likely now (due to the refactor) that we will simply be > able > > > to > > > > > > upgrade the driver when Cassandra 5 is GA? Also does anyone use > > > Netflix's > > > > > > Astyanax [1]? > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > https://cassandra.apache.org/doc/stable/cassandra/getting_started/drivers.html#java > > > > > > > > > > > >> On Tue, Mar 19, 2024 at 3:10 PM Mike Thomsen < > > > mikerthom...@gmail.com> > > > > > wrote: > > > > > >> > > > > > >> Realistically, I think we are only likely to see two drivers: > > > > > >> > > > > > >> * DataStax > > > > > >> * ScyllaDB > > > > > >> > > > > > >> The latter makes a selling point of being a binary compatible, > > > drop-in > > > > > >> replacement for the former. > > > > > >> > > > > > >> That's why I don't see a need to have an abstraction layer per > > se. I > > > > > think > > > > > >> we only need "DataStaxConnectionProviderImpl" and > > > > > >> "ScyllaDBConnectionProviderImpl" with the difference being which > > > jar is > > > > > >> imported by maven. > > > > > >> > > > > > >> On Tue, Mar 19, 2024 at 2:59 PM David Handermann < > > > > > >> exceptionfact...@apache.org> wrote: > > > > > >> > > > > > >>> Mike, > > > > > >>> > > > > > >>> Thanks for the reply and clarification. > > > > > >>> > > > > > >>> I agree there is no need to maintain support for the DataStax 3 > > > driver > > > > > >>> and Java API, any new components should be built on the latest > > > version > > > > > >>> of the driver. > > > > > >>> > > > > > >>> What we do need going forward is to avoid, if at all possible, > > > having > > > > > >>> a DataStax 4 dependency in the Controller Service API. > > > > > >>> > > > > > >>> One example of this is the WebClientServiceProvider interface. > > That > > > > > >>> Controller Service API does not have any third-party > > dependencies. > > > The > > > > > >>> Controller Service implementation, > > > StandardWebClientServiceProvider, > > > > > >>> has a dependency on OkHttp to implement HTTP communication. > That > > is > > > > > >>> the kind of abstraction that would be ideal, and I believe that > > > also > > > > > >>> aligns with what Matt has described. > > > > > >>> > > > > > >>> Regards, > > > > > >>> David Handermann > > > > > >>> > > > > > >>> On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen < > > > mikerthom...@gmail.com> > > > > > >>> wrote: > > > > > >>>> > > > > > >>>> ** we can dump v3 **DRIVER** compatibility, since later 4.X > Java > > > > > >> drivers > > > > > >>>> are backward compatible with Cassandra 3 > > > > > >>>> > > > > > >>>> On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen < > > > mikerthom...@gmail.com> > > > > > >>> wrote: > > > > > >>>> > > > > > >>>>> David, > > > > > >>>>> > > > > > >>>>> Before we proceed, I think we should make sure we're all > > > > > >> understanding > > > > > >>> the > > > > > >>>>> same problem here. Starting with this: > > > > > >>>>> > > > > > >>>>>> I believe the CQL protocol is backwards compatible but the > > Java > > > API > > > > > >>> is > > > > > >>>>> not. > > > > > >>>>>> For example "com.datastax.driver.core.Session" is now > > > > > >>>>>> "com.datastax.oss.driver.api.core.session.Session" and there > > is > > > no > > > > > >>> more > > > > > >>>>>> "Cluster" class. Might be fairly trivial to fix though, if > > > that's > > > > > >> the > > > > > >>>>> path > > > > > >>>>>> of least resistance. > > > > > >>>>> > > > > > >>>>> From what I've learned using Cassandra 3 and 4 in my day job > > and > > > > > >>> reading > > > > > >>>>> up on this stuff for the sake of discussion, that all tracks. > > We > > > used > > > > > >>> the > > > > > >>>>> ~4.11 driver in Spring Boot on both v3 and v4 clusters > without > > > issue > > > > > >>> during > > > > > >>>>> an upgrade. So I don't see any reason to factor in the > "changes > > > from > > > > > >>>>> DataStax 3 to 4" since the changes were likely a one-off > > decision > > > > > >>> meant to > > > > > >>>>> position the driver for better future support and stability. > > > > > >>>>> > > > > > >>>>> TL;DR, we can dump v3 compatibility and the only thing our > > users > > > will > > > > > >>>>> notice is if we make the controller service totally > > incompatible > > > with > > > > > >>> the > > > > > >>>>> one they're already using which is something we can actively > > > avoid. > > > > > >>>>> > > > > > >>>>> On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > > > > > >>>>> exceptionfact...@apache.org> wrote: > > > > > >>>>> > > > > > >>>>>> All, > > > > > >>>>>> > > > > > >>>>>> I support a Controller Service API abstraction around the > > > Cassandra > > > > > >>>>>> Driver. The changes from DataStax 3 to 4 already highlight > the > > > need > > > > > >>>>>> for that abstraction. The donation of the DataStax Java > driver > > > to > > > > > >>>>>> Apache [1] also shows the value of providing some level of > > > > > >> isolation, > > > > > >>>>>> if at all possible. > > > > > >>>>>> > > > > > >>>>>> I have not taken a close look at the Matt's branch, and the > > > details > > > > > >> of > > > > > >>>>>> the abstraction are important, but having the abstraction > can > > be > > > > > >>>>>> useful to avoid getting back to this same situation. > > > > > >>>>>> > > > > > >>>>>> Regards, > > > > > >>>>>> David Handermann > > > > > >>>>>> > > > > > >>>>>> [1] https://github.com/apache/cassandra-java-driver/ > > > > > >>>>>> > > > > > >>>>>> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen < > > > > > >> mikerthom...@gmail.com > > > > > >>>> > > > > > >>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>> Matt, > > > > > >>>>>>> > > > > > >>>>>>> I got that. My point was that the Java changes appear to > be a > > > one > > > > > >>> time > > > > > >>>>>>> thing that DataStax did to make a better driver with a much > > > more > > > > > >>>>>>> future-proof API. Since Scylla tracks them as closely as > > > > > >> possible, I > > > > > >>>>>>> suspect that we don't need to plan for a bunch of > abstraction > > > to > > > > > >>> isolate > > > > > >>>>>>> Java changes. > > > > > >>>>>>> > > > > > >>>>>>> On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < > > > > > >>>>>> steven.mati...@gmail.com> > > > > > >>>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>>> That was kinda where i got stuck and fell out on my > > > branch/jira. > > > > > >>>>>> Mike and > > > > > >>>>>>>> I wanted to make a new controller service , without > backward > > > > > >>>>>> compatibility; > > > > > >>>>>>>> and remove the duplicate driver/connection properties > found > > in > > > > > >>> some > > > > > >>>>>> of the > > > > > >>>>>>>> processors. > > > > > >>>>>>>> > > > > > >>>>>>>> I agree taking out all old stuff and making new controller > > > > > >> service > > > > > >>>>>> makes > > > > > >>>>>>>> most sense. 4.x and 5.x should be mostly backwards > > compatible > > > > > >> to > > > > > >>>>>> 2&3.x > > > > > >>>>>>>> with how it’s used within current processors. > > > > > >>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>>> On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess < > > > > > >>> mattyb...@apache.org> > > > > > >>>>>>>> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> The abstraction is to isolate Java API changes, not > > protocol > > > > > >>>>>>>> compatibility > > > > > >>>>>>>>> Changing to the java-driver comes with a number of > changes > > to > > > > > >>> the > > > > > >>>>>> code > > > > > >>>>>>>> (see > > > > > >>>>>>>>> Steven's and my branches), if we can abstract that API it > > > > > >> should > > > > > >>>>>> lead to > > > > > >>>>>>>>> more maintainable code in the future by not having to > > change > > > > > >> any > > > > > >>>>>>>>> processors, just the controller service implementation. > > > > > >>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen < > > > > > >>>>>> mikerthom...@gmail.com> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>> > > > > > >>> > > > > > >> > > > > > > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Directly quoting Scylla docs here: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> The Scylla Java Driver is a drop-in replacement for the > > > > > >>>>>> DataStax Java > > > > > >>>>>>>>>> Driver. As such, no code changes are needed to use this > > > > > >>> driver. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen < > > > > > >>>>>> mikerthom...@gmail.com> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> Matt, > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> I don't think we need to really "abstract above" the > > > > > >> drivers > > > > > >>>>>> because > > > > > >>>>>>>>> the > > > > > >>>>>>>>>>> Java DataStax driver appears to support 4.X all the way > > > > > >>> back to > > > > > >>>>>> 2.X, > > > > > >>>>>>>> as > > > > > >>>>>>>>>>> well as the enterprise versions from DataStax > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>> > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Similar situation with Scylla. When I looked at the > > > > > >> driver, > > > > > >>> it > > > > > >>>>>>>> appeared > > > > > >>>>>>>>>> to > > > > > >>>>>>>>>>> copy verbatim the entire public API of that driver. So > I > > > > > >>> think > > > > > >>>>>> before > > > > > >>>>>>>>> we > > > > > >>>>>>>>>>> dive into abstractions, it's worth doing a bit more > > > > > >>> validation > > > > > >>>>>> of > > > > > >>>>>>>> these > > > > > >>>>>>>>>>> details. IMHO, this might be a much lighter lift than > > > > > >>>>>> anticipated. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess < > > > > > >>>>>> mattyb...@gmail.com> > > > > > >>>>>>>>>> wrote: > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> Totally agree, that's what my branch does (see link in > > > > > >>> previous > > > > > >>>>>>>>> email). > > > > > >>>>>>>>>>>> The > > > > > >>>>>>>>>>>> more I work with it, the more I think I can abstract > it > > > > > >>>>>> further from > > > > > >>>>>>>>>> their > > > > > >>>>>>>>>>>> JDBC-like API but I started with a bunch of delegate > > > > > >>> classes > > > > > >>>>>> then I > > > > > >>>>>>>>>> figure > > > > > >>>>>>>>>>>> I'll see where I can consolidate to more abstract > > > > > >>> concepts. If > > > > > >>>>>> I > > > > > >>>>>>>> don't > > > > > >>>>>>>>>>>> have > > > > > >>>>>>>>>>>> to support Cassandra 3 with the new API, so much the > > > > > >>> better. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>> Matt > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > > > > > >>>>>>>>>>>> exceptionfact...@apache.org> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Matt et al, > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> It is good to see the background effort on moving > > > > > >>> Cassandra > > > > > >>>>>>>>>>>>> capabilities in a supportable direction. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> I think new Cassandra components will require a > > > > > >>> significant > > > > > >>>>>>>>> departure > > > > > >>>>>>>>>>>>> from current Controller Service abstractions. Right > > > > > >> now, > > > > > >>> the > > > > > >>>>>>>>> existing > > > > > >>>>>>>>>>>>> service interface does not provide a clean > abstraction > > > > > >>> from > > > > > >>>>>> the > > > > > >>>>>>>>>>>>> Cassandra library, which is part of the reason for > the > > > > > >>>>>> current > > > > > >>>>>>>>>>>>> coupling to the legacy driver version. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Following up from Joe's comments, it seems like the > > > > > >>> cleanest > > > > > >>>>>> way > > > > > >>>>>>>>>>>>> forward is to deprecate the current bundle on the 1.x > > > > > >>>>>> branch, and > > > > > >>>>>>>>>>>>> remove the current bundle from the main branch. That > > > > > >> will > > > > > >>>>>> provide > > > > > >>>>>>>> a > > > > > >>>>>>>>>>>>> clean slate for new Service and Processor > > > > > >>> implementations, > > > > > >>>>>> without > > > > > >>>>>>>>>>>>> concern for uncertain compatibility questions. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>> David Handermann > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess < > > > > > >>>>>>>> mattyb...@apache.org> > > > > > >>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> What do y'all think about removing the individual > > > > > >>>>>> connection > > > > > >>>>>>>>>>>> properties > > > > > >>>>>>>>>>>>>> from the Cassandra processors for NiFi 2.0 and > > > > > >>> requiring a > > > > > >>>>>>>>>>>>>> CassandraSessionProvider instead? I think we started > > > > > >>> doing > > > > > >>>>>> that > > > > > >>>>>>>>>>>> elsewhere > > > > > >>>>>>>>>>>>>> (Elasticsearch maybe?), I noticed duplicate code in > > > > > >> the > > > > > >>>>>>>>>>>>>> CassandraSessionProvider and > > > > > >>> AbstractCassandraProcessor, > > > > > >>>>>> if we > > > > > >>>>>>>>> keep > > > > > >>>>>>>>>>>> those > > > > > >>>>>>>>>>>>>> properties I can refactor them into a utility class. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks, > > > > > >>>>>>>>>>>>>> Matt > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < > > > > > >>>>>>>>>>>> steven.mati...@gmail.com > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> I got through quite a bit of work to enable 4.x… > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> The 3.x pieces that were not backwards compatible > > > > > >> is > > > > > >>>>>> very edge > > > > > >>>>>>>>> use > > > > > >>>>>>>>>>>>> case and > > > > > >>>>>>>>>>>>>>> could have been done slightly differently but with > > > > > >>> work > > > > > >>>>>>>> around. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>> https://github.com/steven-matison/nifi/tree/nifi-10120-1 > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess < > > > > > >>>>>>>>>> mattyb...@apache.org> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Oops used the wrong email address so if there > > > > > >> have > > > > > >>> been > > > > > >>>>>>>>>> responses > > > > > >>>>>>>>>>>> to > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>> Cassandra thread since mine I missed them, my > > > > > >> bad! > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:00 PM Matt Burgess < > > > > > >>>>>>>>>> mattyb...@gmail.com > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> I believe the CQL protocol is backwards > > > > > >>> compatible > > > > > >>>>>> but the > > > > > >>>>>>>>>> Java > > > > > >>>>>>>>>>>>> API is > > > > > >>>>>>>>>>>>>>>>> not. For example > > > > > >>> "com.datastax.driver.core.Session" > > > > > >>>>>> is now > > > > > >>>>>>>>>>>>>>>>> > > > > > >>> "com.datastax.oss.driver.api.core.session.Session" > > > > > >>>>>> and > > > > > >>>>>>>> there > > > > > >>>>>>>>>> is > > > > > >>>>>>>>>>>> no > > > > > >>>>>>>>>>>>> more > > > > > >>>>>>>>>>>>>>>>> "Cluster" class. Might be fairly trivial to fix > > > > > >>>>>> though, if > > > > > >>>>>>>>>>>> that's > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>> path > > > > > >>>>>>>>>>>>>>>>> of least resistance. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 1:40 PM Joe Witt < > > > > > >>>>>>>>> joe.w...@gmail.com> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Matt > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> I dont know a ton about Cassandra but when I > > > > > >>> looked > > > > > >>>>>> at > > > > > >>>>>>>>>>>>> client/driver > > > > > >>>>>>>>>>>>>>>> notes > > > > > >>>>>>>>>>>>>>>>>> for 4+ it said it was compatible all the way > > > > > >>> back > > > > > >>>>>> to 3.x. > > > > > >>>>>>>>>> Not > > > > > >>>>>>>>>>>>> sure > > > > > >>>>>>>>>>>>>>>> what > > > > > >>>>>>>>>>>>>>>>>> that means but it surely seems worth > > > > > >> exploring. > > > > > >>>>>> Also I > > > > > >>>>>>>>> dont > > > > > >>>>>>>>>>>> know > > > > > >>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>> 4.x drivers get rid of the vulnerable bits. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 10:39 AM Matt Burgess > > > > > >> < > > > > > >>>>>>>>>>>>> mattyb...@apache.org> > > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> At the very least we should upgrade to > > > > > >>> Cassandra > > > > > >>>>>>>> 3.11.6: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>> > > > > > >>> > > > https://github.com/apache/cassandra/blob/cassandra-3.11.16/CHANGES.txt > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 1:31 PM Matt > > > > > >> Burgess < > > > > > >>>>>>>>>>>>> mattyb...@apache.org> > > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> If the community agrees to get rid of > > > > > >>> Cassandra > > > > > >>>>>> 3 > > > > > >>>>>>>>> that'll > > > > > >>>>>>>>>>>>> save me > > > > > >>>>>>>>>>>>>>>>>> effort > > > > > >>>>>>>>>>>>>>>>>>>> on the refactor after I add Cassandra 4 :) > > > > > >>>>>> Otherwise > > > > > >>>>>>>>>> those > > > > > >>>>>>>>>>>>>>>>>>>> vulnerabilities would only be in a "new" > > > > > >>>>>> Cassandra 3 > > > > > >>>>>>>>>>>> services > > > > > >>>>>>>>>>>>> NAR > > > > > >>>>>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>>>>>>>> would not be included in the convenience > > > > > >>> binary. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 1:28 PM Joe Witt < > > > > > >>>>>>>>>>>> joe.w...@gmail.com> > > > > > >>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Mike, Matt, > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Happy to hear you both have active > > > > > >> efforts > > > > > >>> or > > > > > >>>>>> are > > > > > >>>>>>>>>>>> interested > > > > > >>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>> doing > > > > > >>>>>>>>>>>>>>>>>>> so. > > > > > >>>>>>>>>>>>>>>>>>>>> Can you help me understand more > > > > > >>> specifically > > > > > >>>>>> what > > > > > >>>>>>>> that > > > > > >>>>>>>>>>>> means > > > > > >>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>>>>> current set of components? > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> The CVE hits are concerning and long > > > > > >>> standing. > > > > > >>>>>>>>>> Supporting > > > > > >>>>>>>>>>>>>>>> Cassandra > > > > > >>>>>>>>>>>>>>>>>> 3 > > > > > >>>>>>>>>>>>>>>>>>>>> implies the current set of dependencies > > > > > >>> would > > > > > >>>>>> remain > > > > > >>>>>>>>> too > > > > > >>>>>>>>>>>>> right? > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Is the current set of components we have > > > > > >>> ones > > > > > >>>>>> we > > > > > >>>>>>>> want > > > > > >>>>>>>>> to > > > > > >>>>>>>>>>>>> retain? > > > > > >>>>>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>>>>>>>>>> certainly need Cassandra components - but > > > > > >>> are > > > > > >>>>>> the > > > > > >>>>>>>> ones > > > > > >>>>>>>>>> we > > > > > >>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>>>> now > > > > > >>>>>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>>>>> right ones? > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>>>>> Joe > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 10:25 AM Matt > > > > > >>> Burgess < > > > > > >>>>>>>>>>>>>>>> mattyb...@apache.org> > > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> I'm actively working this, I pushed my > > > > > >>>>>> branch up > > > > > >>>>>>>> in > > > > > >>>>>>>>>> case > > > > > >>>>>>>>>>>>> anyone > > > > > >>>>>>>>>>>>>>>>>> wants > > > > > >>>>>>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>>>>>> take a look [1]. The idea is to > > > > > >> abstract > > > > > >>> the > > > > > >>>>>>>>> Cassandra > > > > > >>>>>>>>>>>> API > > > > > >>>>>>>>>>>>> "up > > > > > >>>>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>>>>>>>> couple > > > > > >>>>>>>>>>>>>>>>>>>>>> levels" and provide implementations for > > > > > >>>>>> Cassandra > > > > > >>>>>>>> 3, > > > > > >>>>>>>>>> 4, > > > > > >>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>> eventually > > > > > >>>>>>>>>>>>>>>>>>>>> 5. > > > > > >>>>>>>>>>>>>>>>>>>>>> For JDBC-like interfaces this is a PITA > > > > > >>>>>> because of > > > > > >>>>>>>>> the > > > > > >>>>>>>>>>>> API > > > > > >>>>>>>>>>>>>>>>>> (Statement, > > > > > >>>>>>>>>>>>>>>>>>>>>> PreparedStatement, BoundStatement, > > > > > >>> ResultSet, > > > > > >>>>>>>> etc.) > > > > > >>>>>>>>>> but > > > > > >>>>>>>>>>>> I'm > > > > > >>>>>>>>>>>>>>>> hoping > > > > > >>>>>>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>>>>>>>>> find a common pattern for abstracting > > > > > >> the > > > > > >>>>>>>>> third-party > > > > > >>>>>>>>>>>>> library > > > > > >>>>>>>>>>>>>>>>>>>>>> implementation and API from the NiFi > > > > > >>>>>> component > > > > > >>>>>>>>>>>> (Processor, > > > > > >>>>>>>>>>>>>>>>>>>>>> ControllerService, etc.) API. I think > > > > > >>> we're > > > > > >>>>>> doing > > > > > >>>>>>>>>>>> something > > > > > >>>>>>>>>>>>>>>> similar > > > > > >>>>>>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>>>>>>> Kafka? > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> Regards, > > > > > >>>>>>>>>>>>>>>>>>>>>> Matt > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> [1] > > > > > >>>>>> https://github.com/mattyb149/nifi/tree/cassy4 > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 8:43 AM Mike > > > > > >>> Thomsen > > > > > >>>>>> < > > > > > >>>>>>>>>>>>>>>>>> mikerthom...@gmail.com> > > > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> That’s been on my todo list for a > > > > > >>> little > > > > > >>>>>> while > > > > > >>>>>>>> but > > > > > >>>>>>>>>>>> things > > > > > >>>>>>>>>>>>>>> kept > > > > > >>>>>>>>>>>>>>>>>>> coming > > > > > >>>>>>>>>>>>>>>>>>>>> up. > > > > > >>>>>>>>>>>>>>>>>>>>>>> I think I could get started on that > > > > > >>> now. > > > > > >>>>>> Based > > > > > >>>>>>>> on > > > > > >>>>>>>>> my > > > > > >>>>>>>>>>>>> initial > > > > > >>>>>>>>>>>>>>>>>>> research > > > > > >>>>>>>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>>>>>>> appears that scylla uses the exact > > > > > >> same > > > > > >>>>>> api as > > > > > >>>>>>>>>>>> datastax > > > > > >>>>>>>>>>>>> so > > > > > >>>>>>>>>>>>>>>>>>> supporting > > > > > >>>>>>>>>>>>>>>>>>>>>> both > > > > > >>>>>>>>>>>>>>>>>>>>>>> in a cql bundle should theoretically > > > > > >> be > > > > > >>>>>> fairly > > > > > >>>>>>>>> easy. > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>> Sent from my iPhone > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> On Mar 14, 2024, at 6:18 PM, Joe > > > > > >>> Witt < > > > > > >>>>>>>>>>>>> joew...@apache.org> > > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> Team, > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> Cassandra remains a really > > > > > >> important > > > > > >>>>>> system to > > > > > >>>>>>>>> be > > > > > >>>>>>>>>>>> able > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> send > > > > > >>>>>>>>>>>>>>>>>>> data > > > > > >>>>>>>>>>>>>>>>>>>>> to. > > > > > >>>>>>>>>>>>>>>>>>>>>>>> However, it seems like we've not > > > > > >>>>>> maintained > > > > > >>>>>>>>> these > > > > > >>>>>>>>>>>>> well. We > > > > > >>>>>>>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>>>>>>>>>> what > > > > > >>>>>>>>>>>>>>>>>>>>>>>> appears to be at least a full > > > > > >>> generation > > > > > >>>>>>>> behind > > > > > >>>>>>>>> on > > > > > >>>>>>>>>>>>> client > > > > > >>>>>>>>>>>>>>>>>> versions > > > > > >>>>>>>>>>>>>>>>>>>>> (we > > > > > >>>>>>>>>>>>>>>>>>>>>>> are > > > > > >>>>>>>>>>>>>>>>>>>>>>>> on 3x vs 4x which is the latest > > > > > >>> stable > > > > > >>>>>> with 5x > > > > > >>>>>>>>>>>>> apparently > > > > > >>>>>>>>>>>>>>>>>> coming > > > > > >>>>>>>>>>>>>>>>>>>>>>> shortly). > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> We have components to send data, > > > > > >>> query > > > > > >>>>>> data, > > > > > >>>>>>>> and > > > > > >>>>>>>>>> use > > > > > >>>>>>>>>>>>>>>> Cassandra > > > > > >>>>>>>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>>>>>>>>>>> cache > > > > > >>>>>>>>>>>>>>>>>>>>>>>> store. We have older mechanisms > > > > > >> for > > > > > >>>>>> json/avro > > > > > >>>>>>>>> and > > > > > >>>>>>>>>>>>> publish > > > > > >>>>>>>>>>>>>>>>>>>>> mechanisms > > > > > >>>>>>>>>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>>>>>>>>> records. > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> The libraries we do have depend on > > > > > >>>>>> outdated > > > > > >>>>>>>>>>>> versions of > > > > > >>>>>>>>>>>>>>> Guava > > > > > >>>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>>>>>> result > > > > > >>>>>>>>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>>>>>>>> many CVE hits. > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> I am inclined to think we should > > > > > >>>>>> deprecate the > > > > > >>>>>>>>> 1.x > > > > > >>>>>>>>>>>>>>> components > > > > > >>>>>>>>>>>>>>>>>> and > > > > > >>>>>>>>>>>>>>>>>>>>>> remove > > > > > >>>>>>>>>>>>>>>>>>>>>>>> them as-is from the 2.x line. Then > > > > > >>>>>>>> re-introduce > > > > > >>>>>>>>>>>> them > > > > > >>>>>>>>>>>>> with > > > > > >>>>>>>>>>>>>>>>>> record > > > > > >>>>>>>>>>>>>>>>>>>>> only > > > > > >>>>>>>>>>>>>>>>>>>>>>>> interfaces and built against the > > > > > >>> latest > > > > > >>>>>> stable > > > > > >>>>>>>>>>>>>>>>>>>>>>> Cassandra/Datastax/ScyllaDB > > > > > >>>>>>>>>>>>>>>>>>>>>>>> interfaces. > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> I'd love to hear thoughts from > > > > > >> those > > > > > >>>>>> closer to > > > > > >>>>>>>>>> this > > > > > >>>>>>>>>>>>> space > > > > > >>>>>>>>>>>>>>>> both > > > > > >>>>>>>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>>>>>>>> a > > > > > >>>>>>>>>>>>>>>>>>>>>> user > > > > > >>>>>>>>>>>>>>>>>>>>>>>> and developer so we can make good > > > > > >>> next > > > > > >>>>>> steps. > > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>> > > > > > >> > > > > > > > > > > >
