Matt, IIRC in your PR you had abstractions on the basic APIs. I think the right approach would be to make the driver controller services provide an API for querying so the processors and other services never have to work with the api or an abstraction of it.
I have a lot of progress on such an an approach I can post to my repo if you're interested. On Fri, May 17, 2024 at 7:31 AM Matt Burgess <mattyb...@gmail.com> wrote: > Now that the Couchbase PR is up I can continue my work on this if > everyone's ok with the approach. > > On Fri, May 17, 2024 at 5:30 AM Pierre Villard < > pierre.villard...@gmail.com> > wrote: > > > Hey guys, what's the latest on this? > > > > Le sam. 23 mars 2024 à 01:49, Mike Thomsen <mikerthom...@gmail.com> a > > écrit : > > > > > Fair enough, Joe. > > > > > > Matt, > > > > > > I poked around your branch a little this evening. I agree with you and > > > David 100% now on the need for some sort of abstraction. I think the > > Graph > > > Bundle's model could serve as a good starting point for how to approach > > the > > > problem. The client drivers in that bundle do the heavy lifting of > doing > > > the querying and passing the results back via a callback system to the > > > processors that call them to ensure that the processors don't know > > anything > > > other than they're getting back a Map of result data each iteration. > > > Thoughts? > > > > > > On Fri, Mar 22, 2024 at 2:48 PM Joe Witt <joe.w...@gmail.com> wrote: > > > > > > > Mike, > > > > > > > > The bundles we include cannot have libs with know vulns and that > last a > > > > very long time. That is a more pressing blocker. > > > > > > > > As noted top of thread we all recognize the importance of being able > to > > > > integrate with Cassandra but including that must come with active mx > > > > especially as it relates to vulns. > > > > > > > > Thanks > > > > Joe > > > > > > > > On Fri, Mar 22, 2024 at 11:42 AM Mike Thomsen < > mikerthom...@gmail.com> > > > > wrote: > > > > > > > > > The scope tag was probably copy pasta. You raise a valid point > about > > > the > > > > > processor dependencies that completely slipped my mind. That said, > I > > > > think > > > > > it's premature to remove the cassandra bundle until we have a > working > > > > > replacement. I would consider that support a blocker for 2.X. > > > > > > > > > > On Fri, Mar 22, 2024 at 12:00 PM Matt Burgess <mattyb...@gmail.com > > > > > > wrote: > > > > > > > > > > > David beat me to it :) IMO the only NAR that should have any > > > > dependencies > > > > > > on Cassandra is the services NAR, not the processors or services > > API. > > > > > > > > > > > > On Fri, Mar 22, 2024 at 11:10 AM David Handermann < > > > > > > exceptionfact...@apache.org> wrote: > > > > > > > > > > > > > Mike, > > > > > > > > > > > > > > Thanks for sharing the branch, it is helpful to have that as a > > > > > > > reference example. Have you been able to exercise any of that > > > > approach > > > > > > > at runtime? > > > > > > > > > > > > > > Based on what is there right now, attempting to mark the > DataStax > > > > > > > java-driver-core as provided does not look like it will work. > It > > > may > > > > > > > pass unit tests, but runtime NAR class loading requires that > > > classes > > > > > > > be available in the same NAR, or in a parent NAR. That means > when > > > > NiFi > > > > > > > tries to load the Controller Service interface, it must have > > access > > > > to > > > > > > > a version of the relevant Cassandra driver classes. By marking > > the > > > > > > > dependency as provided, it will not be available in the API > NAR, > > > and > > > > > > > thus not available when loading the service interface. > Including > > it > > > > in > > > > > > > the API NAR won't work either, because it conflicts with the > > > ScyllaDB > > > > > > > java-driver-core in the implementation NAR. > > > > > > > > > > > > > > This is the reason Matt and I highlighted for providing a layer > > of > > > > > > > abstraction at the Controller Service API level. > > > > > > > > > > > > > > Regards, > > > > > > > David Handermann > > > > > > > > > > > > > > On Fri, Mar 22, 2024 at 8:13 AM Mike Thomsen < > > > mikerthom...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > Work so far: > > > https://github.com/MikeThomsen/nifi/tree/cql-changes > > > > > > > > > > > > > > > > On Thu, Mar 21, 2024 at 9:52 AM Mike Thomsen < > > > > mikerthom...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Matt/David, > > > > > > > > > > > > > > > > > > By this evening, I should be at a point where I can share > my > > > > > branch. > > > > > > It > > > > > > > > > should be far enough along that y'all can see what I mean > > about > > > > how > > > > > > > most of > > > > > > > > > the changes really weren't that complicated. My sense is > that > > > if > > > > we > > > > > > > > > collaborate on it, we can probably get it ready for a PR > > > within a > > > > > > week > > > > > > > or > > > > > > > > > two. > > > > > > > > > > > > > > > > > > It would probably be a good idea to plan to revisit the > > > Cassandra > > > > > > DMC's > > > > > > > > > design and make it more flexible. > > > > > > > > > > > > > > > > > > One nice thing about the new DataStax driver is that it > > > supports > > > > > > > > > configuration by a very detailed configuration file format, > > so > > > we > > > > > can > > > > > > > give > > > > > > > > > users that option + combine it with EL/parameters (I > envision > > > an > > > > > > option > > > > > > > > > where the user puts EL in the file, we load the file, > > > preprocess > > > > > the > > > > > > > EL and > > > > > > > > > load that into the driver) > > > > > > > > > > > > > > > > > > On Wed, Mar 20, 2024 at 4:01 PM Mike Thomsen < > > > > > mikerthom...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> If it were that simple, they would probably have just gone > > > with > > > > > that > > > > > > > > >> solution. That said, the API is functionally vendor > agnostic > > > at > > > > > this > > > > > > > point > > > > > > > > >> at the Java API level. So I see no need to add abstraction > > > above > > > > > > > that. I've > > > > > > > > >> got probably 2/3 of nifi-cassandra-bundle converted. > > Hitting a > > > > few > > > > > > > pain > > > > > > > > >> points where I'm having to dig deep into the docs to make > > > > > progress, > > > > > > > but so > > > > > > > > >> far, so good. > > > > > > > > >> > > > > > > > > >> On Wed, Mar 20, 2024 at 2:38 PM Matt Burgess < > > > > > mattyb...@apache.org> > > > > > > > > >> wrote: > > > > > > > > >> > > > > > > > > >>> It would be interesting to see if you exclude the Scylla > > API > > > > JAR > > > > > > > from the > > > > > > > > >>> Scylla implementation and instead include DataStax's, if > > that > > > > > > works. > > > > > > > > >>> However I'm still leaning towards a vendor-agnostic API. > > > > > > > > >>> > > > > > > > > >>> On Wed, Mar 20, 2024 at 11:26 AM Mike Thomsen < > > > > > > > mikerthom...@gmail.com> > > > > > > > > >>> wrote: > > > > > > > > >>> > > > > > > > > >>> > At first glance, the package names look identical to > me: > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > > > https://java-driver.docs.scylladb.com/scylla-4.15.0.x/api/index.html > > > > > > > > >>> > > > > > > > > > >>> > So I see no reason to not take them at their word that > > it's > > > > > > drop-in > > > > > > > > >>> > > > > > > > > > >>> > On Wed, Mar 20, 2024 at 11:04 AM David Handermann < > > > > > > > > >>> > exceptionfact...@apache.org> wrote: > > > > > > > > >>> > > > > > > > > > >>> > > Mike, > > > > > > > > >>> > > > > > > > > > > >>> > > One important thing to mention about the DataStax vs > > > > ScyllaDB > > > > > > > driver > > > > > > > > >>> > > is that the Maven coordinates are different, and > > managing > > > > the > > > > > > > > >>> > > dependencies correctly will make or break the > > > > implementation. > > > > > > > > >>> > > > > > > > > > > >>> > > In other words, if it is possible to use the > DataStax 4 > > > > core > > > > > > JAR > > > > > > > in > > > > > > > > >>> > > the Controller Service API, but use the ScyllaDB 3 > > query > > > > JAR > > > > > in > > > > > > > the > > > > > > > > >>> > > ScyllaDB implementation, then that could avoid the > need > > > for > > > > > > > > >>> additional > > > > > > > > >>> > > abstraction. Without taking a closer look, however, I > > > would > > > > > be > > > > > > > > >>> > > surprised if this worked. > > > > > > > > >>> > > > > > > > > > > >>> > > Although ScyllaDB highlights their forked driver as a > > > > drop-in > > > > > > > > >>> > > replacement for the DataStax version, and maintains > the > > > > same > > > > > > Java > > > > > > > > >>> > > package names, there is a difference between a > complete > > > > > > > replacement > > > > > > > > >>> > > and a shared API JAR. Without a common API JAR, that > > both > > > > > > > > >>> > > implementations can use, it will be necessary to > > provide > > > an > > > > > > > > >>> > > abstraction in NiFi that avoids depending on either > > > library > > > > > at > > > > > > > the > > > > > > > > >>> > > Controller Service API level. > > > > > > > > >>> > > > > > > > > > > >>> > > Regards, > > > > > > > > >>> > > David Handermann > > > > > > > > >>> > > > > > > > > > > >>> > > On Wed, Mar 20, 2024 at 8:25 AM Mike Thomsen < > > > > > > > mikerthom...@gmail.com > > > > > > > > >>> > > > > > > > > > >>> > > wrote: > > > > > > > > >>> > > > > > > > > > > > >>> > > > Matt/David, > > > > > > > > >>> > > > > > > > > > > > >>> > > > I got some time this morning to take a crack at > > > directly > > > > > > > migrating > > > > > > > > >>> it > > > > > > > > >>> > > over > > > > > > > > >>> > > > to the DataStax 4.17 driver. Definitely got a lot > of > > > work > > > > > to > > > > > > > do, > > > > > > > > >>> but so > > > > > > > > >>> > > far > > > > > > > > >>> > > > I haven't hit any real snags. This is a branch that > > > > reverts > > > > > > the > > > > > > > > >>> commit > > > > > > > > >>> > to > > > > > > > > >>> > > > remove the cassandra bundle and reuses the existing > > > > > features > > > > > > > as a > > > > > > > > >>> > > > foundation. From what I'm seeing so far (feels like > > I'm > > > > > about > > > > > > > 25% > > > > > > > > >>> done) > > > > > > > > >>> > > it > > > > > > > > >>> > > > should be doable to reuse the existing bundle, but > > > rename > > > > > it > > > > > > > to the > > > > > > > > >>> > "CQL > > > > > > > > >>> > > > Bundle" and just add a second controller service > for > > > > Scylla > > > > > > > that is > > > > > > > > >>> > > > otherwise 100% the same codewise. > > > > > > > > >>> > > > > > > > > > > > >>> > > > On Tue, Mar 19, 2024 at 6:41 PM Mike Thomsen < > > > > > > > > >>> mikerthom...@gmail.com> > > > > > > > > >>> > > wrote: > > > > > > > > >>> > > > > > > > > > > > >>> > > > > A cursory look at the Cassandra 5 stuff didn’t > > > indicate > > > > > any > > > > > > > > >>> > > > > incompatibility. So yeah, I think we are likely > > > pretty > > > > > safe > > > > > > > to > > > > > > > > >>> use > > > > > > > > >>> > the > > > > > > > > >>> > > 4.17 > > > > > > > > >>> > > > > driver > > > > > > > > >>> > > > > Sent from my iPhone > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > On Mar 19, 2024, at 3:35 PM, Matt Burgess < > > > > > > > > >>> mattyb...@apache.org> > > > > > > > > >>> > > wrote: > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > Is it likely now (due to the refactor) that we > > > will > > > > > > > simply be > > > > > > > > >>> able > > > > > > > > >>> > > to > > > > > > > > >>> > > > > > upgrade the driver when Cassandra 5 is GA? Also > > > does > > > > > > > anyone use > > > > > > > > >>> > > Netflix's > > > > > > > > >>> > > > > > Astyanax [1]? > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > [1] > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cassandra.apache.org/doc/stable/cassandra/getting_started/drivers.html#java > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > >> On Tue, Mar 19, 2024 at 3:10 PM Mike Thomsen < > > > > > > > > >>> > > mikerthom...@gmail.com> > > > > > > > > >>> > > > > wrote: > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >> Realistically, I think we are only likely to > see > > > two > > > > > > > drivers: > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >> * DataStax > > > > > > > > >>> > > > > >> * ScyllaDB > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >> The latter makes a selling point of being a > > binary > > > > > > > compatible, > > > > > > > > >>> > > drop-in > > > > > > > > >>> > > > > >> replacement for the former. > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >> That's why I don't see a need to have an > > > abstraction > > > > > > > layer per > > > > > > > > >>> > se. I > > > > > > > > >>> > > > > think > > > > > > > > >>> > > > > >> we only need "DataStaxConnectionProviderImpl" > > and > > > > > > > > >>> > > > > >> "ScyllaDBConnectionProviderImpl" with the > > > difference > > > > > > being > > > > > > > > >>> which > > > > > > > > >>> > > jar is > > > > > > > > >>> > > > > >> imported by maven. > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >> On Tue, Mar 19, 2024 at 2:59 PM David > > Handermann < > > > > > > > > >>> > > > > >> exceptionfact...@apache.org> wrote: > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > >>> Mike, > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> Thanks for the reply and clarification. > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> I agree there is no need to maintain support > > for > > > > the > > > > > > > > >>> DataStax 3 > > > > > > > > >>> > > driver > > > > > > > > >>> > > > > >>> and Java API, any new components should be > > built > > > on > > > > > the > > > > > > > > >>> latest > > > > > > > > >>> > > version > > > > > > > > >>> > > > > >>> of the driver. > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> What we do need going forward is to avoid, if > > at > > > > all > > > > > > > > >>> possible, > > > > > > > > >>> > > having > > > > > > > > >>> > > > > >>> a DataStax 4 dependency in the Controller > > Service > > > > > API. > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> One example of this is the > > > WebClientServiceProvider > > > > > > > > >>> interface. > > > > > > > > >>> > That > > > > > > > > >>> > > > > >>> Controller Service API does not have any > > > > third-party > > > > > > > > >>> > dependencies. > > > > > > > > >>> > > The > > > > > > > > >>> > > > > >>> Controller Service implementation, > > > > > > > > >>> > > StandardWebClientServiceProvider, > > > > > > > > >>> > > > > >>> has a dependency on OkHttp to implement HTTP > > > > > > > communication. > > > > > > > > >>> That > > > > > > > > >>> > is > > > > > > > > >>> > > > > >>> the kind of abstraction that would be ideal, > > and > > > I > > > > > > > believe > > > > > > > > >>> that > > > > > > > > >>> > > also > > > > > > > > >>> > > > > >>> aligns with what Matt has described. > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> Regards, > > > > > > > > >>> > > > > >>> David Handermann > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >>> On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen > < > > > > > > > > >>> > > mikerthom...@gmail.com> > > > > > > > > >>> > > > > >>> wrote: > > > > > > > > >>> > > > > >>>> > > > > > > > > >>> > > > > >>>> ** we can dump v3 **DRIVER** compatibility, > > > since > > > > > > later > > > > > > > 4.X > > > > > > > > >>> Java > > > > > > > > >>> > > > > >> drivers > > > > > > > > >>> > > > > >>>> are backward compatible with Cassandra 3 > > > > > > > > >>> > > > > >>>> > > > > > > > > >>> > > > > >>>> On Tue, Mar 19, 2024 at 2:43 PM Mike > Thomsen < > > > > > > > > >>> > > mikerthom...@gmail.com> > > > > > > > > >>> > > > > >>> wrote: > > > > > > > > >>> > > > > >>>> > > > > > > > > >>> > > > > >>>>> David, > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>> Before we proceed, I think we should make > > sure > > > > > we're > > > > > > > all > > > > > > > > >>> > > > > >> understanding > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>> same problem here. Starting with this: > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>>> I believe the CQL protocol is backwards > > > > compatible > > > > > > > but the > > > > > > > > >>> > Java > > > > > > > > >>> > > API > > > > > > > > >>> > > > > >>> is > > > > > > > > >>> > > > > >>>>> not. > > > > > > > > >>> > > > > >>>>>> For example > > "com.datastax.driver.core.Session" > > > > is > > > > > > now > > > > > > > > >>> > > > > >>>>>> > > > > "com.datastax.oss.driver.api.core.session.Session" > > > > > > and > > > > > > > > >>> there > > > > > > > > >>> > is > > > > > > > > >>> > > no > > > > > > > > >>> > > > > >>> more > > > > > > > > >>> > > > > >>>>>> "Cluster" class. Might be fairly trivial > to > > > fix > > > > > > > though, if > > > > > > > > >>> > > that's > > > > > > > > >>> > > > > >> the > > > > > > > > >>> > > > > >>>>> path > > > > > > > > >>> > > > > >>>>>> of least resistance. > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>> From what I've learned using Cassandra 3 > and > > 4 > > > in > > > > > my > > > > > > > day > > > > > > > > >>> job > > > > > > > > >>> > and > > > > > > > > >>> > > > > >>> reading > > > > > > > > >>> > > > > >>>>> up on this stuff for the sake of > discussion, > > > that > > > > > all > > > > > > > > >>> tracks. > > > > > > > > >>> > We > > > > > > > > >>> > > used > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>> ~4.11 driver in Spring Boot on both v3 and > v4 > > > > > > clusters > > > > > > > > >>> without > > > > > > > > >>> > > issue > > > > > > > > >>> > > > > >>> during > > > > > > > > >>> > > > > >>>>> an upgrade. So I don't see any reason to > > factor > > > > in > > > > > > the > > > > > > > > >>> "changes > > > > > > > > >>> > > from > > > > > > > > >>> > > > > >>>>> DataStax 3 to 4" since the changes were > > likely > > > a > > > > > > > one-off > > > > > > > > >>> > decision > > > > > > > > >>> > > > > >>> meant to > > > > > > > > >>> > > > > >>>>> position the driver for better future > support > > > and > > > > > > > > >>> stability. > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>> TL;DR, we can dump v3 compatibility and the > > > only > > > > > > thing > > > > > > > our > > > > > > > > >>> > users > > > > > > > > >>> > > will > > > > > > > > >>> > > > > >>>>> notice is if we make the controller service > > > > totally > > > > > > > > >>> > incompatible > > > > > > > > >>> > > with > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>> one they're already using which is > something > > we > > > > can > > > > > > > > >>> actively > > > > > > > > >>> > > avoid. > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>> On Tue, Mar 19, 2024 at 2:00 PM David > > > Handermann > > > > < > > > > > > > > >>> > > > > >>>>> exceptionfact...@apache.org> wrote: > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>>>>> All, > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>>> I support a Controller Service API > > abstraction > > > > > > around > > > > > > > the > > > > > > > > >>> > > Cassandra > > > > > > > > >>> > > > > >>>>>> Driver. The changes from DataStax 3 to 4 > > > already > > > > > > > > >>> highlight the > > > > > > > > >>> > > need > > > > > > > > >>> > > > > >>>>>> for that abstraction. The donation of the > > > > DataStax > > > > > > > Java > > > > > > > > >>> driver > > > > > > > > >>> > > to > > > > > > > > >>> > > > > >>>>>> Apache [1] also shows the value of > providing > > > > some > > > > > > > level of > > > > > > > > >>> > > > > >> isolation, > > > > > > > > >>> > > > > >>>>>> if at all possible. > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>>> I have not taken a close look at the > Matt's > > > > > branch, > > > > > > > and > > > > > > > > >>> the > > > > > > > > >>> > > details > > > > > > > > >>> > > > > >> of > > > > > > > > >>> > > > > >>>>>> the abstraction are important, but having > > the > > > > > > > abstraction > > > > > > > > >>> can > > > > > > > > >>> > be > > > > > > > > >>> > > > > >>>>>> useful to avoid getting back to this same > > > > > situation. > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>>> Regards, > > > > > > > > >>> > > > > >>>>>> David Handermann > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>>> [1] > > > > > > https://github.com/apache/cassandra-java-driver/ > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>>> On Tue, Mar 19, 2024 at 12:37 PM Mike > > Thomsen > > > < > > > > > > > > >>> > > > > >> mikerthom...@gmail.com > > > > > > > > >>> > > > > >>>> > > > > > > > > >>> > > > > >>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>> > > > > > > > > >>> > > > > >>>>>>> Matt, > > > > > > > > >>> > > > > >>>>>>> > > > > > > > > >>> > > > > >>>>>>> I got that. My point was that the Java > > > changes > > > > > > > appear to > > > > > > > > >>> be a > > > > > > > > >>> > > one > > > > > > > > >>> > > > > >>> time > > > > > > > > >>> > > > > >>>>>>> thing that DataStax did to make a better > > > driver > > > > > > with > > > > > > > a > > > > > > > > >>> much > > > > > > > > >>> > > more > > > > > > > > >>> > > > > >>>>>>> future-proof API. Since Scylla tracks > them > > as > > > > > > > closely as > > > > > > > > >>> > > > > >> possible, I > > > > > > > > >>> > > > > >>>>>>> suspect that we don't need to plan for a > > > bunch > > > > of > > > > > > > > >>> abstraction > > > > > > > > >>> > > to > > > > > > > > >>> > > > > >>> isolate > > > > > > > > >>> > > > > >>>>>>> Java changes. > > > > > > > > >>> > > > > >>>>>>> > > > > > > > > >>> > > > > >>>>>>> On Tue, Mar 19, 2024 at 11:07 AM Steven > > > > Matison < > > > > > > > > >>> > > > > >>>>>> steven.mati...@gmail.com> > > > > > > > > >>> > > > > >>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>> > > > > > > > > >>> > > > > >>>>>>>> That was kinda where i got stuck and > fell > > > out > > > > on > > > > > > my > > > > > > > > >>> > > branch/jira. > > > > > > > > >>> > > > > >>>>>> Mike and > > > > > > > > >>> > > > > >>>>>>>> I wanted to make a new controller > service > > , > > > > > > without > > > > > > > > >>> backward > > > > > > > > >>> > > > > >>>>>> compatibility; > > > > > > > > >>> > > > > >>>>>>>> and remove the duplicate > driver/connection > > > > > > > properties > > > > > > > > >>> found > > > > > > > > >>> > in > > > > > > > > >>> > > > > >>> some > > > > > > > > >>> > > > > >>>>>> of the > > > > > > > > >>> > > > > >>>>>>>> processors. > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> I agree taking out all old stuff and > > making > > > > new > > > > > > > > >>> controller > > > > > > > > >>> > > > > >> service > > > > > > > > >>> > > > > >>>>>> makes > > > > > > > > >>> > > > > >>>>>>>> most sense. 4.x and 5.x should be > mostly > > > > > > backwards > > > > > > > > >>> > compatible > > > > > > > > >>> > > > > >> to > > > > > > > > >>> > > > > >>>>>> 2&3.x > > > > > > > > >>> > > > > >>>>>>>> with how it’s used within current > > > processors. > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> On Tue, Mar 19, 2024 at 10:49 AM Matt > > > Burgess > > > > < > > > > > > > > >>> > > > > >>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>> The abstraction is to isolate Java API > > > > changes, > > > > > > not > > > > > > > > >>> > protocol > > > > > > > > >>> > > > > >>>>>>>> compatibility > > > > > > > > >>> > > > > >>>>>>>>> Changing to the java-driver comes with > a > > > > number > > > > > > of > > > > > > > > >>> changes > > > > > > > > >>> > to > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>>> code > > > > > > > > >>> > > > > >>>>>>>> (see > > > > > > > > >>> > > > > >>>>>>>>> Steven's and my branches), if we can > > > abstract > > > > > > that > > > > > > > API > > > > > > > > >>> it > > > > > > > > >>> > > > > >> should > > > > > > > > >>> > > > > >>>>>> lead to > > > > > > > > >>> > > > > >>>>>>>>> more maintainable code in the future by > > not > > > > > > having > > > > > > > to > > > > > > > > >>> > change > > > > > > > > >>> > > > > >> any > > > > > > > > >>> > > > > >>>>>>>>> processors, just the controller service > > > > > > > implementation. > > > > > > > > >>> > > > > >>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>> On Tue, Mar 19, 2024 at 10:14 AM Mike > > > > Thomsen < > > > > > > > > >>> > > > > >>>>>> mikerthom...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> Directly quoting Scylla docs here: > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> The Scylla Java Driver is a drop-in > > > > > replacement > > > > > > > for > > > > > > > > >>> the > > > > > > > > >>> > > > > >>>>>> DataStax Java > > > > > > > > >>> > > > > >>>>>>>>>> Driver. As such, no code changes are > > > needed > > > > to > > > > > > use > > > > > > > > >>> this > > > > > > > > >>> > > > > >>> driver. > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> On Tue, Mar 19, 2024 at 10:13 AM Mike > > > > Thomsen > > > > > < > > > > > > > > >>> > > > > >>>>>> mikerthom...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> Matt, > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> I don't think we need to really > > "abstract > > > > > > above" > > > > > > > the > > > > > > > > >>> > > > > >> drivers > > > > > > > > >>> > > > > >>>>>> because > > > > > > > > >>> > > > > >>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>> Java DataStax driver appears to > support > > > 4.X > > > > > all > > > > > > > the > > > > > > > > >>> way > > > > > > > > >>> > > > > >>> back to > > > > > > > > >>> > > > > >>>>>> 2.X, > > > > > > > > >>> > > > > >>>>>>>> as > > > > > > > > >>> > > > > >>>>>>>>>>> well as the enterprise versions from > > > > DataStax > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > > > > > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> Similar situation with Scylla. When I > > > > looked > > > > > at > > > > > > > the > > > > > > > > >>> > > > > >> driver, > > > > > > > > >>> > > > > >>> it > > > > > > > > >>> > > > > >>>>>>>> appeared > > > > > > > > >>> > > > > >>>>>>>>>> to > > > > > > > > >>> > > > > >>>>>>>>>>> copy verbatim the entire public API > of > > > that > > > > > > > driver. > > > > > > > > >>> So I > > > > > > > > >>> > > > > >>> think > > > > > > > > >>> > > > > >>>>>> before > > > > > > > > >>> > > > > >>>>>>>>> we > > > > > > > > >>> > > > > >>>>>>>>>>> dive into abstractions, it's worth > > doing > > > a > > > > > bit > > > > > > > more > > > > > > > > >>> > > > > >>> validation > > > > > > > > >>> > > > > >>>>>> of > > > > > > > > >>> > > > > >>>>>>>> these > > > > > > > > >>> > > > > >>>>>>>>>>> details. IMHO, this might be a much > > > lighter > > > > > > lift > > > > > > > than > > > > > > > > >>> > > > > >>>>>> anticipated. > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> On Mon, Mar 18, 2024 at 4:30 PM Matt > > > > Burgess > > > > > < > > > > > > > > >>> > > > > >>>>>> mattyb...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>> Totally agree, that's what my branch > > > does > > > > > (see > > > > > > > link > > > > > > > > >>> in > > > > > > > > >>> > > > > >>> previous > > > > > > > > >>> > > > > >>>>>>>>> email). > > > > > > > > >>> > > > > >>>>>>>>>>>> The > > > > > > > > >>> > > > > >>>>>>>>>>>> more I work with it, the more I > think > > I > > > > can > > > > > > > > >>> abstract it > > > > > > > > >>> > > > > >>>>>> further from > > > > > > > > >>> > > > > >>>>>>>>>> their > > > > > > > > >>> > > > > >>>>>>>>>>>> JDBC-like API but I started with a > > bunch > > > > of > > > > > > > delegate > > > > > > > > >>> > > > > >>> classes > > > > > > > > >>> > > > > >>>>>> then I > > > > > > > > >>> > > > > >>>>>>>>>> figure > > > > > > > > >>> > > > > >>>>>>>>>>>> I'll see where I can consolidate to > > more > > > > > > > abstract > > > > > > > > >>> > > > > >>> concepts. If > > > > > > > > >>> > > > > >>>>>> I > > > > > > > > >>> > > > > >>>>>>>> don't > > > > > > > > >>> > > > > >>>>>>>>>>>> have > > > > > > > > >>> > > > > >>>>>>>>>>>> to support Cassandra 3 with the new > > API, > > > > so > > > > > > > much the > > > > > > > > >>> > > > > >>> better. > > > > > > > > >>> > > > > >>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>> Regards, > > > > > > > > >>> > > > > >>>>>>>>>>>> Matt > > > > > > > > >>> > > > > >>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>> On Mon, Mar 18, 2024 at 4:14 PM > David > > > > > > > Handermann < > > > > > > > > >>> > > > > >>>>>>>>>>>> exceptionfact...@apache.org> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> Matt et al, > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> It is good to see the background > > effort > > > > on > > > > > > > moving > > > > > > > > >>> > > > > >>> Cassandra > > > > > > > > >>> > > > > >>>>>>>>>>>>> capabilities in a supportable > > > direction. > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> I think new Cassandra components > will > > > > > > require a > > > > > > > > >>> > > > > >>> significant > > > > > > > > >>> > > > > >>>>>>>>> departure > > > > > > > > >>> > > > > >>>>>>>>>>>>> from current Controller Service > > > > > abstractions. > > > > > > > Right > > > > > > > > >>> > > > > >> now, > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>>>>>> existing > > > > > > > > >>> > > > > >>>>>>>>>>>>> service interface does not provide > a > > > > clean > > > > > > > > >>> abstraction > > > > > > > > >>> > > > > >>> from > > > > > > > > >>> > > > > >>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>> Cassandra library, which is part of > > the > > > > > > reason > > > > > > > for > > > > > > > > >>> the > > > > > > > > >>> > > > > >>>>>> current > > > > > > > > >>> > > > > >>>>>>>>>>>>> coupling to the legacy driver > > version. > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> Following up from Joe's comments, > it > > > > seems > > > > > > > like the > > > > > > > > >>> > > > > >>> cleanest > > > > > > > > >>> > > > > >>>>>> way > > > > > > > > >>> > > > > >>>>>>>>>>>>> forward is to deprecate the current > > > > bundle > > > > > on > > > > > > > the > > > > > > > > >>> 1.x > > > > > > > > >>> > > > > >>>>>> branch, and > > > > > > > > >>> > > > > >>>>>>>>>>>>> remove the current bundle from the > > main > > > > > > branch. > > > > > > > > >>> That > > > > > > > > >>> > > > > >> will > > > > > > > > >>> > > > > >>>>>> provide > > > > > > > > >>> > > > > >>>>>>>> a > > > > > > > > >>> > > > > >>>>>>>>>>>>> clean slate for new Service and > > > Processor > > > > > > > > >>> > > > > >>> implementations, > > > > > > > > >>> > > > > >>>>>> without > > > > > > > > >>> > > > > >>>>>>>>>>>>> concern for uncertain compatibility > > > > > > questions. > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> Regards, > > > > > > > > >>> > > > > >>>>>>>>>>>>> David Handermann > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> On Mon, Mar 18, 2024 at 2:35 PM > Matt > > > > > Burgess > > > > > > < > > > > > > > > >>> > > > > >>>>>>>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>> What do y'all think about removing > > the > > > > > > > individual > > > > > > > > >>> > > > > >>>>>> connection > > > > > > > > >>> > > > > >>>>>>>>>>>> properties > > > > > > > > >>> > > > > >>>>>>>>>>>>>> from the Cassandra processors for > > NiFi > > > > 2.0 > > > > > > and > > > > > > > > >>> > > > > >>> requiring a > > > > > > > > >>> > > > > >>>>>>>>>>>>>> CassandraSessionProvider instead? > I > > > > think > > > > > we > > > > > > > > >>> started > > > > > > > > >>> > > > > >>> doing > > > > > > > > >>> > > > > >>>>>> that > > > > > > > > >>> > > > > >>>>>>>>>>>> elsewhere > > > > > > > > >>> > > > > >>>>>>>>>>>>>> (Elasticsearch maybe?), I noticed > > > > > duplicate > > > > > > > code > > > > > > > > >>> in > > > > > > > > >>> > > > > >> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>> CassandraSessionProvider and > > > > > > > > >>> > > > > >>> AbstractCassandraProcessor, > > > > > > > > >>> > > > > >>>>>> if we > > > > > > > > >>> > > > > >>>>>>>>> keep > > > > > > > > >>> > > > > >>>>>>>>>>>> those > > > > > > > > >>> > > > > >>>>>>>>>>>>>> properties I can refactor them > into > > a > > > > > > utility > > > > > > > > >>> class. > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>> Thanks, > > > > > > > > >>> > > > > >>>>>>>>>>>>>> Matt > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:44 PM > > Steven > > > > > > > Matison < > > > > > > > > >>> > > > > >>>>>>>>>>>> steven.mati...@gmail.com > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> I got through quite a bit of work > > to > > > > > enable > > > > > > > 4.x… > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> The 3.x pieces that were not > > > backwards > > > > > > > compatible > > > > > > > > >>> > > > > >> is > > > > > > > > >>> > > > > >>>>>> very edge > > > > > > > > >>> > > > > >>>>>>>>> use > > > > > > > > >>> > > > > >>>>>>>>>>>>> case and > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> could have been done slightly > > > > differently > > > > > > but > > > > > > > > >>> with > > > > > > > > >>> > > > > >>> work > > > > > > > > >>> > > > > >>>>>>>> around. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>> > > > > > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:30 PM > > Matt > > > > > > Burgess > > > > > > > < > > > > > > > > >>> > > > > >>>>>>>>>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> Oops used the wrong email > address > > so > > > > if > > > > > > > there > > > > > > > > >>> > > > > >> have > > > > > > > > >>> > > > > >>> been > > > > > > > > >>> > > > > >>>>>>>>>> responses > > > > > > > > >>> > > > > >>>>>>>>>>>> to > > > > > > > > >>> > > > > >>>>>>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> Cassandra thread since mine I > > missed > > > > > them, > > > > > > > my > > > > > > > > >>> > > > > >> bad! > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 2:00 PM > > Matt > > > > > > > Burgess < > > > > > > > > >>> > > > > >>>>>>>>>> mattyb...@gmail.com > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> I believe the CQL protocol is > > > > backwards > > > > > > > > >>> > > > > >>> compatible > > > > > > > > >>> > > > > >>>>>> but the > > > > > > > > >>> > > > > >>>>>>>>>> Java > > > > > > > > >>> > > > > >>>>>>>>>>>>> API is > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> not. For example > > > > > > > > >>> > > > > >>> "com.datastax.driver.core.Session" > > > > > > > > >>> > > > > >>>>>> is now > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>> > > > "com.datastax.oss.driver.api.core.session.Session" > > > > > > > > >>> > > > > >>>>>> and > > > > > > > > >>> > > > > >>>>>>>> there > > > > > > > > >>> > > > > >>>>>>>>>> is > > > > > > > > >>> > > > > >>>>>>>>>>>> no > > > > > > > > >>> > > > > >>>>>>>>>>>>> more > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> "Cluster" class. Might be > fairly > > > > > trivial > > > > > > > to fix > > > > > > > > >>> > > > > >>>>>> though, if > > > > > > > > >>> > > > > >>>>>>>>>>>> that's > > > > > > > > >>> > > > > >>>>>>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> path > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> of least resistance. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at 1:40 PM > > Joe > > > > > Witt > > > > > > < > > > > > > > > >>> > > > > >>>>>>>>> joe.w...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> Matt > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> I dont know a ton about > > Cassandra > > > > but > > > > > > > when I > > > > > > > > >>> > > > > >>> looked > > > > > > > > >>> > > > > >>>>>> at > > > > > > > > >>> > > > > >>>>>>>>>>>>> client/driver > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> notes > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> for 4+ it said it was > compatible > > > all > > > > > the > > > > > > > way > > > > > > > > >>> > > > > >>> back > > > > > > > > >>> > > > > >>>>>> to 3.x. > > > > > > > > >>> > > > > >>>>>>>>>> Not > > > > > > > > >>> > > > > >>>>>>>>>>>>> sure > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> what > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> that means but it surely seems > > > worth > > > > > > > > >>> > > > > >> exploring. > > > > > > > > >>> > > > > >>>>>> Also I > > > > > > > > >>> > > > > >>>>>>>>> dont > > > > > > > > >>> > > > > >>>>>>>>>>>> know > > > > > > > > >>> > > > > >>>>>>>>>>>>> if > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> 4.x drivers get rid of the > > > > vulnerable > > > > > > > bits. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> Thanks > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at > 10:39 AM > > > > Matt > > > > > > > Burgess > > > > > > > > >>> > > > > >> < > > > > > > > > >>> > > > > >>>>>>>>>>>>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> At the very least we should > > > upgrade > > > > > to > > > > > > > > >>> > > > > >>> Cassandra > > > > > > > > >>> > > > > >>>>>>>> 3.11.6: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > > > https://github.com/apache/cassandra/blob/cassandra-3.11.16/CHANGES.txt > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at > 1:31 PM > > > > Matt > > > > > > > > >>> > > > > >> Burgess < > > > > > > > > >>> > > > > >>>>>>>>>>>>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> If the community agrees to > get > > > rid > > > > > of > > > > > > > > >>> > > > > >>> Cassandra > > > > > > > > >>> > > > > >>>>>> 3 > > > > > > > > >>> > > > > >>>>>>>>> that'll > > > > > > > > >>> > > > > >>>>>>>>>>>>> save me > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> effort > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> on the refactor after I add > > > > > Cassandra > > > > > > 4 > > > > > > > :) > > > > > > > > >>> > > > > >>>>>> Otherwise > > > > > > > > >>> > > > > >>>>>>>>>> those > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> vulnerabilities would only > be > > > in a > > > > > > "new" > > > > > > > > >>> > > > > >>>>>> Cassandra 3 > > > > > > > > >>> > > > > >>>>>>>>>>>> services > > > > > > > > >>> > > > > >>>>>>>>>>>>> NAR > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> that > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> would not be included in the > > > > > > convenience > > > > > > > > >>> > > > > >>> binary. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at > > 1:28 PM > > > > Joe > > > > > > > Witt < > > > > > > > > >>> > > > > >>>>>>>>>>>> joe.w...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Mike, Matt, > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Happy to hear you both have > > > > active > > > > > > > > >>> > > > > >> efforts > > > > > > > > >>> > > > > >>> or > > > > > > > > >>> > > > > >>>>>> are > > > > > > > > >>> > > > > >>>>>>>>>>>> interested > > > > > > > > >>> > > > > >>>>>>>>>>>>> in > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> doing > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> so. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Can you help me understand > > more > > > > > > > > >>> > > > > >>> specifically > > > > > > > > >>> > > > > >>>>>> what > > > > > > > > >>> > > > > >>>>>>>> that > > > > > > > > >>> > > > > >>>>>>>>>>>> means > > > > > > > > >>> > > > > >>>>>>>>>>>>> for > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> current set of components? > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> The CVE hits are concerning > > and > > > > > long > > > > > > > > >>> > > > > >>> standing. > > > > > > > > >>> > > > > >>>>>>>>>> Supporting > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> Cassandra > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> 3 > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> implies the current set of > > > > > > dependencies > > > > > > > > >>> > > > > >>> would > > > > > > > > >>> > > > > >>>>>> remain > > > > > > > > >>> > > > > >>>>>>>>> too > > > > > > > > >>> > > > > >>>>>>>>>>>>> right? > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Is the current set of > > > components > > > > we > > > > > > > have > > > > > > > > >>> > > > > >>> ones > > > > > > > > >>> > > > > >>>>>> we > > > > > > > > >>> > > > > >>>>>>>> want > > > > > > > > >>> > > > > >>>>>>>>> to > > > > > > > > >>> > > > > >>>>>>>>>>>>> retain? > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> We > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> certainly need Cassandra > > > > > components - > > > > > > > but > > > > > > > > >>> > > > > >>> are > > > > > > > > >>> > > > > >>>>>> the > > > > > > > > >>> > > > > >>>>>>>> ones > > > > > > > > >>> > > > > >>>>>>>>>> we > > > > > > > > >>> > > > > >>>>>>>>>>>>> have > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> now > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> right ones? > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Thanks > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> Joe > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at > > > 10:25 AM > > > > > Matt > > > > > > > > >>> > > > > >>> Burgess < > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> mattyb...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> I'm actively working > this, I > > > > > pushed > > > > > > my > > > > > > > > >>> > > > > >>>>>> branch up > > > > > > > > >>> > > > > >>>>>>>> in > > > > > > > > >>> > > > > >>>>>>>>>> case > > > > > > > > >>> > > > > >>>>>>>>>>>>> anyone > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> wants > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> to > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> take a look [1]. The idea > is > > > to > > > > > > > > >>> > > > > >> abstract > > > > > > > > >>> > > > > >>> the > > > > > > > > >>> > > > > >>>>>>>>> Cassandra > > > > > > > > >>> > > > > >>>>>>>>>>>> API > > > > > > > > >>> > > > > >>>>>>>>>>>>> "up > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> a > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> couple > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> levels" and provide > > > > > implementations > > > > > > > for > > > > > > > > >>> > > > > >>>>>> Cassandra > > > > > > > > >>> > > > > >>>>>>>> 3, > > > > > > > > >>> > > > > >>>>>>>>>> 4, > > > > > > > > >>> > > > > >>>>>>>>>>>> and > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> eventually > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> 5. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> For JDBC-like interfaces > > this > > > > is a > > > > > > > PITA > > > > > > > > >>> > > > > >>>>>> because of > > > > > > > > >>> > > > > >>>>>>>>> the > > > > > > > > >>> > > > > >>>>>>>>>>>> API > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> (Statement, > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> PreparedStatement, > > > > BoundStatement, > > > > > > > > >>> > > > > >>> ResultSet, > > > > > > > > >>> > > > > >>>>>>>> etc.) > > > > > > > > >>> > > > > >>>>>>>>>> but > > > > > > > > >>> > > > > >>>>>>>>>>>> I'm > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> hoping > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> we > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> can > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> find a common pattern for > > > > > > abstracting > > > > > > > > >>> > > > > >> the > > > > > > > > >>> > > > > >>>>>>>>> third-party > > > > > > > > >>> > > > > >>>>>>>>>>>>> library > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> implementation and API > from > > > the > > > > > NiFi > > > > > > > > >>> > > > > >>>>>> component > > > > > > > > >>> > > > > >>>>>>>>>>>> (Processor, > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> ControllerService, etc.) > > API. > > > I > > > > > > think > > > > > > > > >>> > > > > >>> we're > > > > > > > > >>> > > > > >>>>>> doing > > > > > > > > >>> > > > > >>>>>>>>>>>> something > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> similar > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> for > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> Kafka? > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> Regards, > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> Matt > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> [1] > > > > > > > > >>> > > > > >>>>>> > > https://github.com/mattyb149/nifi/tree/cassy4 > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 15, 2024 at > > > 8:43 AM > > > > > Mike > > > > > > > > >>> > > > > >>> Thomsen > > > > > > > > >>> > > > > >>>>>> < > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> mikerthom...@gmail.com> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> That’s been on my todo > list > > > > for a > > > > > > > > >>> > > > > >>> little > > > > > > > > >>> > > > > >>>>>> while > > > > > > > > >>> > > > > >>>>>>>> but > > > > > > > > >>> > > > > >>>>>>>>>>>> things > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> kept > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> coming > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> up. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> I think I could get > started > > > on > > > > > that > > > > > > > > >>> > > > > >>> now. > > > > > > > > >>> > > > > >>>>>> Based > > > > > > > > >>> > > > > >>>>>>>> on > > > > > > > > >>> > > > > >>>>>>>>> my > > > > > > > > >>> > > > > >>>>>>>>>>>>> initial > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> research > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> it > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> appears that scylla uses > > the > > > > > exact > > > > > > > > >>> > > > > >> same > > > > > > > > >>> > > > > >>>>>> api as > > > > > > > > >>> > > > > >>>>>>>>>>>> datastax > > > > > > > > >>> > > > > >>>>>>>>>>>>> so > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> supporting > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> both > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> in a cql bundle should > > > > > > theoretically > > > > > > > > >>> > > > > >> be > > > > > > > > >>> > > > > >>>>>> fairly > > > > > > > > >>> > > > > >>>>>>>>> easy. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> Sent from my iPhone > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> On Mar 14, 2024, at > > 6:18 PM, > > > > Joe > > > > > > > > >>> > > > > >>> Witt < > > > > > > > > >>> > > > > >>>>>>>>>>>>> joew...@apache.org> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> wrote: > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Team, > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Cassandra remains a > really > > > > > > > > >>> > > > > >> important > > > > > > > > >>> > > > > >>>>>> system to > > > > > > > > >>> > > > > >>>>>>>>> be > > > > > > > > >>> > > > > >>>>>>>>>>>> able > > > > > > > > >>> > > > > >>>>>>>>>>>>> to > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> send > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> data > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> to. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> However, it seems like > > we've > > > > not > > > > > > > > >>> > > > > >>>>>> maintained > > > > > > > > >>> > > > > >>>>>>>>> these > > > > > > > > >>> > > > > >>>>>>>>>>>>> well. We > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> have > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> what > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> appears to be at least a > > > full > > > > > > > > >>> > > > > >>> generation > > > > > > > > >>> > > > > >>>>>>>> behind > > > > > > > > >>> > > > > >>>>>>>>> on > > > > > > > > >>> > > > > >>>>>>>>>>>>> client > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> versions > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> (we > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> are > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> on 3x vs 4x which is the > > > > latest > > > > > > > > >>> > > > > >>> stable > > > > > > > > >>> > > > > >>>>>> with 5x > > > > > > > > >>> > > > > >>>>>>>>>>>>> apparently > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> coming > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> shortly). > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> We have components to > send > > > > data, > > > > > > > > >>> > > > > >>> query > > > > > > > > >>> > > > > >>>>>> data, > > > > > > > > >>> > > > > >>>>>>>> and > > > > > > > > >>> > > > > >>>>>>>>>> use > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> Cassandra > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> as > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> a > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> cache > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> store. We have older > > > > mechanisms > > > > > > > > >>> > > > > >> for > > > > > > > > >>> > > > > >>>>>> json/avro > > > > > > > > >>> > > > > >>>>>>>>> and > > > > > > > > >>> > > > > >>>>>>>>>>>>> publish > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> mechanisms > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> for > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> records. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> The libraries we do have > > > > depend > > > > > on > > > > > > > > >>> > > > > >>>>>> outdated > > > > > > > > >>> > > > > >>>>>>>>>>>> versions of > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> Guava > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> and > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> result > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> in > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> many CVE hits. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> I am inclined to think > we > > > > should > > > > > > > > >>> > > > > >>>>>> deprecate the > > > > > > > > >>> > > > > >>>>>>>>> 1.x > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> components > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> and > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> remove > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> them as-is from the 2.x > > > line. > > > > > > Then > > > > > > > > >>> > > > > >>>>>>>> re-introduce > > > > > > > > >>> > > > > >>>>>>>>>>>> them > > > > > > > > >>> > > > > >>>>>>>>>>>>> with > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> record > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> only > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> interfaces and built > > against > > > > the > > > > > > > > >>> > > > > >>> latest > > > > > > > > >>> > > > > >>>>>> stable > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> > Cassandra/Datastax/ScyllaDB > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> interfaces. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> I'd love to hear > thoughts > > > from > > > > > > > > >>> > > > > >> those > > > > > > > > >>> > > > > >>>>>> closer to > > > > > > > > >>> > > > > >>>>>>>>>> this > > > > > > > > >>> > > > > >>>>>>>>>>>>> space > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> both > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> as > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> a > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> user > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> and developer so we can > > make > > > > > good > > > > > > > > >>> > > > > >>> next > > > > > > > > >>> > > > > >>>>>> steps. > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Thanks > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>>> > > > > > > > > >>> > > > > >>>>>>>> > > > > > > > > >>> > > > > >>>>>> > > > > > > > > >>> > > > > >>>>> > > > > > > > > >>> > > > > >>> > > > > > > > > >>> > > > > >> > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
