Re: [discuss] What to do with the Cassandra components
A cursory look at the Cassandra 5 stuff didn’t indicate any incompatibility. So yeah, I think we are likely pretty safe to use the 4.17 driver Sent from my iPhone > On Mar 19, 2024, at 3:35 PM, Matt Burgess wrote: > > Is it likely now (due to the refactor) that we will simply be able to > upgrade the driver when Cassandra 5 is GA? Also does anyone use Netflix's > Astyanax [1]? > > [1] > https://cassandra.apache.org/doc/stable/cassandra/getting_started/drivers.html#java > >> On Tue, Mar 19, 2024 at 3:10 PM Mike Thomsen wrote: >> >> Realistically, I think we are only likely to see two drivers: >> >> * DataStax >> * ScyllaDB >> >> The latter makes a selling point of being a binary compatible, drop-in >> replacement for the former. >> >> That's why I don't see a need to have an abstraction layer per se. I think >> we only need "DataStaxConnectionProviderImpl" and >> "ScyllaDBConnectionProviderImpl" with the difference being which jar is >> imported by maven. >> >> On Tue, Mar 19, 2024 at 2:59 PM David Handermann < >> exceptionfact...@apache.org> wrote: >> >>> Mike, >>> >>> Thanks for the reply and clarification. >>> >>> I agree there is no need to maintain support for the DataStax 3 driver >>> and Java API, any new components should be built on the latest version >>> of the driver. >>> >>> What we do need going forward is to avoid, if at all possible, having >>> a DataStax 4 dependency in the Controller Service API. >>> >>> One example of this is the WebClientServiceProvider interface. That >>> Controller Service API does not have any third-party dependencies. The >>> Controller Service implementation, StandardWebClientServiceProvider, >>> has a dependency on OkHttp to implement HTTP communication. That is >>> the kind of abstraction that would be ideal, and I believe that also >>> aligns with what Matt has described. >>> >>> Regards, >>> David Handermann >>> >>> On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen >>> wrote: ** we can dump v3 **DRIVER** compatibility, since later 4.X Java >> drivers are backward compatible with Cassandra 3 On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen >>> wrote: > David, > > Before we proceed, I think we should make sure we're all >> understanding >>> the > same problem here. Starting with this: > >> I believe the CQL protocol is backwards compatible but the Java API >>> is > not. >> For example "com.datastax.driver.core.Session" is now >> "com.datastax.oss.driver.api.core.session.Session" and there is no >>> more >> "Cluster" class. Might be fairly trivial to fix though, if that's >> the > path >> of least resistance. > > From what I've learned using Cassandra 3 and 4 in my day job and >>> reading > up on this stuff for the sake of discussion, that all tracks. We used >>> the > ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue >>> during > an upgrade. So I don't see any reason to factor in the "changes from > DataStax 3 to 4" since the changes were likely a one-off decision >>> meant to > position the driver for better future support and stability. > > TL;DR, we can dump v3 compatibility and the only thing our users will > notice is if we make the controller service totally incompatible with >>> the > one they're already using which is something we can actively avoid. > > On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > exceptionfact...@apache.org> wrote: > >> All, >> >> I support a Controller Service API abstraction around the Cassandra >> Driver. The changes from DataStax 3 to 4 already highlight the need >> for that abstraction. The donation of the DataStax Java driver to >> Apache [1] also shows the value of providing some level of >> isolation, >> if at all possible. >> >> I have not taken a close look at the Matt's branch, and the details >> of >> the abstraction are important, but having the abstraction can be >> useful to avoid getting back to this same situation. >> >> Regards, >> David Handermann >> >> [1] https://github.com/apache/cassandra-java-driver/ >> >> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen < >> mikerthom...@gmail.com >> wrote: >>> >>> Matt, >>> >>> I got that. My point was that the Java changes appear to be a one >>> time >>> thing that DataStax did to make a better driver with a much more >>> future-proof API. Since Scylla tracks them as closely as >> possible, I >>> suspect that we don't need to plan for a bunch of abstraction to >>> isolate >>> Java changes. >>> >>> On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < >> steven.mati...@gmail.com> >>> wrote: >>> That was kinda where i got stuck and fell out on my branch/jira. >> Mike and I wanted to make a new controller service , without backward >>
Fw: Java version for NiFi Registry
Hello,
we are trying to run NiFi Registry in Java 11. Receiving the following error:
Text:
>
D:\NiFi\nifi-registry-1.23.2\bin>run-nifi-registry.bat
cmd.exe /C "C:\Program Files\CleverDevices\Amazon Corretto\jdk11\bin\java.exe"
-cp
D:\NiFi\nifi-registry-1.23.2\bin\..\conf;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\*;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\shared\*;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\bootstrap\*
-Xms512m -Xmx1024m
-Dorg.apache.nifi.registry.bootstrap.config.file=D:\NiFi\nifi-registry-1.23.2\bin\..\conf\bootstrap.conf
-Dorg.apache.nifi.registry.bootstrap.config.log.dir=D:\NiFi\nifi-registry-1.23.2\bin\..\logs
org.apache.nifi.registry.NiFiRegistry run
Apache NiFi _ _
_ __ ___ __ _(_)___| |_ _ __ _ _
| '__/ _ \/ _` | / __| __| '__| | | |
| | | __/ (_| | \__ \ |_| | | |_| |
|_| \___|\__, |_|___/\__|_| \__, |
==|___/|___/=
v
Server start failed: java.lang.IllegalStateException: Unable to load cache item
D:\NiFi\nifi-registry-1.23.2\bin>
>>
The log contains the following:
2024-03-19 14:26:38,774 INFO [main] o.e.j.a.AnnotationConfiguration Scanning
elapsed time=33ms
2024-03-19 14:26:38,774 INFO [main] o.e.jetty.server.handler.ContextHandler
Started
o.e.j.w.WebAppContext@5170bc02{nifi-registry-docs,/nifi-registry-docs,file:///D:/NiFi/nifi-registry-1.23.2/work/jetty/nifi-registry-web-docs-1.23.2.war/webapp/,AVAILABLE}{.\lib\nifi-registry-web-docs-1.23.2.war}
2024-03-19 14:26:39,008 INFO [main] o.eclipse.jetty.server.AbstractConnector
Started ServerConnector@59cba5a{HTTP/1.1, (http/1.1)}{0.0.0.0:18080}
2024-03-19 14:26:39,008 INFO [main] org.eclipse.jetty.server.Server Started
@26817ms
2024-03-19 14:26:39,013 ERROR [main] o.apache.nifi.registry.jetty.JettyServer
Server start failed
java.lang.RuntimeException: java.lang.IllegalStateException: Unable to load
cache item
at
org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:69)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:356)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.nifi.registry.jetty.JettyServer.start(JettyServer.java:93)
at org.apache.nifi.registry.NiFiRegistry.(NiFiRegistry.java:110)
at org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:159)
Caused by: java.lang.IllegalStateException: Unable to load cache item
at
org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:79)
at
org.springframework.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
at
org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:134)
at
org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:319)
at
org.springframework.cglib.proxy.Enhancer.createHelper(Enhancer.java:572)
at org.springframework.cglib.proxy.Enhancer.createClass(Enhancer.java:419)
at
org.springframework.context.annotation.ConfigurationClassEnhancer.createClass(ConfigurationClassEnhancer.java:137)
Re: [discuss] What to do with the Cassandra components
Is it likely now (due to the refactor) that we will simply be able to upgrade the driver when Cassandra 5 is GA? Also does anyone use Netflix's Astyanax [1]? [1] https://cassandra.apache.org/doc/stable/cassandra/getting_started/drivers.html#java On Tue, Mar 19, 2024 at 3:10 PM Mike Thomsen wrote: > Realistically, I think we are only likely to see two drivers: > > * DataStax > * ScyllaDB > > The latter makes a selling point of being a binary compatible, drop-in > replacement for the former. > > That's why I don't see a need to have an abstraction layer per se. I think > we only need "DataStaxConnectionProviderImpl" and > "ScyllaDBConnectionProviderImpl" with the difference being which jar is > imported by maven. > > On Tue, Mar 19, 2024 at 2:59 PM David Handermann < > exceptionfact...@apache.org> wrote: > > > Mike, > > > > Thanks for the reply and clarification. > > > > I agree there is no need to maintain support for the DataStax 3 driver > > and Java API, any new components should be built on the latest version > > of the driver. > > > > What we do need going forward is to avoid, if at all possible, having > > a DataStax 4 dependency in the Controller Service API. > > > > One example of this is the WebClientServiceProvider interface. That > > Controller Service API does not have any third-party dependencies. The > > Controller Service implementation, StandardWebClientServiceProvider, > > has a dependency on OkHttp to implement HTTP communication. That is > > the kind of abstraction that would be ideal, and I believe that also > > aligns with what Matt has described. > > > > Regards, > > David Handermann > > > > On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen > > wrote: > > > > > > ** we can dump v3 **DRIVER** compatibility, since later 4.X Java > drivers > > > are backward compatible with Cassandra 3 > > > > > > On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen > > wrote: > > > > > > > David, > > > > > > > > Before we proceed, I think we should make sure we're all > understanding > > the > > > > same problem here. Starting with this: > > > > > > > > > I believe the CQL protocol is backwards compatible but the Java API > > is > > > > not. > > > > > For example "com.datastax.driver.core.Session" is now > > > > > "com.datastax.oss.driver.api.core.session.Session" and there is no > > more > > > > > "Cluster" class. Might be fairly trivial to fix though, if that's > the > > > > path > > > > > of least resistance. > > > > > > > > From what I've learned using Cassandra 3 and 4 in my day job and > > reading > > > > up on this stuff for the sake of discussion, that all tracks. We used > > the > > > > ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue > > during > > > > an upgrade. So I don't see any reason to factor in the "changes from > > > > DataStax 3 to 4" since the changes were likely a one-off decision > > meant to > > > > position the driver for better future support and stability. > > > > > > > > TL;DR, we can dump v3 compatibility and the only thing our users will > > > > notice is if we make the controller service totally incompatible with > > the > > > > one they're already using which is something we can actively avoid. > > > > > > > > On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > > > > exceptionfact...@apache.org> wrote: > > > > > > > >> All, > > > >> > > > >> I support a Controller Service API abstraction around the Cassandra > > > >> Driver. The changes from DataStax 3 to 4 already highlight the need > > > >> for that abstraction. The donation of the DataStax Java driver to > > > >> Apache [1] also shows the value of providing some level of > isolation, > > > >> if at all possible. > > > >> > > > >> I have not taken a close look at the Matt's branch, and the details > of > > > >> the abstraction are important, but having the abstraction can be > > > >> useful to avoid getting back to this same situation. > > > >> > > > >> Regards, > > > >> David Handermann > > > >> > > > >> [1] https://github.com/apache/cassandra-java-driver/ > > > >> > > > >> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen < > mikerthom...@gmail.com > > > > > > >> wrote: > > > >> > > > > >> > Matt, > > > >> > > > > >> > I got that. My point was that the Java changes appear to be a one > > time > > > >> > thing that DataStax did to make a better driver with a much more > > > >> > future-proof API. Since Scylla tracks them as closely as > possible, I > > > >> > suspect that we don't need to plan for a bunch of abstraction to > > isolate > > > >> > Java changes. > > > >> > > > > >> > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < > > > >> steven.mati...@gmail.com> > > > >> > wrote: > > > >> > > > > >> > > That was kinda where i got stuck and fell out on my branch/jira. > > > >> Mike and > > > >> > > I wanted to make a new controller service , without backward > > > >> compatibility; > > > >> > > and remove the duplicate driver/connection properties found in > > some > > > >> of the > > > >> > > processors. >
Re: [discuss] What to do with the Cassandra components
Realistically, I think we are only likely to see two drivers: * DataStax * ScyllaDB The latter makes a selling point of being a binary compatible, drop-in replacement for the former. That's why I don't see a need to have an abstraction layer per se. I think we only need "DataStaxConnectionProviderImpl" and "ScyllaDBConnectionProviderImpl" with the difference being which jar is imported by maven. On Tue, Mar 19, 2024 at 2:59 PM David Handermann < exceptionfact...@apache.org> wrote: > Mike, > > Thanks for the reply and clarification. > > I agree there is no need to maintain support for the DataStax 3 driver > and Java API, any new components should be built on the latest version > of the driver. > > What we do need going forward is to avoid, if at all possible, having > a DataStax 4 dependency in the Controller Service API. > > One example of this is the WebClientServiceProvider interface. That > Controller Service API does not have any third-party dependencies. The > Controller Service implementation, StandardWebClientServiceProvider, > has a dependency on OkHttp to implement HTTP communication. That is > the kind of abstraction that would be ideal, and I believe that also > aligns with what Matt has described. > > Regards, > David Handermann > > On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen > wrote: > > > > ** we can dump v3 **DRIVER** compatibility, since later 4.X Java drivers > > are backward compatible with Cassandra 3 > > > > On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen > wrote: > > > > > David, > > > > > > Before we proceed, I think we should make sure we're all understanding > the > > > same problem here. Starting with this: > > > > > > > I believe the CQL protocol is backwards compatible but the Java API > is > > > not. > > > > For example "com.datastax.driver.core.Session" is now > > > > "com.datastax.oss.driver.api.core.session.Session" and there is no > more > > > > "Cluster" class. Might be fairly trivial to fix though, if that's the > > > path > > > > of least resistance. > > > > > > From what I've learned using Cassandra 3 and 4 in my day job and > reading > > > up on this stuff for the sake of discussion, that all tracks. We used > the > > > ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue > during > > > an upgrade. So I don't see any reason to factor in the "changes from > > > DataStax 3 to 4" since the changes were likely a one-off decision > meant to > > > position the driver for better future support and stability. > > > > > > TL;DR, we can dump v3 compatibility and the only thing our users will > > > notice is if we make the controller service totally incompatible with > the > > > one they're already using which is something we can actively avoid. > > > > > > On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > > > exceptionfact...@apache.org> wrote: > > > > > >> All, > > >> > > >> I support a Controller Service API abstraction around the Cassandra > > >> Driver. The changes from DataStax 3 to 4 already highlight the need > > >> for that abstraction. The donation of the DataStax Java driver to > > >> Apache [1] also shows the value of providing some level of isolation, > > >> if at all possible. > > >> > > >> I have not taken a close look at the Matt's branch, and the details of > > >> the abstraction are important, but having the abstraction can be > > >> useful to avoid getting back to this same situation. > > >> > > >> Regards, > > >> David Handermann > > >> > > >> [1] https://github.com/apache/cassandra-java-driver/ > > >> > > >> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen > > > >> wrote: > > >> > > > >> > Matt, > > >> > > > >> > I got that. My point was that the Java changes appear to be a one > time > > >> > thing that DataStax did to make a better driver with a much more > > >> > future-proof API. Since Scylla tracks them as closely as possible, I > > >> > suspect that we don't need to plan for a bunch of abstraction to > isolate > > >> > Java changes. > > >> > > > >> > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < > > >> steven.mati...@gmail.com> > > >> > wrote: > > >> > > > >> > > That was kinda where i got stuck and fell out on my branch/jira. > > >> Mike and > > >> > > I wanted to make a new controller service , without backward > > >> compatibility; > > >> > > and remove the duplicate driver/connection properties found in > some > > >> of the > > >> > > processors. > > >> > > > > >> > > I agree taking out all old stuff and making new controller service > > >> makes > > >> > > most sense. 4.x and 5.x should be mostly backwards compatible to > > >> 2&3.x > > >> > > with how it’s used within current processors. > > >> > > > > >> > > > > >> > > > > >> > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess < > mattyb...@apache.org> > > >> > > wrote: > > >> > > > > >> > > > The abstraction is to isolate Java API changes, not protocol > > >> > > compatibility > > >> > > > Changing to the java-driver comes with a number of changes to > the > > >> code >
Re: [discuss] What to do with the Cassandra components
Mike, Thanks for the reply and clarification. I agree there is no need to maintain support for the DataStax 3 driver and Java API, any new components should be built on the latest version of the driver. What we do need going forward is to avoid, if at all possible, having a DataStax 4 dependency in the Controller Service API. One example of this is the WebClientServiceProvider interface. That Controller Service API does not have any third-party dependencies. The Controller Service implementation, StandardWebClientServiceProvider, has a dependency on OkHttp to implement HTTP communication. That is the kind of abstraction that would be ideal, and I believe that also aligns with what Matt has described. Regards, David Handermann On Tue, Mar 19, 2024 at 1:45 PM Mike Thomsen wrote: > > ** we can dump v3 **DRIVER** compatibility, since later 4.X Java drivers > are backward compatible with Cassandra 3 > > On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen wrote: > > > David, > > > > Before we proceed, I think we should make sure we're all understanding the > > same problem here. Starting with this: > > > > > I believe the CQL protocol is backwards compatible but the Java API is > > not. > > > For example "com.datastax.driver.core.Session" is now > > > "com.datastax.oss.driver.api.core.session.Session" and there is no more > > > "Cluster" class. Might be fairly trivial to fix though, if that's the > > path > > > of least resistance. > > > > From what I've learned using Cassandra 3 and 4 in my day job and reading > > up on this stuff for the sake of discussion, that all tracks. We used the > > ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue during > > an upgrade. So I don't see any reason to factor in the "changes from > > DataStax 3 to 4" since the changes were likely a one-off decision meant to > > position the driver for better future support and stability. > > > > TL;DR, we can dump v3 compatibility and the only thing our users will > > notice is if we make the controller service totally incompatible with the > > one they're already using which is something we can actively avoid. > > > > On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > > exceptionfact...@apache.org> wrote: > > > >> All, > >> > >> I support a Controller Service API abstraction around the Cassandra > >> Driver. The changes from DataStax 3 to 4 already highlight the need > >> for that abstraction. The donation of the DataStax Java driver to > >> Apache [1] also shows the value of providing some level of isolation, > >> if at all possible. > >> > >> I have not taken a close look at the Matt's branch, and the details of > >> the abstraction are important, but having the abstraction can be > >> useful to avoid getting back to this same situation. > >> > >> Regards, > >> David Handermann > >> > >> [1] https://github.com/apache/cassandra-java-driver/ > >> > >> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen > >> wrote: > >> > > >> > Matt, > >> > > >> > I got that. My point was that the Java changes appear to be a one time > >> > thing that DataStax did to make a better driver with a much more > >> > future-proof API. Since Scylla tracks them as closely as possible, I > >> > suspect that we don't need to plan for a bunch of abstraction to isolate > >> > Java changes. > >> > > >> > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < > >> steven.mati...@gmail.com> > >> > wrote: > >> > > >> > > That was kinda where i got stuck and fell out on my branch/jira. > >> Mike and > >> > > I wanted to make a new controller service , without backward > >> compatibility; > >> > > and remove the duplicate driver/connection properties found in some > >> of the > >> > > processors. > >> > > > >> > > I agree taking out all old stuff and making new controller service > >> makes > >> > > most sense. 4.x and 5.x should be mostly backwards compatible to > >> 2&3.x > >> > > with how it’s used within current processors. > >> > > > >> > > > >> > > > >> > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess > >> > > wrote: > >> > > > >> > > > The abstraction is to isolate Java API changes, not protocol > >> > > compatibility > >> > > > Changing to the java-driver comes with a number of changes to the > >> code > >> > > (see > >> > > > Steven's and my branches), if we can abstract that API it should > >> lead to > >> > > > more maintainable code in the future by not having to change any > >> > > > processors, just the controller service implementation. > >> > > > > >> > > > > >> > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen < > >> mikerthom...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > >> > > > > > >> > > > > Directly quoting Scylla docs here: > >> > > > > > >> > > > > > The Scylla Java Driver is a drop-in replacement for the > >> DataStax Java > >> > > > > Driver. As such, no code changes are needed to
Re: [discuss] What to do with the Cassandra components
** we can dump v3 **DRIVER** compatibility, since later 4.X Java drivers are backward compatible with Cassandra 3 On Tue, Mar 19, 2024 at 2:43 PM Mike Thomsen wrote: > David, > > Before we proceed, I think we should make sure we're all understanding the > same problem here. Starting with this: > > > I believe the CQL protocol is backwards compatible but the Java API is > not. > > For example "com.datastax.driver.core.Session" is now > > "com.datastax.oss.driver.api.core.session.Session" and there is no more > > "Cluster" class. Might be fairly trivial to fix though, if that's the > path > > of least resistance. > > From what I've learned using Cassandra 3 and 4 in my day job and reading > up on this stuff for the sake of discussion, that all tracks. We used the > ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue during > an upgrade. So I don't see any reason to factor in the "changes from > DataStax 3 to 4" since the changes were likely a one-off decision meant to > position the driver for better future support and stability. > > TL;DR, we can dump v3 compatibility and the only thing our users will > notice is if we make the controller service totally incompatible with the > one they're already using which is something we can actively avoid. > > On Tue, Mar 19, 2024 at 2:00 PM David Handermann < > exceptionfact...@apache.org> wrote: > >> All, >> >> I support a Controller Service API abstraction around the Cassandra >> Driver. The changes from DataStax 3 to 4 already highlight the need >> for that abstraction. The donation of the DataStax Java driver to >> Apache [1] also shows the value of providing some level of isolation, >> if at all possible. >> >> I have not taken a close look at the Matt's branch, and the details of >> the abstraction are important, but having the abstraction can be >> useful to avoid getting back to this same situation. >> >> Regards, >> David Handermann >> >> [1] https://github.com/apache/cassandra-java-driver/ >> >> On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen >> wrote: >> > >> > Matt, >> > >> > I got that. My point was that the Java changes appear to be a one time >> > thing that DataStax did to make a better driver with a much more >> > future-proof API. Since Scylla tracks them as closely as possible, I >> > suspect that we don't need to plan for a bunch of abstraction to isolate >> > Java changes. >> > >> > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < >> steven.mati...@gmail.com> >> > wrote: >> > >> > > That was kinda where i got stuck and fell out on my branch/jira. >> Mike and >> > > I wanted to make a new controller service , without backward >> compatibility; >> > > and remove the duplicate driver/connection properties found in some >> of the >> > > processors. >> > > >> > > I agree taking out all old stuff and making new controller service >> makes >> > > most sense. 4.x and 5.x should be mostly backwards compatible to >> 2&3.x >> > > with how it’s used within current processors. >> > > >> > > >> > > >> > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess >> > > wrote: >> > > >> > > > The abstraction is to isolate Java API changes, not protocol >> > > compatibility >> > > > Changing to the java-driver comes with a number of changes to the >> code >> > > (see >> > > > Steven's and my branches), if we can abstract that API it should >> lead to >> > > > more maintainable code in the future by not having to change any >> > > > processors, just the controller service implementation. >> > > > >> > > > >> > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen < >> mikerthom...@gmail.com> >> > > > wrote: >> > > > >> > > > > >> > > > > >> > > > >> > > >> https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html >> > > > > >> > > > > Directly quoting Scylla docs here: >> > > > > >> > > > > > The Scylla Java Driver is a drop-in replacement for the >> DataStax Java >> > > > > Driver. As such, no code changes are needed to use this driver. >> > > > > >> > > > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen < >> mikerthom...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Matt, >> > > > > > >> > > > > > I don't think we need to really "abstract above" the drivers >> because >> > > > the >> > > > > > Java DataStax driver appears to support 4.X all the way back to >> 2.X, >> > > as >> > > > > > well as the enterprise versions from DataStax >> > > > > > >> > > > > > >> https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html >> > > > > > >> > > > > > Similar situation with Scylla. When I looked at the driver, it >> > > appeared >> > > > > to >> > > > > > copy verbatim the entire public API of that driver. So I think >> before >> > > > we >> > > > > > dive into abstractions, it's worth doing a bit more validation >> of >> > > these >> > > > > > details. IMHO, this might be a much lighter lift than >> anticipated. >> > > > > > >> > > > > > >> > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess < >>
Re: [discuss] What to do with the Cassandra components
David, Before we proceed, I think we should make sure we're all understanding the same problem here. Starting with this: > I believe the CQL protocol is backwards compatible but the Java API is not. > For example "com.datastax.driver.core.Session" is now > "com.datastax.oss.driver.api.core.session.Session" and there is no more > "Cluster" class. Might be fairly trivial to fix though, if that's the path > of least resistance. >From what I've learned using Cassandra 3 and 4 in my day job and reading up on this stuff for the sake of discussion, that all tracks. We used the ~4.11 driver in Spring Boot on both v3 and v4 clusters without issue during an upgrade. So I don't see any reason to factor in the "changes from DataStax 3 to 4" since the changes were likely a one-off decision meant to position the driver for better future support and stability. TL;DR, we can dump v3 compatibility and the only thing our users will notice is if we make the controller service totally incompatible with the one they're already using which is something we can actively avoid. On Tue, Mar 19, 2024 at 2:00 PM David Handermann < exceptionfact...@apache.org> wrote: > All, > > I support a Controller Service API abstraction around the Cassandra > Driver. The changes from DataStax 3 to 4 already highlight the need > for that abstraction. The donation of the DataStax Java driver to > Apache [1] also shows the value of providing some level of isolation, > if at all possible. > > I have not taken a close look at the Matt's branch, and the details of > the abstraction are important, but having the abstraction can be > useful to avoid getting back to this same situation. > > Regards, > David Handermann > > [1] https://github.com/apache/cassandra-java-driver/ > > On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen > wrote: > > > > Matt, > > > > I got that. My point was that the Java changes appear to be a one time > > thing that DataStax did to make a better driver with a much more > > future-proof API. Since Scylla tracks them as closely as possible, I > > suspect that we don't need to plan for a bunch of abstraction to isolate > > Java changes. > > > > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison < > steven.mati...@gmail.com> > > wrote: > > > > > That was kinda where i got stuck and fell out on my branch/jira. Mike > and > > > I wanted to make a new controller service , without backward > compatibility; > > > and remove the duplicate driver/connection properties found in some of > the > > > processors. > > > > > > I agree taking out all old stuff and making new controller service > makes > > > most sense. 4.x and 5.x should be mostly backwards compatible to 2&3.x > > > with how it’s used within current processors. > > > > > > > > > > > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess > > > wrote: > > > > > > > The abstraction is to isolate Java API changes, not protocol > > > compatibility > > > > Changing to the java-driver comes with a number of changes to the > code > > > (see > > > > Steven's and my branches), if we can abstract that API it should > lead to > > > > more maintainable code in the future by not having to change any > > > > processors, just the controller service implementation. > > > > > > > > > > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen < > mikerthom...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > > > > > > > Directly quoting Scylla docs here: > > > > > > > > > > > The Scylla Java Driver is a drop-in replacement for the DataStax > Java > > > > > Driver. As such, no code changes are needed to use this driver. > > > > > > > > > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen < > mikerthom...@gmail.com> > > > > > wrote: > > > > > > > > > > > Matt, > > > > > > > > > > > > I don't think we need to really "abstract above" the drivers > because > > > > the > > > > > > Java DataStax driver appears to support 4.X all the way back to > 2.X, > > > as > > > > > > well as the enterprise versions from DataStax > > > > > > > > > > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > > > > > > > > Similar situation with Scylla. When I looked at the driver, it > > > appeared > > > > > to > > > > > > copy verbatim the entire public API of that driver. So I think > before > > > > we > > > > > > dive into abstractions, it's worth doing a bit more validation of > > > these > > > > > > details. IMHO, this might be a much lighter lift than > anticipated. > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess < > mattyb...@gmail.com> > > > > > wrote: > > > > > > > > > > > >> Totally agree, that's what my branch does (see link in previous > > > > email). > > > > > >> The > > > > > >> more I work with it, the more I think I can abstract it further > from > > > > > their > > > > > >> JDBC-like API but I started with a bunch of delegate classes >
Java version for NiFi Registry
Hello,
we are trying to run NiFi Registry in Java 11. Receiving the following error:
[cid:373db5b6-713b-443b-8aa3-0aa896934bf5]
Text:
>
D:\NiFi\nifi-registry-1.23.2\bin>run-nifi-registry.bat
cmd.exe /C "C:\Program Files\CleverDevices\Amazon Corretto\jdk11\bin\java.exe"
-cp
D:\NiFi\nifi-registry-1.23.2\bin\..\conf;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\*;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\shared\*;D:\NiFi\nifi-registry-1.23.2\bin\..\lib\bootstrap\*
-Xms512m -Xmx1024m
-Dorg.apache.nifi.registry.bootstrap.config.file=D:\NiFi\nifi-registry-1.23.2\bin\..\conf\bootstrap.conf
-Dorg.apache.nifi.registry.bootstrap.config.log.dir=D:\NiFi\nifi-registry-1.23.2\bin\..\logs
org.apache.nifi.registry.NiFiRegistry run
Apache NiFi _ _
_ __ ___ __ _(_)___| |_ _ __ _ _
| '__/ _ \/ _` | / __| __| '__| | | |
| | | __/ (_| | \__ \ |_| | | |_| |
|_| \___|\__, |_|___/\__|_| \__, |
==|___/|___/=
v
Server start failed: java.lang.IllegalStateException: Unable to load cache item
D:\NiFi\nifi-registry-1.23.2\bin>
>>
The log contains the following:
2024-03-19 14:26:38,774 INFO [main] o.e.j.a.AnnotationConfiguration Scanning
elapsed time=33ms
2024-03-19 14:26:38,774 INFO [main] o.e.jetty.server.handler.ContextHandler
Started
o.e.j.w.WebAppContext@5170bc02{nifi-registry-docs,/nifi-registry-docs,file:///D:/NiFi/nifi-registry-1.23.2/work/jetty/nifi-registry-web-docs-1.23.2.war/webapp/,AVAILABLE}{.\lib\nifi-registry-web-docs-1.23.2.war}
2024-03-19 14:26:39,008 INFO [main] o.eclipse.jetty.server.AbstractConnector
Started ServerConnector@59cba5a{HTTP/1.1, (http/1.1)}{0.0.0.0:18080}
2024-03-19 14:26:39,008 INFO [main] org.eclipse.jetty.server.Server Started
@26817ms
2024-03-19 14:26:39,013 ERROR [main] o.apache.nifi.registry.jetty.JettyServer
Server start failed
java.lang.RuntimeException: java.lang.IllegalStateException: Unable to load
cache item
at
org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:69)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:356)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.nifi.registry.jetty.JettyServer.start(JettyServer.java:93)
at org.apache.nifi.registry.NiFiRegistry.(NiFiRegistry.java:110)
at org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:159)
Caused by: java.lang.IllegalStateException: Unable to load cache item
at
org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:79)
at
org.springframework.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
at
org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:134)
at
org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:319)
at
org.springframework.cglib.proxy.Enhancer.createHelper(Enhancer.java:572)
at org.springframework.cglib.proxy.Enhancer.createClass(Enhancer.java:419)
at
Re: [discuss] What to do with the Cassandra components
All, I support a Controller Service API abstraction around the Cassandra Driver. The changes from DataStax 3 to 4 already highlight the need for that abstraction. The donation of the DataStax Java driver to Apache [1] also shows the value of providing some level of isolation, if at all possible. I have not taken a close look at the Matt's branch, and the details of the abstraction are important, but having the abstraction can be useful to avoid getting back to this same situation. Regards, David Handermann [1] https://github.com/apache/cassandra-java-driver/ On Tue, Mar 19, 2024 at 12:37 PM Mike Thomsen wrote: > > Matt, > > I got that. My point was that the Java changes appear to be a one time > thing that DataStax did to make a better driver with a much more > future-proof API. Since Scylla tracks them as closely as possible, I > suspect that we don't need to plan for a bunch of abstraction to isolate > Java changes. > > On Tue, Mar 19, 2024 at 11:07 AM Steven Matison > wrote: > > > That was kinda where i got stuck and fell out on my branch/jira. Mike and > > I wanted to make a new controller service , without backward compatibility; > > and remove the duplicate driver/connection properties found in some of the > > processors. > > > > I agree taking out all old stuff and making new controller service makes > > most sense. 4.x and 5.x should be mostly backwards compatible to 2&3.x > > with how it’s used within current processors. > > > > > > > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess > > wrote: > > > > > The abstraction is to isolate Java API changes, not protocol > > compatibility > > > Changing to the java-driver comes with a number of changes to the code > > (see > > > Steven's and my branches), if we can abstract that API it should lead to > > > more maintainable code in the future by not having to change any > > > processors, just the controller service implementation. > > > > > > > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen > > > wrote: > > > > > > > > > > > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > > > > > Directly quoting Scylla docs here: > > > > > > > > > The Scylla Java Driver is a drop-in replacement for the DataStax Java > > > > Driver. As such, no code changes are needed to use this driver. > > > > > > > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen > > > > wrote: > > > > > > > > > Matt, > > > > > > > > > > I don't think we need to really "abstract above" the drivers because > > > the > > > > > Java DataStax driver appears to support 4.X all the way back to 2.X, > > as > > > > > well as the enterprise versions from DataStax > > > > > > > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > > > > > > Similar situation with Scylla. When I looked at the driver, it > > appeared > > > > to > > > > > copy verbatim the entire public API of that driver. So I think before > > > we > > > > > dive into abstractions, it's worth doing a bit more validation of > > these > > > > > details. IMHO, this might be a much lighter lift than anticipated. > > > > > > > > > > > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess > > > > wrote: > > > > > > > > > >> Totally agree, that's what my branch does (see link in previous > > > email). > > > > >> The > > > > >> more I work with it, the more I think I can abstract it further from > > > > their > > > > >> JDBC-like API but I started with a bunch of delegate classes then I > > > > figure > > > > >> I'll see where I can consolidate to more abstract concepts. If I > > don't > > > > >> have > > > > >> to support Cassandra 3 with the new API, so much the better. > > > > >> > > > > >> Regards, > > > > >> Matt > > > > >> > > > > >> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > > > > >> exceptionfact...@apache.org> wrote: > > > > >> > > > > >> > Matt et al, > > > > >> > > > > > >> > It is good to see the background effort on moving Cassandra > > > > >> > capabilities in a supportable direction. > > > > >> > > > > > >> > I think new Cassandra components will require a significant > > > departure > > > > >> > from current Controller Service abstractions. Right now, the > > > existing > > > > >> > service interface does not provide a clean abstraction from the > > > > >> > Cassandra library, which is part of the reason for the current > > > > >> > coupling to the legacy driver version. > > > > >> > > > > > >> > Following up from Joe's comments, it seems like the cleanest way > > > > >> > forward is to deprecate the current bundle on the 1.x branch, and > > > > >> > remove the current bundle from the main branch. That will provide > > a > > > > >> > clean slate for new Service and Processor implementations, without > > > > >> > concern for uncertain compatibility questions. > > > > >> > > > > > >> > Regards, > > > > >> > David Handermann > > > > >> > > > > > >> > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess < > > mattyb...@apache.org>
Re: [discuss] What to do with the Cassandra components
Matt, I got that. My point was that the Java changes appear to be a one time thing that DataStax did to make a better driver with a much more future-proof API. Since Scylla tracks them as closely as possible, I suspect that we don't need to plan for a bunch of abstraction to isolate Java changes. On Tue, Mar 19, 2024 at 11:07 AM Steven Matison wrote: > That was kinda where i got stuck and fell out on my branch/jira. Mike and > I wanted to make a new controller service , without backward compatibility; > and remove the duplicate driver/connection properties found in some of the > processors. > > I agree taking out all old stuff and making new controller service makes > most sense. 4.x and 5.x should be mostly backwards compatible to 2&3.x > with how it’s used within current processors. > > > > On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess > wrote: > > > The abstraction is to isolate Java API changes, not protocol > compatibility > > Changing to the java-driver comes with a number of changes to the code > (see > > Steven's and my branches), if we can abstract that API it should lead to > > more maintainable code in the future by not having to change any > > processors, just the controller service implementation. > > > > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen > > wrote: > > > > > > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > > > Directly quoting Scylla docs here: > > > > > > > The Scylla Java Driver is a drop-in replacement for the DataStax Java > > > Driver. As such, no code changes are needed to use this driver. > > > > > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen > > > wrote: > > > > > > > Matt, > > > > > > > > I don't think we need to really "abstract above" the drivers because > > the > > > > Java DataStax driver appears to support 4.X all the way back to 2.X, > as > > > > well as the enterprise versions from DataStax > > > > > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > > > > Similar situation with Scylla. When I looked at the driver, it > appeared > > > to > > > > copy verbatim the entire public API of that driver. So I think before > > we > > > > dive into abstractions, it's worth doing a bit more validation of > these > > > > details. IMHO, this might be a much lighter lift than anticipated. > > > > > > > > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess > > > wrote: > > > > > > > >> Totally agree, that's what my branch does (see link in previous > > email). > > > >> The > > > >> more I work with it, the more I think I can abstract it further from > > > their > > > >> JDBC-like API but I started with a bunch of delegate classes then I > > > figure > > > >> I'll see where I can consolidate to more abstract concepts. If I > don't > > > >> have > > > >> to support Cassandra 3 with the new API, so much the better. > > > >> > > > >> Regards, > > > >> Matt > > > >> > > > >> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > > > >> exceptionfact...@apache.org> wrote: > > > >> > > > >> > Matt et al, > > > >> > > > > >> > It is good to see the background effort on moving Cassandra > > > >> > capabilities in a supportable direction. > > > >> > > > > >> > I think new Cassandra components will require a significant > > departure > > > >> > from current Controller Service abstractions. Right now, the > > existing > > > >> > service interface does not provide a clean abstraction from the > > > >> > Cassandra library, which is part of the reason for the current > > > >> > coupling to the legacy driver version. > > > >> > > > > >> > Following up from Joe's comments, it seems like the cleanest way > > > >> > forward is to deprecate the current bundle on the 1.x branch, and > > > >> > remove the current bundle from the main branch. That will provide > a > > > >> > clean slate for new Service and Processor implementations, without > > > >> > concern for uncertain compatibility questions. > > > >> > > > > >> > Regards, > > > >> > David Handermann > > > >> > > > > >> > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess < > mattyb...@apache.org> > > > >> wrote: > > > >> > > > > > >> > > What do y'all think about removing the individual connection > > > >> properties > > > >> > > from the Cassandra processors for NiFi 2.0 and requiring a > > > >> > > CassandraSessionProvider instead? I think we started doing that > > > >> elsewhere > > > >> > > (Elasticsearch maybe?), I noticed duplicate code in the > > > >> > > CassandraSessionProvider and AbstractCassandraProcessor, if we > > keep > > > >> those > > > >> > > properties I can refactor them into a utility class. > > > >> > > > > > >> > > Thanks, > > > >> > > Matt > > > >> > > > > > >> > > > > > >> > > On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < > > > >> steven.mati...@gmail.com > > > >> > > > > > >> > > wrote: > > > >> > > > > > >> > > > I got through quite a bit of work to enable 4.x… > > > >> > > > > > > >> > > > The 3.x pieces
Re: [discuss] What to do with the Cassandra components
That was kinda where i got stuck and fell out on my branch/jira. Mike and I wanted to make a new controller service , without backward compatibility; and remove the duplicate driver/connection properties found in some of the processors. I agree taking out all old stuff and making new controller service makes most sense. 4.x and 5.x should be mostly backwards compatible to 2&3.x with how it’s used within current processors. On Tue, Mar 19, 2024 at 10:49 AM Matt Burgess wrote: > The abstraction is to isolate Java API changes, not protocol compatibility > Changing to the java-driver comes with a number of changes to the code (see > Steven's and my branches), if we can abstract that API it should lead to > more maintainable code in the future by not having to change any > processors, just the controller service implementation. > > > On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen > wrote: > > > > > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > > > Directly quoting Scylla docs here: > > > > > The Scylla Java Driver is a drop-in replacement for the DataStax Java > > Driver. As such, no code changes are needed to use this driver. > > > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen > > wrote: > > > > > Matt, > > > > > > I don't think we need to really "abstract above" the drivers because > the > > > Java DataStax driver appears to support 4.X all the way back to 2.X, as > > > well as the enterprise versions from DataStax > > > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > > > Similar situation with Scylla. When I looked at the driver, it appeared > > to > > > copy verbatim the entire public API of that driver. So I think before > we > > > dive into abstractions, it's worth doing a bit more validation of these > > > details. IMHO, this might be a much lighter lift than anticipated. > > > > > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess > > wrote: > > > > > >> Totally agree, that's what my branch does (see link in previous > email). > > >> The > > >> more I work with it, the more I think I can abstract it further from > > their > > >> JDBC-like API but I started with a bunch of delegate classes then I > > figure > > >> I'll see where I can consolidate to more abstract concepts. If I don't > > >> have > > >> to support Cassandra 3 with the new API, so much the better. > > >> > > >> Regards, > > >> Matt > > >> > > >> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > > >> exceptionfact...@apache.org> wrote: > > >> > > >> > Matt et al, > > >> > > > >> > It is good to see the background effort on moving Cassandra > > >> > capabilities in a supportable direction. > > >> > > > >> > I think new Cassandra components will require a significant > departure > > >> > from current Controller Service abstractions. Right now, the > existing > > >> > service interface does not provide a clean abstraction from the > > >> > Cassandra library, which is part of the reason for the current > > >> > coupling to the legacy driver version. > > >> > > > >> > Following up from Joe's comments, it seems like the cleanest way > > >> > forward is to deprecate the current bundle on the 1.x branch, and > > >> > remove the current bundle from the main branch. That will provide a > > >> > clean slate for new Service and Processor implementations, without > > >> > concern for uncertain compatibility questions. > > >> > > > >> > Regards, > > >> > David Handermann > > >> > > > >> > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess > > >> wrote: > > >> > > > > >> > > What do y'all think about removing the individual connection > > >> properties > > >> > > from the Cassandra processors for NiFi 2.0 and requiring a > > >> > > CassandraSessionProvider instead? I think we started doing that > > >> elsewhere > > >> > > (Elasticsearch maybe?), I noticed duplicate code in the > > >> > > CassandraSessionProvider and AbstractCassandraProcessor, if we > keep > > >> those > > >> > > properties I can refactor them into a utility class. > > >> > > > > >> > > Thanks, > > >> > > Matt > > >> > > > > >> > > > > >> > > On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < > > >> steven.mati...@gmail.com > > >> > > > > >> > > wrote: > > >> > > > > >> > > > I got through quite a bit of work to enable 4.x… > > >> > > > > > >> > > > The 3.x pieces that were not backwards compatible is very edge > use > > >> > case and > > >> > > > could have been done slightly differently but with work around. > > >> > > > > > >> > > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess < > > mattyb...@apache.org> > > >> > wrote: > > >> > > > > > >> > > > > Oops used the wrong email address so if there have been > > responses > > >> to > > >> > the > > >> > > > > Cassandra thread since mine I missed them, my bad! > > >> > > > > > > >> > > > > On
Re: [discuss] What to do with the Cassandra components
The abstraction is to isolate Java API changes, not protocol compatibility. Changing to the java-driver comes with a number of changes to the code (see Steven's and my branches), if we can abstract that API it should lead to more maintainable code in the future by not having to change any processors, just the controller service implementation. On Tue, Mar 19, 2024 at 10:14 AM Mike Thomsen wrote: > > https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html > > Directly quoting Scylla docs here: > > > The Scylla Java Driver is a drop-in replacement for the DataStax Java > Driver. As such, no code changes are needed to use this driver. > > On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen > wrote: > > > Matt, > > > > I don't think we need to really "abstract above" the drivers because the > > Java DataStax driver appears to support 4.X all the way back to 2.X, as > > well as the enterprise versions from DataStax > > > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > > > Similar situation with Scylla. When I looked at the driver, it appeared > to > > copy verbatim the entire public API of that driver. So I think before we > > dive into abstractions, it's worth doing a bit more validation of these > > details. IMHO, this might be a much lighter lift than anticipated. > > > > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess > wrote: > > > >> Totally agree, that's what my branch does (see link in previous email). > >> The > >> more I work with it, the more I think I can abstract it further from > their > >> JDBC-like API but I started with a bunch of delegate classes then I > figure > >> I'll see where I can consolidate to more abstract concepts. If I don't > >> have > >> to support Cassandra 3 with the new API, so much the better. > >> > >> Regards, > >> Matt > >> > >> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > >> exceptionfact...@apache.org> wrote: > >> > >> > Matt et al, > >> > > >> > It is good to see the background effort on moving Cassandra > >> > capabilities in a supportable direction. > >> > > >> > I think new Cassandra components will require a significant departure > >> > from current Controller Service abstractions. Right now, the existing > >> > service interface does not provide a clean abstraction from the > >> > Cassandra library, which is part of the reason for the current > >> > coupling to the legacy driver version. > >> > > >> > Following up from Joe's comments, it seems like the cleanest way > >> > forward is to deprecate the current bundle on the 1.x branch, and > >> > remove the current bundle from the main branch. That will provide a > >> > clean slate for new Service and Processor implementations, without > >> > concern for uncertain compatibility questions. > >> > > >> > Regards, > >> > David Handermann > >> > > >> > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess > >> wrote: > >> > > > >> > > What do y'all think about removing the individual connection > >> properties > >> > > from the Cassandra processors for NiFi 2.0 and requiring a > >> > > CassandraSessionProvider instead? I think we started doing that > >> elsewhere > >> > > (Elasticsearch maybe?), I noticed duplicate code in the > >> > > CassandraSessionProvider and AbstractCassandraProcessor, if we keep > >> those > >> > > properties I can refactor them into a utility class. > >> > > > >> > > Thanks, > >> > > Matt > >> > > > >> > > > >> > > On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < > >> steven.mati...@gmail.com > >> > > > >> > > wrote: > >> > > > >> > > > I got through quite a bit of work to enable 4.x… > >> > > > > >> > > > The 3.x pieces that were not backwards compatible is very edge use > >> > case and > >> > > > could have been done slightly differently but with work around. > >> > > > > >> > > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess < > mattyb...@apache.org> > >> > wrote: > >> > > > > >> > > > > Oops used the wrong email address so if there have been > responses > >> to > >> > the > >> > > > > Cassandra thread since mine I missed them, my bad! > >> > > > > > >> > > > > On Fri, Mar 15, 2024 at 2:00 PM Matt Burgess < > mattyb...@gmail.com > >> > > >> > > > wrote: > >> > > > > > >> > > > > > I believe the CQL protocol is backwards compatible but the > Java > >> > API is > >> > > > > > not. For example "com.datastax.driver.core.Session" is now > >> > > > > > "com.datastax.oss.driver.api.core.session.Session" and there > is > >> no > >> > more > >> > > > > > "Cluster" class. Might be fairly trivial to fix though, if > >> that's > >> > the > >> > > > > path > >> > > > > > of least resistance. > >> > > > > > > >> > > > > > On Fri, Mar 15, 2024 at 1:40 PM Joe Witt > >> > wrote: > >> > > > > > > >> > > > > >> Matt > >> > > > > >> > >> > > > > >> I dont know a ton about Cassandra but when I looked at > >> >
Re: [discuss] What to do with the Cassandra components
https://opensource.docs.scylladb.com/stable/using-scylla/drivers/cql-drivers/scylla-java-driver.html Directly quoting Scylla docs here: > The Scylla Java Driver is a drop-in replacement for the DataStax Java Driver. As such, no code changes are needed to use this driver. On Tue, Mar 19, 2024 at 10:13 AM Mike Thomsen wrote: > Matt, > > I don't think we need to really "abstract above" the drivers because the > Java DataStax driver appears to support 4.X all the way back to 2.X, as > well as the enterprise versions from DataStax > > https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html > > Similar situation with Scylla. When I looked at the driver, it appeared to > copy verbatim the entire public API of that driver. So I think before we > dive into abstractions, it's worth doing a bit more validation of these > details. IMHO, this might be a much lighter lift than anticipated. > > > On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess wrote: > >> Totally agree, that's what my branch does (see link in previous email). >> The >> more I work with it, the more I think I can abstract it further from their >> JDBC-like API but I started with a bunch of delegate classes then I figure >> I'll see where I can consolidate to more abstract concepts. If I don't >> have >> to support Cassandra 3 with the new API, so much the better. >> >> Regards, >> Matt >> >> On Mon, Mar 18, 2024 at 4:14 PM David Handermann < >> exceptionfact...@apache.org> wrote: >> >> > Matt et al, >> > >> > It is good to see the background effort on moving Cassandra >> > capabilities in a supportable direction. >> > >> > I think new Cassandra components will require a significant departure >> > from current Controller Service abstractions. Right now, the existing >> > service interface does not provide a clean abstraction from the >> > Cassandra library, which is part of the reason for the current >> > coupling to the legacy driver version. >> > >> > Following up from Joe's comments, it seems like the cleanest way >> > forward is to deprecate the current bundle on the 1.x branch, and >> > remove the current bundle from the main branch. That will provide a >> > clean slate for new Service and Processor implementations, without >> > concern for uncertain compatibility questions. >> > >> > Regards, >> > David Handermann >> > >> > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess >> wrote: >> > > >> > > What do y'all think about removing the individual connection >> properties >> > > from the Cassandra processors for NiFi 2.0 and requiring a >> > > CassandraSessionProvider instead? I think we started doing that >> elsewhere >> > > (Elasticsearch maybe?), I noticed duplicate code in the >> > > CassandraSessionProvider and AbstractCassandraProcessor, if we keep >> those >> > > properties I can refactor them into a utility class. >> > > >> > > Thanks, >> > > Matt >> > > >> > > >> > > On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < >> steven.mati...@gmail.com >> > > >> > > wrote: >> > > >> > > > I got through quite a bit of work to enable 4.x… >> > > > >> > > > The 3.x pieces that were not backwards compatible is very edge use >> > case and >> > > > could have been done slightly differently but with work around. >> > > > >> > > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess >> > wrote: >> > > > >> > > > > Oops used the wrong email address so if there have been responses >> to >> > the >> > > > > Cassandra thread since mine I missed them, my bad! >> > > > > >> > > > > On Fri, Mar 15, 2024 at 2:00 PM Matt Burgess > > >> > > > wrote: >> > > > > >> > > > > > I believe the CQL protocol is backwards compatible but the Java >> > API is >> > > > > > not. For example "com.datastax.driver.core.Session" is now >> > > > > > "com.datastax.oss.driver.api.core.session.Session" and there is >> no >> > more >> > > > > > "Cluster" class. Might be fairly trivial to fix though, if >> that's >> > the >> > > > > path >> > > > > > of least resistance. >> > > > > > >> > > > > > On Fri, Mar 15, 2024 at 1:40 PM Joe Witt >> > wrote: >> > > > > > >> > > > > >> Matt >> > > > > >> >> > > > > >> I dont know a ton about Cassandra but when I looked at >> > client/driver >> > > > > notes >> > > > > >> for 4+ it said it was compatible all the way back to 3.x. Not >> > sure >> > > > > what >> > > > > >> that means but it surely seems worth exploring. Also I dont >> know >> > if >> > > > the >> > > > > >> 4.x drivers get rid of the vulnerable bits. >> > > > > >> >> > > > > >> Thanks >> > > > > >> >> > > > > >> On Fri, Mar 15, 2024 at 10:39 AM Matt Burgess < >> > mattyb...@apache.org> >> > > > > >> wrote: >> > > > > >> >> > > > > >> > At the very least we should upgrade to Cassandra 3.11.6: >> > > > > >> > >> > > > > >> > https://github.com/apache/cassandra/blob/cassandra-3.11.16/CHANGES.txt >> > > > > >> > >> > > > > >> > On Fri, Mar 15, 2024 at 1:31 PM Matt
Re: [discuss] What to do with the Cassandra components
Matt, I don't think we need to really "abstract above" the drivers because the Java DataStax driver appears to support 4.X all the way back to 2.X, as well as the enterprise versions from DataStax https://docs.datastax.com/en/driver-matrix/docs/java-drivers.html Similar situation with Scylla. When I looked at the driver, it appeared to copy verbatim the entire public API of that driver. So I think before we dive into abstractions, it's worth doing a bit more validation of these details. IMHO, this might be a much lighter lift than anticipated. On Mon, Mar 18, 2024 at 4:30 PM Matt Burgess wrote: > Totally agree, that's what my branch does (see link in previous email). The > more I work with it, the more I think I can abstract it further from their > JDBC-like API but I started with a bunch of delegate classes then I figure > I'll see where I can consolidate to more abstract concepts. If I don't have > to support Cassandra 3 with the new API, so much the better. > > Regards, > Matt > > On Mon, Mar 18, 2024 at 4:14 PM David Handermann < > exceptionfact...@apache.org> wrote: > > > Matt et al, > > > > It is good to see the background effort on moving Cassandra > > capabilities in a supportable direction. > > > > I think new Cassandra components will require a significant departure > > from current Controller Service abstractions. Right now, the existing > > service interface does not provide a clean abstraction from the > > Cassandra library, which is part of the reason for the current > > coupling to the legacy driver version. > > > > Following up from Joe's comments, it seems like the cleanest way > > forward is to deprecate the current bundle on the 1.x branch, and > > remove the current bundle from the main branch. That will provide a > > clean slate for new Service and Processor implementations, without > > concern for uncertain compatibility questions. > > > > Regards, > > David Handermann > > > > On Mon, Mar 18, 2024 at 2:35 PM Matt Burgess > wrote: > > > > > > What do y'all think about removing the individual connection properties > > > from the Cassandra processors for NiFi 2.0 and requiring a > > > CassandraSessionProvider instead? I think we started doing that > elsewhere > > > (Elasticsearch maybe?), I noticed duplicate code in the > > > CassandraSessionProvider and AbstractCassandraProcessor, if we keep > those > > > properties I can refactor them into a utility class. > > > > > > Thanks, > > > Matt > > > > > > > > > On Fri, Mar 15, 2024 at 2:44 PM Steven Matison < > steven.mati...@gmail.com > > > > > > wrote: > > > > > > > I got through quite a bit of work to enable 4.x… > > > > > > > > The 3.x pieces that were not backwards compatible is very edge use > > case and > > > > could have been done slightly differently but with work around. > > > > > > > > https://github.com/steven-matison/nifi/tree/nifi-10120-1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 15, 2024 at 2:30 PM Matt Burgess > > wrote: > > > > > > > > > Oops used the wrong email address so if there have been responses > to > > the > > > > > Cassandra thread since mine I missed them, my bad! > > > > > > > > > > On Fri, Mar 15, 2024 at 2:00 PM Matt Burgess > > > > wrote: > > > > > > > > > > > I believe the CQL protocol is backwards compatible but the Java > > API is > > > > > > not. For example "com.datastax.driver.core.Session" is now > > > > > > "com.datastax.oss.driver.api.core.session.Session" and there is > no > > more > > > > > > "Cluster" class. Might be fairly trivial to fix though, if that's > > the > > > > > path > > > > > > of least resistance. > > > > > > > > > > > > On Fri, Mar 15, 2024 at 1:40 PM Joe Witt > > wrote: > > > > > > > > > > > >> Matt > > > > > >> > > > > > >> I dont know a ton about Cassandra but when I looked at > > client/driver > > > > > notes > > > > > >> for 4+ it said it was compatible all the way back to 3.x. Not > > sure > > > > > what > > > > > >> that means but it surely seems worth exploring. Also I dont > know > > if > > > > the > > > > > >> 4.x drivers get rid of the vulnerable bits. > > > > > >> > > > > > >> Thanks > > > > > >> > > > > > >> On Fri, Mar 15, 2024 at 10:39 AM Matt Burgess < > > mattyb...@apache.org> > > > > > >> wrote: > > > > > >> > > > > > >> > At the very least we should upgrade to Cassandra 3.11.6: > > > > > >> > > > > > > > > https://github.com/apache/cassandra/blob/cassandra-3.11.16/CHANGES.txt > > > > > >> > > > > > > >> > On Fri, Mar 15, 2024 at 1:31 PM Matt Burgess < > > mattyb...@apache.org> > > > > > >> wrote: > > > > > >> > > > > > > >> > > If the community agrees to get rid of Cassandra 3 that'll > > save me > > > > > >> effort > > > > > >> > > on the refactor after I add Cassandra 4 :) Otherwise those > > > > > >> > > vulnerabilities would only be in a "new" Cassandra 3 > services > > NAR > > > > > that > > > > > >> > > would not be included in the convenience binary. > > > > > >> > > > > > > > >> > > On Fri, Mar 15, 2024 at 1:28 PM Joe
