This feels analogous to other past discussions around prioritizing a config that enables new users to clone + build + run as easily as possible, vs. having better prod recommendations out of the box.
Both are important. I personally think we should default configuration to make it just work for new users, and have a config to allow power users to fail if ACCP is not present, and warn once on startup if we tolerate missing ACCP but detect it is absent. > On Jul 20, 2023, at 14:51, Brandon Williams <dri...@gmail.com> wrote: > > I think we could special-case and default to 'auto' but allow other > more explicit options. > > Kind Regards, > Brandon > >> On Thu, Jul 20, 2023 at 4:18 PM German Eichberger via dev >> <dev@cassandra.apache.org> wrote: >> >> In general I agree with Joey -- but I would prefer if this behavior is >> configurable, e.g. there is an option to get a startup failure if the >> configured fastest provider can't run for any reason to avoid a "silent" >> performance degradation as Jordan was experiencing. >> >> Thanks, >> German >> >> ________________________________ >> From: Joseph Lynch <joe.e.ly...@gmail.com> >> Sent: Thursday, July 20, 2023 7:38 AM >> To: dev@cassandra.apache.org <dev@cassandra.apache.org> >> Subject: [EXTERNAL] Re: [DISCUSS] Using ACCP or tc-native by default >> >> Having native dependencies shouldn't make the project x86 only, it >> should just accelerate the performance on x86 when available. Can't we >> just try to load the fastest available provider (so arm will use >> native java but x86 will use proper hardware acceleration) and failing >> that fall-back to the default? If I recall correctly from the >> messaging service patches (and zstd/lz4) it's reasonably >> straightforward to try to load native code and then fail-back if you >> fail. >> >> -Joey >> >>> On Thu, Jul 20, 2023 at 10:27 AM J. D. Jordan <jeremiah.jor...@gmail.com> >>> wrote: >>> >>> Maybe we could start providing Dockerfile’s and/or make arch specific >>> rpm/deb packages that have everything setup correctly per architecture? >>> We could also download them all and have the startup scripts put stuff in >>> the right places depending on the arch of the machine running them? >>> I feel like there are probably multiple ways we could solve this without >>> requiring users to jump through a bunch of hoops? >>> But I do agree we can’t make the project x86 only. >>> >>> -Jeremiah >>> >>>> On Jul 20, 2023, at 2:01 AM, Miklosovic, Stefan >>>> <stefan.mikloso...@netapp.com> wrote: >>>> >>>> Hi, >>>> >>>> as I was reviewing the patch for this feature (1), we realized that it is >>>> not quite easy to bundle this directly into Cassandra. >>>> >>>> The problem is that this was supposed to be introduced as a new dependency: >>>> >>>> <dependency> >>>> <groupId>software.amazon.cryptools</groupId> >>>> <artifactId>AmazonCorrettoCryptoProvider</artifactId> >>>> <version>2.2.0</version> >>>> <classifier>linux-x86_64</classifier> >>>> <dependency> >>>> >>>> Notice "classifier". That means that if we introduced this dependency into >>>> the project, what about ARM users? (there is corresponding aarch >>>> classifier as well). ACCP is platform-specific but we have to ship >>>> Cassandra platform-agnostic. It just needs to run OOTB everywhere. If we >>>> shipped that with x86 and a user runs Cassandra on ARM, I guess that would >>>> break things, right? >>>> >>>> We also can not just add both dependencies (both x86 and aarch) because >>>> how would we differentiate between them in runtime? That all is just too >>>> tricky / error prone. >>>> >>>> So, the approach we want to take is this: >>>> >>>> 1) nothing will be bundled in Cassandra by default >>>> 2) a user is supposed to download the library and put it to the class path >>>> 3) a user is supposed to put the implementation of ICryptoProvider >>>> interface Cassandra exposes to the class path >>>> 3) a user is supposed to configure cassandra.yaml and its section >>>> "crypto_provider" to reference the implementation he wants >>>> >>>> That way, we avoid the situation when somebody runs x86 lib on ARM or vice >>>> versa. >>>> >>>> By default, NoOpProvider will be used, that means that the default crypto >>>> provider from JRE will be used. >>>> >>>> It can seem like we have not done too much progress here but hey ... we >>>> opened the project to the custom implementations of crypto providers a >>>> community can create. E.g. as 3rd party extensions etc ... >>>> >>>> I want to be sure that everybody is aware of this change (that we plan to >>>> do that in such a way that it will not be "bundled") and that everybody is >>>> on board with this. Otherwise I am all ears about how to do that >>>> differently. >>>> >>>> (1) >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCASSANDRA-18624&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kYGSZGi3caINvm%2FDT4ms3%2BrcnMTxg0E921cMjmUvHQw%3D&reserved=0 >>>> >>>> ________________________________________ >>>> From: German Eichberger via dev <dev@cassandra.apache.org> >>>> Sent: Friday, June 23, 2023 22:43 >>>> To: dev >>>> Subject: Re: [DISCUSS] Using ACCP or tc-native by default >>>> >>>> NetApp Security WARNING: This is an external email. Do not click links or >>>> open attachments unless you recognize the sender and know the content is >>>> safe. >>>> >>>> >>>> >>>> +1 to ACCP - we love performance. >>>> ________________________________ >>>> From: David Capwell <dcapw...@apple.com> >>>> Sent: Thursday, June 22, 2023 4:21 PM >>>> To: dev <dev@cassandra.apache.org> >>>> Subject: [EXTERNAL] Re: [DISCUSS] Using ACCP or tc-native by default >>>> >>>> +1 to ACCP >>>> >>>> On Jun 22, 2023, at 3:05 PM, C. Scott Andreas <sc...@paradoxica.net> wrote: >>>> >>>> +1 for ACCP and can attest to its results. ACCP also optimizes for a range >>>> of hash functions and other cryptographic primitives beyond TLS >>>> acceleration for Netty. >>>> >>>> On Jun 22, 2023, at 2:07 PM, Jeff Jirsa <jji...@gmail.com> wrote: >>>> >>>> >>>> Either would be better than today. >>>> >>>> On Thu, Jun 22, 2023 at 1:57 PM Jordan West >>>> <jw...@apache.org<mailto:jw...@apache.org>> wrote: >>>> Hi, >>>> >>>> I’m wondering if there is appetite to change the default SSL provider for >>>> Cassandra going forward to either ACCP [1] or tc-native in Netty? Our >>>> deployment as well as others I’m aware of make this change in their fork >>>> and it can lead to significant performance improvement. When recently >>>> qualifying 4.1 without using ACCP (by accident) we noticed p99 latencies >>>> were 2x higher than 3.0 w/ ACCP. Wiring up ACCP can be a bit of a pain and >>>> also requires some amount of customization. I think it could be great for >>>> the wider community to adopt it. >>>> >>>> The biggest hurdle I foresee is licensing but ACCP is Apache 2.0 licensed. >>>> Anything else I am missing before opening a JIRA and submitting a patch? >>>> >>>> Jordan >>>> >>>> >>>> [1] >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcorretto%2Famazon-corretto-crypto-provider&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bGU%2B7jerm8fiS0Tmav8geFStI%2FcvZKinF35YGKuEQtY%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam04.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fgithub.com*2Fcorretto*2Famazon-corretto-crypto-provider%26data%3D05*7C01*7CStefan.Miklosovic*40netapp.com*7C4d73ac88a46f4386826e08db742a82f3*7C4b0911a0929b4715944bc03745165b3a*7C0*7C0*7C638231498154472637*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C%26sdata%3DIvtqDSHL*2BOunrhagmYKTfPa8zlknIPijr8TwVvCKKQw*3D%26reserved%3D0__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!PbtH5S7Ebw!fuCaq60SW0zceq0aFBYG3J8ga4gAtUG5n1qZyiyRSo8X-epQfmmdXqSOXzX4tQbeAMSc5QuTXtz2W0xWyPp59OcTF5Ra%24&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZWsg5GPkPCDf6EFKcqhEtbYLtEpHtcFbY8ir2i69Nfw%3D&reserved=0 >>>> > >>>> >>>>
