Gary, Can you have a look at this?
https://github.com/apache/commons-crypto/pull/242 Alex On Sun, Aug 27, 2023 at 8:38 AM Gary Gregory <garydgreg...@gmail.com> wrote: > On Fri, Aug 25, 2023 at 9:48 PM Alex Remily <alex.rem...@gmail.com> wrote: > > > > <A big item to consider is if and how 1.1 vs 3.0 should be handled. > Breakup > > the current module into different maven modules? Not support both?> > > > > Agreed. Just to provide some history, when I was working on the 1.1.x > > upgrade I was guided by commons committer Marcelo Vanzin. Marcelo > required > > a design that supported runtime discovery of the underlying openssl > API. I > > don't recall all of the rationale for the requirement, but he insisted > that > > any commons-crypto upgrade must support legacy and current versions of > > openssl transparently to the calling program. The end result is what we > > have now. I don't recall where in the code commons-crypto makes the > > underlying version checks, at library initialization time or when a > > specific function is called, but the end result is that users need only > > download the latest commons-crypto release regardless of their underlying > > openssl API--as long as they are running supported openssl versions. > > > > In my view there are a few open questions regarding the current approach > as > > compared to an API-specific one. One, what is the performance penalty > > associated with the dynamic version checks? Two, how much complexity > does > > it introduce into the codebase? Finally, what was the use case that > drove > > the runtime checking requirement? Marcelo could answer the last > question. > > I don't know if he is still involved in the community (I haven't seen him > > around for awhile. IIRC, he was primarily a Spark committer). > > > > Another consideration is the FIPS certification. I work in a heavily > > regulated industry and FIPS is a real constraint. I haven't personally > > encountered a requirement to deploy a FIPS compliant openssl in our > > application code, but it's probably just a matter of time. In terms of > > expanding our user base, it may make sense to provide that capability. > It > > doesn't seem to be generally available from an existing provider. > > > > Regarding message digests/HMAC, I question whether the performance gain > > from native code would significantly outperform some of the modern JCA > > providers. As Matt Sicker pointed out, there are other implementations > > supported by major vendors, like AWS, that may be as fast as a JNI > wrapper > > on OpenSSL, or at least close enough not to bother with the added > > complexity of the native stuff. The only way I know to answer that > > question is to write the code and run a load test comparison. > > > > > https://aws.amazon.com/blogs/opensource/introducing-amazon-corretto-crypto-provider-accp/ > > > > In what forum should we discuss these issues? > > This developer's mailing list is where all decisions MUST be recorded, > so it makes sense that discussions occur here. We can also use Jira > tickets for specific tasks. Different discussions can use these > different forums as we best see fit. I would not recommend using Slack > as that data does or will eventually disappear. I do use Slack as a > convenience at times for reminding people/channels to vote or pay > attention to something happening in Jira or on a mailing list. > > > Are we limited to this > > distro or do we have other options? > > I'm not sure what this means. > > > How do we form teams? > I'm not sure what that means in the context of Apache. Apache has no > notion of teams that I am aware of. We have Projects like Apache > Commons, and Commons has Components like Commons Crypto. > We are all on this mailing list, we can work on whatever we want, and > we can use Jira to track tasks. If someone wants to work on a task, > they can announce it here or in a Jira ticket, to avoid clashes or > just be informative. > > > What is our > > governance model? > The same as any Apache project with the usual leeway allowed to each PMC. > > > How do we make decisions? > The same as any Apache project with the usual leeway allowed to each PMC. > See https://httpd.apache.org/dev/guidelines.html > A project can come up with it's own processes, for example see > https://www.apache.org/foundation/voting.html > > Gary > > > > > FYI: There's already issues on the backlog for OpenSSL 3 and HMAC: > > https://issues.apache.org/jira/browse/CRYPTO-165 > > https://issues.apache.org/jira/browse/CRYPTO-164 > > > > Alex > > > > > > On Wed, Aug 23, 2023 at 10:21 PM Gary Gregory <garydgreg...@gmail.com> > > wrote: > > > > > That would be great. I think this is worth the effort. A big item to > > > consider is if and how 1.1 vs 3.0 should be handled. Breakup the > current > > > module into different maven modules? Not support both? > > > > > > Gary > > > > > > On Wed, Aug 23, 2023, 8:37 PM Alex Remily <alex.rem...@gmail.com> > wrote: > > > > > > > <Ack about licensing. The idea wouldn't be to copy the code, but to > learn > > > > how to implement message digests and HMAC on top of OpenSSL 3.0.8.> > > > > > > > > Implementing the OpenSSL 3 API and exposing OpenSSL HMAC > functionality in > > > > commons-crypto are things I've wanted to engage on for a while now. > I > > > was > > > > involved in the commons-crypto OpenSSL 1.1.x upgrade so I have some > > > > familiarity with the code base, albeit dated. The reason that > neither > > > the > > > > OpenSSL 3 API nor the HMAC functionality have been added to > > > commons-crypto > > > > yet, IMHO, is because both are non-trivial efforts and we all have > day > > > > jobs. Until now, there hasn't been much of an ask for either > feature, but > > > > considering the recent chatter maybe there's enough interest to > generate > > > > some momentum. I would be happy to collaborate on this effort if it > > > gains > > > > traction. > > > > > > > > Alex > > > > > > > > On Wed, Aug 23, 2023 at 5:21 PM Matt Sicker <m...@musigma.org> > wrote: > > > > > > > >> You may find this project (also Apache licensed, so fairly safe for > > > reuse > > > >> with commons-crypto updates) to be helpful for that aspect of > message > > > >> digests, Macs, etc. > > > >> > > > >> [image: amazon-corretto-crypto-provider.png] > > > >> > > > >> corretto/amazon-corretto-crypto-provider: The Amazon Corretto Crypto > > > >> Provider is a collection of high-performance cryptographic > > > implementations > > > >> exposed via standard JCA/JCE interfaces. > > > >> <https://github.com/corretto/amazon-corretto-crypto-provider> > > > >> github.com < > https://github.com/corretto/amazon-corretto-crypto-provider > > > > > > > >> <https://github.com/corretto/amazon-corretto-crypto-provider> > > > >> > > > >> > > > >> On Aug 1, 2023, at 9:17 AM, Jim Showalter < > jamesleeshowal...@gmail.com> > > > >> wrote: > > > >> > > > >> I'm still trying to come up to speed on your PR/fork. A lot to > learn! > > > >> > > > >> Ack about licensing. The idea wouldn't be to copy the code, but to > learn > > > >> how to implement message digests and HMAC on top of OpenSSL 3.0.8. > > > >> > > > >> On Tue, Aug 1, 2023, 5:03 AM Gary Gregory <garydgreg...@gmail.com> > > > wrote: > > > >> > > > >> In the short, the best way to help is to provide PRs. > > > >> In more detail, we should probably come up with some kind of a plan > so > > > >> that > > > >> everyone uses their time wisely. > > > >> > > > >> I'll review my branch this week or next and see where that stands, > but > > > >> feel > > > >> free to look at it, use it, PR it, as I might not actually be able > to > > > take > > > >> the time this week. > > > >> > > > >> WRT openssl4j, you CANNOT bring in anything licensed under the LGPL. > > > >> IANAL, > > > >> but our documentation seems clear to me, please see > > > >> https://www.apache.org/legal/resolved.html#category-x > > > >> > > > >> Hope this helps and we can make it work! > > > >> Gary > > > >> > > > >> > > > >> On Mon, Jul 31, 2023, 8:02 PM Jim Showalter < > > > jamesleeshowal...@gmail.com> > > > >> wrote: > > > >> > > > >> A split seems reasonable. We were amazed at how few changes you had > to > > > >> > > > >> make > > > >> > > > >> to support OpenSSL 3.x. The EVPs are very different. But it sounds > like > > > >> there's more to do. > > > >> > > > >> The problem with commons-codec is that it doesn't use OpenSSL or any > > > >> > > > >> other > > > >> > > > >> FIPS-certified cryptographic module. For example, HmacUtils uses > Mac, > > > >> > > > >> which > > > >> > > > >> is supplied by the JRE, which isn't FIPS-certified. > > > >> > > > >> In order to qualify for FedRAMP High, which is table stakes for a > lot of > > > >> corporate and government contracts, FIPS has to be used. It's > mandated > > > by > > > >> statute. No wiggle room. > > > >> > > > >> The promise of bc-fips is that it is FIPS-certified, is a JSP, and > > > >> implements the full JCE. The drawback is that the bc-fips org is > funded > > > >> through donations and consulting, and is always very far behind Java > > > >> releases (it's still on Java 11, for example). > > > >> > > > >> What we need is a full-featured JSP that is based on a > FIPS-certified > > > >> cryptographic module that is implemented using native code and JNI. > > > >> > > > >> We can get FIPS-certified digests and HMAC from openssl4j. What we > need > > > >> from commons-crypto are the ciphers, and they need to be on OpenSSL > > > >> > > > >> 3.0.8. > > > >> > > > >> > > > >> If there's anything we can do to help make that happen, please let > us > > > >> > > > >> know. > > > >> > > > >> > > > >> On Mon, Jul 31, 2023 at 2:58 PM Gary Gregory < > garydgreg...@gmail.com> > > > >> wrote: > > > >> > > > >> Hi Jim, > > > >> > > > >> My branch has not been merged because it does not fully work. It's > > > >> challenging to update the code such that one can use either OpenSSL > > > >> > > > >> 1.1.1 > > > >> > > > >> or 3.0.0 or both. We might need the component split into more than > one > > > >> Maven module. > > > >> > > > >> The name commons-crypto might have been poorly chosen because it's > > > >> > > > >> current > > > >> > > > >> remit is an OpelSSL wrapper. That said there is room for more > features, > > > >> which may mean splitting things up into more than one Maven module. > > > >> > > > >> Commons Code provides more convenience wrappers for JRE message > digests > > > >> including HMAC: > > > >> https://commons.apache.org/proper/commons-codec/apidocs/index.html > > > >> > > > >> Are you looking to wrap or implement HMAC and message digests > > > >> > > > >> differently? > > > >> > > > >> > > > >> Gary > > > >> > > > >> > > > >> On Mon, Jul 31, 2023, 5:04 PM Jim Showalter < > > > >> > > > >> jamesleeshowal...@gmail.com > > > >> > > > >> > > > >> wrote: > > > >> > > > >> We are trying to replace bc-fips ( > > > >> > > > >> https://www.bouncycastle.org/fips-java/) > > > >> > > > >> with a JSP that is based on a cryptographic module that is 1) a > > > >> > > > >> native > > > >> > > > >> library and 2) is certified for FIPS 140-2 ( > > > >> https://csrc.nist.gov/pubs/fips/140-2/upd2/final). > > > >> > > > >> A native library is faster, plus it doesn't entangle the Java > > > >> > > > >> classpath > > > >> > > > >> with restrictions on Java versions or load order the way bc-fips > > > >> > > > >> does. > > > >> > > > >> > > > >> The two available native libraries we're aware of are BoringSSL and > > > >> OpenSSL. > > > >> > > > >> For various reasons, we want to use OpenSSL. > > > >> > > > >> OpenSSL 1.1.1 was only FIPS-certified on RedHat (and they had to > > > >> > > > >> modify > > > >> > > > >> it > > > >> > > > >> to add FIPS support), and the certification expires soon. > > > >> > > > >> OpenSSL 1.1.1 is the version commons-crypto is currently based on. > > > >> > > > >> OpenSSL 3.0.8 is FIPS-certified on a variety of platforms, supports > > > >> > > > >> FIPS > > > >> > > > >> mode natively, and its successor (3.1.x) will be certified for FIPS > > > >> > > > >> 140-3. > > > >> > > > >> > > > >> We're very interested in > > > >> https://github.com/garydgregory/commons-crypto/tree/openssl3, which > > > >> > > > >> adds > > > >> > > > >> support for OpenSSL 3.0.8 to commons-crypto, per > > > >> https://issues.apache.org/jira/browse/CRYPTO-164. > > > >> > > > >> But that PR was never merged, hasn't been touched since December 20, > > > >> > > > >> 2022, > > > >> > > > >> and is currently 92 commits behind the main branch. > > > >> > > > >> What would it take to update that PR with all of the commits since > > > >> > > > >> then, > > > >> > > > >> and get it merged? > > > >> > > > >> Once that's done, we'd be happy to submit a PR to add FIPS mode, per > > > >> https://issues.apache.org/jira/browse/CRYPTO-136. > > > >> > > > >> Also, commons-crypto doesn't support message digests or HMAC. We're > > > >> > > > >> in > > > >> > > > >> the > > > >> > > > >> process of adding HMAC and FIPS mode to > > > >> https://github.com/sfuhrm/openssl4j, > > > >> which has message digests, and targets OpenSSL 3.0.8. > > > >> > > > >> It seems like the message digests and HMAC from openssl4j could be > > > >> > > > >> merged > > > >> > > > >> into commons-crypto, to bring it closer to being a full JCE > > > >> > > > >> implementation. > > > >> > > > >> Is there any interest in seeing that happen? > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
