Arg, idiotic line wrapping removed: https://pastebin.com/raw/nPczrrd8
Gary On Wed, Jun 29, 2022 at 8:49 PM Gary Gregory <garydgreg...@gmail.com> wrote: > > FWIW, I just checked out the rel/commons-crypto-1.1.0 on macOS and ran > 'mvn clean verify' and everything built just fine. > > The Maven ant-run output was: > > [INFO] --- maven-antrun-plugin:1.8:run (make) @ commons-crypto --- > [INFO] Executing tasks > > make: > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > -classpath target/classes -o > target/jni-classes/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.h > org.apache.commons.crypto.random.OpenSslCryptoRandomNative > [exec] gcc -arch x86_64 -Ilib/inc_mac > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > -I"src/main/native/org/apache/commons/crypto/" > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > -I"target/jni-classes/org/apache/commons/crypto/cipher" > -I"target/jni-classes/org/apache/commons/crypto/random" -c > src/main/native/org/apache/commons/crypto/random/OpenSslCryptoRandomNative.c > -o target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > -classpath target/classes -o > target/jni-classes/org/apache/commons/crypto/cipher/OpenSslNative.h > org.apache.commons.crypto.cipher.OpenSslNative > [exec] gcc -arch x86_64 -Ilib/inc_mac > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > -I"src/main/native/org/apache/commons/crypto/" > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > -I"target/jni-classes/org/apache/commons/crypto/cipher" > -I"target/jni-classes/org/apache/commons/crypto/random" -c > src/main/native/org/apache/commons/crypto/cipher/OpenSslNative.c -o > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o > [exec] "/usr/local/Cellar/openjdk@8/1.8.0+322/bin/javah" -force > -classpath target/classes -o > target/jni-classes/org/apache/commons/crypto/OpenSslInfoNative.h > org.apache.commons.crypto.OpenSslInfoNative > [exec] gcc -arch x86_64 -Ilib/inc_mac > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > -I"src/main/native/org/apache/commons/crypto/" > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > -I"target/jni-classes/org/apache/commons/crypto/cipher" > -I"target/jni-classes/org/apache/commons/crypto/random" > -DVERSION='"1.1.0"' -DPROJECT_NAME='"Apache Commons Crypto"' > -I"target/jni-classes/org/apache/commons/crypto" -c > src/main/native/org/apache/commons/crypto/OpenSslInfoNative.c -o > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o > [exec] gcc -arch x86_64 -Ilib/inc_mac > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/include" -O2 -fPIC > -mmacosx-version-min=10.7 -fvisibility=hidden -I/usr/local/include > -I/usr/local/opt/openssl/include -Ilib/include -I/usr/include > -I"/usr/local/Cellar/openjdk@8/1.8.0+322/libexec/openjdk.jdk/Contents/Home/include/darwin" > -I"target/jni-classes/org/apache/commons/crypto/cipher" > -I"target/jni-classes/org/apache/commons/crypto/random" -o > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslCryptoRandomNative.o > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslNative.o > target/commons-crypto-1.1.0-Mac-x86_64/OpenSslInfoNative.o -dynamiclib > -L/usr/local/lib > [exec] strip -x > target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib > [exec] cp target/commons-crypto-1.1.0-Mac-x86_64/libcommons-crypto.jnilib > target/classes/org/apache/commons/crypto/native/Mac/x86_64/libcommons-crypto.jnilib > > Gary > > On Wed, Jun 29, 2022 at 8:48 PM Gary Gregory <garydgreg...@gmail.com> wrote: > > > > I agree with Alex. > > > > Forget LibreSSL. Commons Crypto is for OpenSSL, nice, simple, and > > tight. Last time I checked I had an antique version of LibreSSL on my > > mac years ago, I just installed OpenSSL and never looked back. > > > > Gary > > > > On Wed, Jun 29, 2022 at 8:11 PM Alex Remily <alex.rem...@gmail.com> wrote: > > > > > > <The question is whether to do anything about the previously undetected > > > issues with JNA on macOS and Windows, and if so, whether this needs to > > > be done for the current or a later release. If we don't fix this > > > release, obviously it needs some mention in the release notes.> > > > > > > I wouldn't characterize the issues running against LibreSSL as > > > "undetected". I also don't see this as an issue with Mac or Windows, but > > > with LibreSSL. Install any supported OpenSSL library on any supported > > > architecture (to include Mac and Windows) and all commons crypto > > > functionality is available. I first encountered the rand issue you > > > describe years ago when I was becoming familiar with commons crypto. I > > > did > > > a little research, discovered that I was running LibreSSL (and an old > > > version at that), installed a supported version of OpenSSL and forgot > > > all about it until this thread. I don't think it's unreasonable to expect > > > users to install a supported version of OpenSSL on a supported > > > architecture > > > as a precondition of using commons crypto. I think it could become > > > cumbersome to try and support every vendor default *SSL install. That > > > said, I don't have a problem committing this particular "fix" to the > > > baseline, particularly since you have already done the work. I just don't > > > think that the project should obligate itself to do so or advertise > > > LibreSSL (or any other non-OpenSSL branch) support as such. I'm > > > advocating > > > a "use at your own risk" approach to anything but OpenSSL proper. I agree > > > that we should update the documentation to reflect whatever we move > > > forward > > > with. > > > > > > Anyway, that's my $0.02 worth. > > > > > > Alex > > > > > > On Wed, Jun 29, 2022 at 6:14 PM sebb <seb...@gmail.com> wrote: > > > > > > > On Wed, 29 Jun 2022 at 18:06, Gary Gregory <garydgreg...@gmail.com> > > > > wrote: > > > > > > > > > > We cannot remove support for Windows and macOS, that's silly. > > > > > > > > AFAICT that means we must support the different set of function names > > > > in LibreSSL. > > > > Note that we only currently use a small proportion of them. > > > > > > > > > I have not followed all the branches and commits, so I'm not sure what > > > > the > > > > > current problems are, but I know I was able to release all OSs last go > > > > > around. I don't see why we need to rip out anything as fundamental. > > > > > > > > Well, I have tried running the Crypto and OpenSslJna main classes on > > > > macOS and Windows, and they both fail with the 1.1.0 release. > > > > > > > > With current master, Crypto succeeds, but OpenSslJna fails to find > > > > ENGINE_load_rdrand on both macOS and Windows. > > > > (The job step succeeds, because the code catches the exception) > > > > > > > > It looks like the test which would have exposed at least one of the > > > > issues was never enabled because of a failures on macOS; this hid the > > > > same problem on Windows. > > > > > > > > I am not suggesting we rip out anything at this point. > > > > > > > > The question is whether to do anything about the previously undetected > > > > issues with JNA on macOS and Windows, and if so, whether this needs to > > > > be done for the current or a later release. If we don't fix this > > > > release, obviously it needs some mention in the release notes. > > > > > > > > Sebb > > > > https://github.com/apache/commons-crypto/actions/runs/2586011129 > > > > > Gary > > > > > > > > > > On Wed, Jun 29, 2022, 12:00 sebb <seb...@gmail.com> wrote: > > > > > > > > > > > On Wed, 29 Jun 2022 at 16:11, Alex Remily <alex.rem...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > I agree with Gary that we just don't support LibreSSL. To my > > > > knowledge > > > > > > > we've never advertised LibreSSL support, so I don't see it as an > > > > issue. > > > > > > > > > > > > In that case AFAICT we will have to drop *all* support for macOS and > > > > > > Windows, as they both seem to default to LibreSSL. > > > > > > > > > > > > Note that adding support for LibreSSL was much easier for JNI, as it > > > > > > uses far fewer methods. > > > > > > Rather than checking the version, I changed the code to try OpenSSL > > > > > > 1.1 names first, then a fallback. > > > > > > That happens to work for 1.0.x and 1.1.x and LibreSSL. > > > > > > > > > > > > If you want to try it out, compare 16345bc (old) with 3ee3f65 (new) > > > > > > macOS fails on 16345bc because it now uses LibreSSL which has a > > > > > > different mix of names. > > > > > > > > > > > > I think it's vital we support JNI as far as possible (and the code > > > > > > already does with commit 3ee3f65). > > > > > > > > > > > > However JNA is more of a backstop, so the fact that it has stopped > > > > > > working for macOS and Windows is less of an issue. > > > > > > > > > > > > But I don't think we can say we are not supporting LibreSSL at all. > > > > > > > > > > > > > On Wed, Jun 29, 2022 at 10:21 AM sebb <seb...@gmail.com> wrote: > > > > > > > > > > > > > > > On Wed, 29 Jun 2022 at 14:17, Gary Gregory > > > > > > > > <garydgreg...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > The simple solution is to keep doing what we do now: only > > > > > > > > > support > > > > > > OpenSSL > > > > > > > > > and not whatever Apple does with LibreSSL which may or may not > > > > be up > > > > > > to > > > > > > > > > date. > > > > > > > > > > > > > > > > I think this also affects Windows. > > > > > > > > > > > > > > > > I don't have Windows box at present, but I have seen this on GH > > > > > > Actions. > > > > > > > > > > > > > > > > If you have a WIndows build, perhaps you could try these tests: > > > > > > > > > > > > > > > > mvn -q exec:java > > > > -D"exec.mainClass=org.apache.commons.crypto.Crypto" > > > > > > > > mvn -q exec:java > > > > > > > > -D"exec.mainClass=org.apache.commons.crypto.jna.OpenSslJna" > > > > > > > > > > > > > > > > The first one should show the SSL details; on GH the output > > > > includes: > > > > > > > > > > > > > > > > OpenSSL library loaded OK, version: 0x20000000 > > > > > > > > OpenSSL library info: LibreSSL 3.0.2 > > > > > > > > Additional OpenSSL_version(n) details: > > > > > > > > 4: OPENSSLDIR: "C:/Windows/libressl/ssl" > > > > > > > > > > > > > > > > The second test crashes with: > > > > > > > > java.lang.UnsatisfiedLinkError: Error looking up function > > > > > > > > 'ENGINE_load_rdrand': The specified procedure could not be > > > > > > > > found. > > > > > > > > isEnabled(): false > > > > > > > > > > > > > > > > initialisationError(): java.lang.UnsatisfiedLinkError: Error > > > > looking > > > > > > > > up function 'ENGINE_load_rdrand': The specified procedure could > > > > not be > > > > > > > > found. > > > > > > > > at com.sun.jna.Function.<init>(Function.java:252) > > > > > > > > ... > > > > > > > > > > > > > > > > It would certainly be easier to ignore the problem for now. > > > > > > > > > > > > > > > > > Gary > > > > > > > > > > > > > > > > > > On Wed, Jun 29, 2022, 06:59 sebb <seb...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > It looks like macOS 10.5+ and Windows (latest) use LibreSSL > > > > > > > > > > by > > > > > > default > > > > > > > > > > rather than OpenSSL. > > > > > > > > > > > > > > > > > > > > The LibreSSL API does not have the same functions as either > > > > 1.0.2 > > > > > > or > > > > > > > > > > 1.1.1, so needs its own JNA class. In particular it looks > > > > > > > > > > like > > > > > > > > > > ENGINE_load_rdrand is not present, nor is > > > > > > > > > > OpenSSL_version_num; > > > > > > 1.0.2 > > > > > > > > > > has the former only, and 1.1.1 has the latter only. > > > > > > > > > > > > > > > > > > > > This makes it hard to support JNA with the current design. > > > > > > > > > > It would require another OpenSsl<version>NativeJna class, > > > > > > > > > > and > > > > the > > > > > > > > > > parent class OpenSslNativeJna would need to use yet another > > > > > > condition > > > > > > > > > > in each of its methods. > > > > > > > > > > > > > > > > > > > > This is quite tedious and error-prone. > > > > > > > > > > > > > > > > > > > > Seems to me it would be better to use something like a set > > > > > > > > > > of > > > > > > > > > > singleton classes that implement the required methods. The > > > > > > appropriate > > > > > > > > > > one can then be initialised and used by OpenSslNativeJna to > > > > field > > > > > > the > > > > > > > > > > actual methods. i.e. replace the conditional logic with a > > > > static > > > > > > > > > > reference to the correct API interface instance. This should > > > > only > > > > > > > > > > affect non-public classes. > > > > > > > > > > > > > > > > > > > > Any other suggestions? > > > > > > > > > > > > > > > > > > > > Sebb > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
