Thanks Flavio for the investigation. I’ll update the README file to include instructions on supported Java 8 versions. I’m wondering if I have to update the admin docs based on your problems running TLS quorum on a single machine.
Andor > On 2019. Apr 29., at 15:06, Enrico Olivelli <[email protected]> wrote: > > Il lun 29 apr 2019, 13:44 Ted Dunning <[email protected]> ha scritto: > >> Other changes in u211+ substantially improve how Java 8 applications behave >> in containers. I am seeing this more and more with customers. >> >> Combined with the crypto issues, it might be worth a release note >> suggesting that if you are going to compile with Java 1.8, you should use a >> recent release at u211 (u212?) Or above. >> > > +1 for a note on release docs > > > Enrico > > > > >> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <[email protected]> wrote: >> >>> I did a bit more research and it turns out that the crypto.policy option >>> was introduced u151: >>> >>> >> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>> < >>> >> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>>> >>> >>> And started being defined by default with the "unlimited" option in u161: >>> >>> >> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>> < >>> >> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>>> >>> >>> I have installed a more recent version, 1.8.0_211, and it builds fine >> (all >>> tests pass consistently for me). >>> >>> >>> I'm now trying to start an ensemble with ssl enabled locally, but it is >>> failing for me. It looks like the instructions in the admin doc assumes >>> different hosts. I need to look more closely into it to determine what >> not >>> is that I'm doing wrong, but in any case, the instructions do not make it >>> very clear whether one can run locally. >>> >>> -Flavio >>> >>>> On 27 Apr 2019, at 19:28, Patrick Hunt <[email protected]> wrote: >>>> >>>> Odd. I had done my testing on jdk11/macos which is fine. >>>> >>>> I just tried jdk8 and 3 times in a row it's failing with: >>>> [ERROR] SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 » >>>> Timeout Failed t... >>>> >>>> I don't see the error Flavio is seeing. I have never installed special >>>> crypto libraries, etc... just vanilla jdk. >>>> >>>> ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn --version >>>> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; >>>> 2019-04-04T12:00:29-07:00) >>>> Maven home: /usr/local/Cellar/maven/3.6.1/libexec >>>> Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: >>>> /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre >>>> Default locale: en_US, platform encoding: UTF-8 >>>> OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: "mac" >>>> >>>> >>>> 2019-04-27 10:11:51,635 [myid:] - INFO >>>> [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting >>> authorizedID: >>>> super >>>> 2019-04-27 10:11:51,636 [myid:] - INFO >>>> [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL authorization >> for >>>> authorizationID: super >>>> 2019-04-27 10:11:51,813 [myid:] - INFO >>>> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited >>> loop! >>>> 2019-04-27 10:12:21,596 [myid:] - INFO >>>> [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD FAILED >>>> testZKOperationsAfterClientSaslAuthFailure >>>> java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper >>>> server. >>>> at >>>> >>> >> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150) >>>> at >>>> >>> >> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> >>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>> at >>>> >>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at >>>> >>> >> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) >>>> at >>>> >>> >> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) >>>> >>>> >>>> Patrick >>>> >>>> >>>> On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli <[email protected]> >>> wrote: >>>> >>>>> On my local tests I usually don't get the error because I am using >>>>> jdk11 and unlimited strength cryptography is enabled by default >>>>> >>>>> >>> >> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce >>>>> >>>>> In production it depends on the configuration of SSL, if you require >>>>> strong ciphers/big keys you will have problems, but the server won't >>>>> start so you will find soon the problem. >>>>> I think this is not a real issue (for production I mean). >>>>> I see these ways: >>>>> 1) adapt the tests in order to make default jdk8 happy >>>>> 2) tweak the tests enabling "unlimited strenght cryptography" using >>>>> reflection >>>>> 3) just write a line in documentation that says that in order to make >>>>> tests pass you have to enable the flag >>>>> >>>>> That flag is deprecated and enabled by default in modern JREs, so I >>>>> would go for 2) or 3) >>>>> >>>>> I guess that on ASF Jenkins if the JDK8 we are using has the flag >>> turned >>>>> on >>>>> >>>>> Enrico >>>>> >>>>> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar >>>>> <[email protected]> ha scritto: >>>>>> >>>>>> I’m running the tests fine without setting the policy to unlimited: >>>>>> >>>>>> java version "1.8.0_161" >>>>>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12) >>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) >>>>>> >>>>>> Have you tried to run it with a more recent version of Java? >>>>>> >>>>>> Andor >>>>>> >>>>>> >>>>>> >>>>>>> On 2019. Apr 27., at 17:33, Andor Molnar <[email protected]> wrote: >>>>>>> >>>>>>> Good catch, thanks Flavio for reporting this. We need to double >> check >>>>> the tests with Ilya I believe. >>>>>>> >>>>>>> Having tests failure means that you were actually able to _build_ >>>>> ZooKeeper successfully without changing the crypto policy setting. >> Have >>> you >>>>> tried to start an ensemble with Quorum TLS by any chance? That would >> add >>>>> some more color to this issue. >>>>>>> >>>>>>> This might be just a testing issue. >>>>>>> >>>>>>> Regards, >>>>>>> Andor >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <[email protected]> >> wrote: >>>>>>>> >>>>>>>> Hi Enrico, >>>>>>>> >>>>>>>> Here is the info you are requesting: >>>>>>>> >>>>>>>> *Java version* >>>>>>>> >>>>>>>> $ java -version >>>>>>>> java version "1.8.0_152" >>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16) >>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode) >>>>>>>> >>>>>>>> *Test case errors* >>>>>>>> >>>>>>>> I won’t post all of them, I get a good number of errors: >>>>>>>> >>>>>>>> ================================ >>>>>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0, Time >>>>> elapsed: 9.21 s <<< FAILURE! - in >>> org.apache.zookeeper.util.PemReaderTest >>>>>>>> [ERROR] >>>>> >>> >> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest) >>>>> Time elapsed: 1.593 s <<< ERROR! >>>>>>>> java.io.IOException: >>>>> org.bouncycastle.operator.OperatorCreationException: Illegal key size >> or >>>>> default parameters >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>> Illegal key size or default parameters >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size or >>>>> default parameters >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>> >>>>>>>> [ERROR] >>>>> >>> >> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest) >>>>> Time elapsed: 0.004 s <<< ERROR! >>>>>>>> java.lang.Exception: Unexpected exception, >>>>> expected<java.security.GeneralSecurityException> but >>>>> was<java.io.IOException> >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>> Illegal key size or default parameters >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size or >>>>> default parameters >>>>>>>> at >>>>> >>> >> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>> ... >>>>>>>> ================================ >>>>>>>> >>>>>>>> >>>>>>>> *Crypto policy* >>>>>>>> If I uncomment this configuration option: >>>>>>>> >>>>>>>> # Please see the JCA documentation for additional information on >>> these >>>>>>>> # files and formats. >>>>>>>> # crypto.policy=unlimited >>>>>>>> >>>>>>>> in: >>>>>>>> >>>>>>>> $JAVA_HOME/jre/lib/security/java.security >>>>>>>> >>>>>>>> then it all works and I get no error at all. This option controls >>>>> cryptographic strengths according to the documentation, and is present >>>>> because of crypto regulations in different countries. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -Flavio >>>>>>>> >>>>>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli <[email protected]> >>>>> wrote: >>>>>>>>> >>>>>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <[email protected]> ha >>>>> scritto: >>>>>>>>> >>>>>>>>>> I have a clarification question about the RC. To build the RC, I >>>>> had to >>>>>>>>>> enable crypto.policy unlimited in the jre (I'm using build >>>>> 1.8.0_152-b16). >>>>>>>>> >>>>>>>>> >>>>>>>>> Flavio >>>>>>>>> What do you mean with 'build' ? >>>>>>>>> Make tests pass? >>>>>>>>> AFAIK we are not using tweaked jdks in CI builds, so in theory >> there >>>>> is no >>>>>>>>> need. >>>>>>>>> >>>>>>>>> Can you please share your error? >>>>>>>>> >>>>>>>>> Enrico >>>>>>>>> >>>>>>>>> >>>>>>>>> I'm wondering if this is going to be an issue for some users as >> this >>>>> option >>>>>>>>>> is related to import/export regulation. Has anyone looked into it >>>>> and could >>>>>>>>>> clarify it to me, please? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> -Flavio >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <[email protected]> >> wrote: >>>>>>>>>>> >>>>>>>>>>> This is the first stable release of 3.5 branch: 3.5.5. It >> resolves >>>>> 117 >>>>>>>>>> issues, including Maven migration, Quorum TLS, TTL nodes and lots >>>>> of other >>>>>>>>>> performance and stability improvements. >>>>>>>>>>> >>>>>>>>>>> The full release notes is available at: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>> >>> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268 >>>>>>>>>>> >>>>>>>>>>> *** Please download, test and vote by May 3rd 2019, 23:59 UTC+0. >>>>> *** >>>>>>>>>>> >>>>>>>>>>> Source files: >>>>>>>>>>> >>>>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/ >>>>>>>>>>> >>>>>>>>>>> Maven staging repos: >>>>>>>>>>> >>>>>>>>>> >>>>> >>> >> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/ >>>>>>>>>>> >>>>>>>>>> >>>>> >>> >> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/ >>>>>>>>>>> >>>>>>>>>> >>>>> >>> >> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/ >>>>>>>>>>> >>>>>>>>>>> The release candidate tag in git to be voted upon: >>>>> release-3.5.5-rc5 >>>>>>>>>>> >>>>>>>>>>> ZooKeeper's KEYS file containing PGP keys we use to sign the >>>>> release: >>>>>>>>>>> http://www.apache.org/dist/zookeeper/KEYS >>>>>>>>>>> >>>>>>>>>>> Should we release this candidate? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>> >>> >>
