Makes sense. I’ve tested it on a single machine with the same cert/key for all instances: keystore / truststore only contained a single entry and it worked fine. We’ve also tested on multiple instances with multiple keys / instances which also worked fine.
Let me give it another go with single machine / multiple certs combo. I might need to modify the docs to emphasize keys must be generated on a per machine basis, not ZK instance. Regards, Andor > On 2019. Apr 29., at 16:53, Flavio Junqueira <f...@apache.org> wrote: > > I'm also +1 for adding a comment to the release notes (thanks for the > suggestion, Ted). Updating the readme makes sense, but the release notes will > be the main source to indicate that we require a specific or later version of > Java from that particular release. My preference would be to update the > release notes. > > As for running TLS on a single node, have you been able to do it? I haven't > had a chance to look further into it throughout my day, so if anyone has > successfully done it and can share some instructions, it would help me. > Otherwise, I'll keep investigating once I have a chance. To be specific, I > created the keystore, certificate and truststore files according to > instructions, but the instructions assume that the aliases are different when > it comes to populating the truststore. At that point, I had to get creative > and I have tried a couple of options that didn't work. Either way, I think > that being able to run locally and documenting is desirable, although not a > blocker. If I can get it right, then I can write a gist describing it that we > can use as a reference until we properly document it. > > -Flavio > >> On 29 Apr 2019, at 15:42, Andor Molnar <an...@apache.org> wrote: >> >> Thanks Flavio for the investigation. I’ll update the README file to include >> instructions on supported Java 8 versions. >> I’m wondering if I have to update the admin docs based on your problems >> running TLS quorum on a single machine. >> >> Andor >> >> >> >>> On 2019. Apr 29., at 15:06, Enrico Olivelli <eolive...@gmail.com> wrote: >>> >>> Il lun 29 apr 2019, 13:44 Ted Dunning <ted.dunn...@gmail.com> ha scritto: >>> >>>> Other changes in u211+ substantially improve how Java 8 applications behave >>>> in containers. I am seeing this more and more with customers. >>>> >>>> Combined with the crypto issues, it might be worth a release note >>>> suggesting that if you are going to compile with Java 1.8, you should use a >>>> recent release at u211 (u212?) Or above. >>>> >>> >>> +1 for a note on release docs >>> >>> >>> Enrico >>> >>> >>> >>> >>>> On Mon, Apr 29, 2019, 11:43 AM Flavio Junqueira <f...@apache.org> wrote: >>>> >>>>> I did a bit more research and it turns out that the crypto.policy option >>>>> was introduced u151: >>>>> >>>>> >>>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>>>> < >>>>> >>>> https://www.oracle.com/technetwork/java/javase/8u151-relnotes-3850493.html >>>>>> >>>>> >>>>> And started being defined by default with the "unlimited" option in u161: >>>>> >>>>> >>>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>>>> < >>>>> >>>> https://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html >>>>>> >>>>> >>>>> I have installed a more recent version, 1.8.0_211, and it builds fine >>>> (all >>>>> tests pass consistently for me). >>>>> >>>>> >>>>> I'm now trying to start an ensemble with ssl enabled locally, but it is >>>>> failing for me. It looks like the instructions in the admin doc assumes >>>>> different hosts. I need to look more closely into it to determine what >>>> not >>>>> is that I'm doing wrong, but in any case, the instructions do not make it >>>>> very clear whether one can run locally. >>>>> >>>>> -Flavio >>>>> >>>>>> On 27 Apr 2019, at 19:28, Patrick Hunt <ph...@apache.org> wrote: >>>>>> >>>>>> Odd. I had done my testing on jdk11/macos which is fine. >>>>>> >>>>>> I just tried jdk8 and 3 times in a row it's failing with: >>>>>> [ERROR] SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure:176 » >>>>>> Timeout Failed t... >>>>>> >>>>>> I don't see the error Flavio is seeing. I have never installed special >>>>>> crypto libraries, etc... just vanilla jdk. >>>>>> >>>>>> ⌂102% [phunt:~/Downloads/z/apache-zookeeper-3.5.5] 3s $ mvn --version >>>>>> Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; >>>>>> 2019-04-04T12:00:29-07:00) >>>>>> Maven home: /usr/local/Cellar/maven/3.6.1/libexec >>>>>> Java version: 1.8.0_201, vendor: Oracle Corporation, runtime: >>>>>> /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/jre >>>>>> Default locale: en_US, platform encoding: UTF-8 >>>>>> OS name: "mac os x", version: "10.14.4", arch: "x86_64", family: "mac" >>>>>> >>>>>> >>>>>> 2019-04-27 10:11:51,635 [myid:] - INFO >>>>>> [NIOWorkerThread-6:SaslServerCallbackHandler@136] - Setting >>>>> authorizedID: >>>>>> super >>>>>> 2019-04-27 10:11:51,636 [myid:] - INFO >>>>>> [NIOWorkerThread-6:ZooKeeperServer@1170] - adding SASL authorization >>>> for >>>>>> authorizationID: super >>>>>> 2019-04-27 10:11:51,813 [myid:] - INFO >>>>>> [SessionTracker:SessionTrackerImpl@163] - SessionTrackerImpl exited >>>>> loop! >>>>>> 2019-04-27 10:12:21,596 [myid:] - INFO >>>>>> [main:JUnit4ZKTestRunner$LoggedInvokeMethod@98] - TEST METHOD FAILED >>>>>> testZKOperationsAfterClientSaslAuthFailure >>>>>> java.util.concurrent.TimeoutException: Failed to connect to ZooKeeper >>>>>> server. >>>>>> at >>>>>> >>>>> >>>> org.apache.zookeeper.test.ClientBase$CountdownWatcher.waitForConnected(ClientBase.java:150) >>>>>> at >>>>>> >>>>> >>>> org.apache.zookeeper.SaslAuthTest.testZKOperationsAfterClientSaslAuthFailure(SaslAuthTest.java:176) >>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>>> at >>>>>> >>>>> >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>> at >>>>>> >>>>> >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>>> at >>>>>> >>>>> >>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) >>>>>> at >>>>>> >>>>> >>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) >>>>>> >>>>>> >>>>>> Patrick >>>>>> >>>>>> >>>>>> On Sat, Apr 27, 2019 at 8:54 AM Enrico Olivelli <eolive...@gmail.com> >>>>> wrote: >>>>>> >>>>>>> On my local tests I usually don't get the error because I am using >>>>>>> jdk11 and unlimited strength cryptography is enabled by default >>>>>>> >>>>>>> >>>>> >>>> https://www.oracle.com/technetwork/java/javase/documentation/jdk11-readme-5097204.html#jce >>>>>>> >>>>>>> In production it depends on the configuration of SSL, if you require >>>>>>> strong ciphers/big keys you will have problems, but the server won't >>>>>>> start so you will find soon the problem. >>>>>>> I think this is not a real issue (for production I mean). >>>>>>> I see these ways: >>>>>>> 1) adapt the tests in order to make default jdk8 happy >>>>>>> 2) tweak the tests enabling "unlimited strenght cryptography" using >>>>>>> reflection >>>>>>> 3) just write a line in documentation that says that in order to make >>>>>>> tests pass you have to enable the flag >>>>>>> >>>>>>> That flag is deprecated and enabled by default in modern JREs, so I >>>>>>> would go for 2) or 3) >>>>>>> >>>>>>> I guess that on ASF Jenkins if the JDK8 we are using has the flag >>>>> turned >>>>>>> on >>>>>>> >>>>>>> Enrico >>>>>>> >>>>>>> Il giorno sab 27 apr 2019 alle ore 17:48 Andor Molnar >>>>>>> <an...@apache.org> ha scritto: >>>>>>>> >>>>>>>> I’m running the tests fine without setting the policy to unlimited: >>>>>>>> >>>>>>>> java version "1.8.0_161" >>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_161-b12) >>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode) >>>>>>>> >>>>>>>> Have you tried to run it with a more recent version of Java? >>>>>>>> >>>>>>>> Andor >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 2019. Apr 27., at 17:33, Andor Molnar <an...@apache.org> wrote: >>>>>>>>> >>>>>>>>> Good catch, thanks Flavio for reporting this. We need to double >>>> check >>>>>>> the tests with Ilya I believe. >>>>>>>>> >>>>>>>>> Having tests failure means that you were actually able to _build_ >>>>>>> ZooKeeper successfully without changing the crypto policy setting. >>>> Have >>>>> you >>>>>>> tried to start an ensemble with Quorum TLS by any chance? That would >>>> add >>>>>>> some more color to this issue. >>>>>>>>> >>>>>>>>> This might be just a testing issue. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Andor >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 2019. Apr 27., at 16:09, Flavio Junqueira <f...@apache.org> >>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Enrico, >>>>>>>>>> >>>>>>>>>> Here is the info you are requesting: >>>>>>>>>> >>>>>>>>>> *Java version* >>>>>>>>>> >>>>>>>>>> $ java -version >>>>>>>>>> java version "1.8.0_152" >>>>>>>>>> Java(TM) SE Runtime Environment (build 1.8.0_152-b16) >>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM (build 25.152-b16, mixed mode) >>>>>>>>>> >>>>>>>>>> *Test case errors* >>>>>>>>>> >>>>>>>>>> I won’t post all of them, I get a good number of errors: >>>>>>>>>> >>>>>>>>>> ================================ >>>>>>>>>> [ERROR] Tests run: 64, Failures: 0, Errors: 16, Skipped: 0, Time >>>>>>> elapsed: 9.21 s <<< FAILURE! - in >>>>> org.apache.zookeeper.util.PemReaderTest >>>>>>>>>> [ERROR] >>>>>>> >>>>> >>>> testLoadCertificateFromKeyStore[1](org.apache.zookeeper.util.PemReaderTest) >>>>>>> Time elapsed: 1.593 s <<< ERROR! >>>>>>>>>> java.io.IOException: >>>>>>> org.bouncycastle.operator.OperatorCreationException: Illegal key size >>>> or >>>>>>> default parameters >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>>>> Illegal key size or default parameters >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size or >>>>>>> default parameters >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadCertificateFromKeyStore(PemReaderTest.java:125) >>>>>>>>>> >>>>>>>>>> [ERROR] >>>>>>> >>>>> >>>> testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword[1](org.apache.zookeeper.util.PemReaderTest) >>>>>>> Time elapsed: 0.004 s <<< ERROR! >>>>>>>>>> java.lang.Exception: Unexpected exception, >>>>>>> expected<java.security.GeneralSecurityException> but >>>>>>> was<java.io.IOException> >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>> Caused by: org.bouncycastle.operator.OperatorCreationException: >>>>>>> Illegal key size or default parameters >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>> Caused by: java.security.InvalidKeyException: Illegal key size or >>>>>>> default parameters >>>>>>>>>> at >>>>>>> >>>>> >>>> org.apache.zookeeper.util.PemReaderTest.testLoadEncryptedPrivateKeyFromKeyStoreWithWrongPassword(PemReaderTest.java:93) >>>>>>>>>> ... >>>>>>>>>> ================================ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Crypto policy* >>>>>>>>>> If I uncomment this configuration option: >>>>>>>>>> >>>>>>>>>> # Please see the JCA documentation for additional information on >>>>> these >>>>>>>>>> # files and formats. >>>>>>>>>> # crypto.policy=unlimited >>>>>>>>>> >>>>>>>>>> in: >>>>>>>>>> >>>>>>>>>> $JAVA_HOME/jre/lib/security/java.security >>>>>>>>>> >>>>>>>>>> then it all works and I get no error at all. This option controls >>>>>>> cryptographic strengths according to the documentation, and is present >>>>>>> because of crypto regulations in different countries. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> -Flavio >>>>>>>>>> >>>>>>>>>>> On 27 Apr 2019, at 15:52, Enrico Olivelli <eolive...@gmail.com> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Il sab 27 apr 2019, 14:18 Flavio Junqueira <f...@apache.org> ha >>>>>>> scritto: >>>>>>>>>>> >>>>>>>>>>>> I have a clarification question about the RC. To build the RC, I >>>>>>> had to >>>>>>>>>>>> enable crypto.policy unlimited in the jre (I'm using build >>>>>>> 1.8.0_152-b16). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Flavio >>>>>>>>>>> What do you mean with 'build' ? >>>>>>>>>>> Make tests pass? >>>>>>>>>>> AFAIK we are not using tweaked jdks in CI builds, so in theory >>>> there >>>>>>> is no >>>>>>>>>>> need. >>>>>>>>>>> >>>>>>>>>>> Can you please share your error? >>>>>>>>>>> >>>>>>>>>>> Enrico >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I'm wondering if this is going to be an issue for some users as >>>> this >>>>>>> option >>>>>>>>>>>> is related to import/export regulation. Has anyone looked into it >>>>>>> and could >>>>>>>>>>>> clarify it to me, please? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> -Flavio >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 25 Apr 2019, at 15:10, Andor Molnar <an...@apache.org> >>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> This is the first stable release of 3.5 branch: 3.5.5. It >>>> resolves >>>>>>> 117 >>>>>>>>>>>> issues, including Maven migration, Quorum TLS, TTL nodes and lots >>>>>>> of other >>>>>>>>>>>> performance and stability improvements. >>>>>>>>>>>>> >>>>>>>>>>>>> The full release notes is available at: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310801&version=12343268 >>>>>>>>>>>>> >>>>>>>>>>>>> *** Please download, test and vote by May 3rd 2019, 23:59 UTC+0. >>>>>>> *** >>>>>>>>>>>>> >>>>>>>>>>>>> Source files: >>>>>>>>>>>>> >>>>>>> https://dist.apache.org/repos/dist/dev/zookeeper/zookeeper-3.5.5-rc5/ >>>>>>>>>>>>> >>>>>>>>>>>>> Maven staging repos: >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/parent/3.5.5/ >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper-jute/3.5.5/ >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >>>> https://repository.apache.org/content/groups/staging/org/apache/zookeeper/zookeeper/3.5.5/ >>>>>>>>>>>>> >>>>>>>>>>>>> The release candidate tag in git to be voted upon: >>>>>>> release-3.5.5-rc5 >>>>>>>>>>>>> >>>>>>>>>>>>> ZooKeeper's KEYS file containing PGP keys we use to sign the >>>>>>> release: >>>>>>>>>>>>> http://www.apache.org/dist/zookeeper/KEYS >>>>>>>>>>>>> >>>>>>>>>>>>> Should we release this candidate? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >> >
