Re: [OT] Custom protocol implementation not found
> Op 14 okt. 2020 om 21:09 heeft Christopher Schultz > het volgende geschreven: > > Maartin, > >> On 10/14/20 09:07, Maarten van den Broek wrote: >> Op 14-10-2020 om 14:10 schreef Rémy Maucherat: >>> On Wed, Oct 14, 2020 at 11:38 AM Maarten van den Broek < >>> mbr...@messagedesign.nl> wrote: >>> I use tomcat 9.0.33 with windows10 home and amazon corretto jdk1.8.0_212. Below a snapshot of two different Connector definitions in server.xml >>> maxThreads="150" SSLEnabled="true" scheme="https" secure="true" protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" clientAuth="false" sslEnabledProtocols="TLSv1.2" minSpareThreads="5" enableLookups="true" disableUploadTimeout="true" keystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" keystorePass="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" /> >>> disableUploadTimeout="true" enableLookups="true" maxThreads="150" minSpareThreads="5" port="443" protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" SSLEnabled="true" scheme="https" secure="true"> >>> certificateKeystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" certificateKeystorePassword="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" certificateKeystoreType="JKS"/> Using the first Connector everything is working fine. Debugging the setKeystorePass method of the class nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the protocol attribute shows that the encrypted password gets decrypted. Using the second connector with the SSLHostConfig element instead of the deprecated attributes debugging shows that the setKeystorePass method is not called and I get errors for the incorrect password of the keystore. What am I doing wrong in migrating to the configuration with the SSLHostConfig element? Sincerely yours,Maarten van den Broek >>> If you simply want to obfuscate server.xml attributes, you should look >>> into >>> the digester property sources instead of engaging in this sort of stuff. >>> One such property source out there: >>> https://github.com/web-servers/tomcat-vault >>> >>> Note: This capability is not included directly into Tomcat itself because >>> it provides no actual extra security. >>> >>> Rémy >> >> Dear Rémy, >> >> Thank you for your swift response. >> >> Customers are happy with this solution because they only need to provide >> these passwords during the first installation and it can be done by the >> owner of the certificate. The key for the en/decryption is in a keystore >> with a password, that can only be obtained by debugging the code. > > So you have the password for the keystore hard-coded into your Java > code? Doesn't that mean it's in revision-control? > >> In a production environment this is in most cases impossible. This >> mechanism is also used to encrypt columns in the database and it is >> easy to reuse for the encryption of the keystore passwords. I prefer >> this over the use of a tomcat vault, because there is increased >> complexity installing tomcat. Alas, it may break with major releases >> and then you have to fix it again, but with your swift help it is no >> problem. > > -chris > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > Hello Chris, The password is generated with an algoritm that can only be cracked by reverse engineering the code from a seed given by the user unknown to the makers of the code. So only reverse engineering the code is a liability. Maarten - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [OT] Custom protocol implementation not found
Maartin, On 10/14/20 09:07, Maarten van den Broek wrote: > Op 14-10-2020 om 14:10 schreef Rémy Maucherat: >> On Wed, Oct 14, 2020 at 11:38 AM Maarten van den Broek < >> mbr...@messagedesign.nl> wrote: >> >>> I use tomcat 9.0.33 with windows10 home and amazon corretto >>> jdk1.8.0_212. >>> >>> Below a snapshot of two different Connector definitions in server.xml >>> >>> >> maxThreads="150" SSLEnabled="true" scheme="https" >>> secure="true" >>> protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" >>> >>> clientAuth="false" sslEnabledProtocols="TLSv1.2" >>> minSpareThreads="5" >>> enableLookups="true" disableUploadTimeout="true" >>> keystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" >>> keystorePass="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" >>> /> >>> >>> >> disableUploadTimeout="true" enableLookups="true" maxThreads="150" >>> minSpareThreads="5" port="443" >>> protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" >>> >>> SSLEnabled="true" scheme="https" secure="true"> >>> >>> >> certificateKeystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" >>> >>> >>> certificateKeystorePassword="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" >>> certificateKeystoreType="JKS"/> >>> >>> >>> >>> Using the first Connector everything is working fine. Debugging the >>> setKeystorePass method of the class >>> nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the >>> protocol attribute shows that the encrypted password gets decrypted. >>> >>> Using the second connector with the SSLHostConfig element instead of the >>> deprecated attributes debugging shows that the setKeystorePass method is >>> not called and I get errors for the incorrect password of the keystore. >>> >>> What am I doing wrong in migrating to the configuration with the >>> SSLHostConfig element? >>> >>> Sincerely yours, Maarten van den Broek >>> >> If you simply want to obfuscate server.xml attributes, you should look >> into >> the digester property sources instead of engaging in this sort of stuff. >> One such property source out there: >> https://github.com/web-servers/tomcat-vault >> >> Note: This capability is not included directly into Tomcat itself because >> it provides no actual extra security. >> >> Rémy > > Dear Rémy, > > Thank you for your swift response. > > Customers are happy with this solution because they only need to provide > these passwords during the first installation and it can be done by the > owner of the certificate. The key for the en/decryption is in a keystore > with a password, that can only be obtained by debugging the code. So you have the password for the keystore hard-coded into your Java code? Doesn't that mean it's in revision-control? > In a production environment this is in most cases impossible. This > mechanism is also used to encrypt columns in the database and it is > easy to reuse for the encryption of the keystore passwords. I prefer > this over the use of a tomcat vault, because there is increased > complexity installing tomcat. Alas, it may break with major releases > and then you have to fix it again, but with your swift help it is no > problem. -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat SecurityListener
Shawn, On 10/12/20 15:59, Beard, Shawn wrote: > Tomcat 9.0.31.0 loads a org.apache.catalina.security.SecurityListener by > default in the catalina.sh file. This comes from server.xml, and it's not "on" by default. > This SecurityListener also sets the UMASK of files to 0027. This has the > effect of any file tomcat creates or the app running in tomcat creates > with permissions or -rw-r- This is untrue: SecurityListener does not set any umask (nor can it). It simply checks the effective umask (as passed into the JVM as a system property) against a configured minimum. > This is causing a problem for us as it prevents certain people from > being able to read log files or read any file the application might > create. Putting these users in the group of the user that tomcat runs as > is not an option. :( > I’ve tried changing the catalina.sh to set the UMASK to something like > 0022 but that prevents tomcat from starting with an error that it has to > me at least as restrictive as 0027. Do not change catalina.sh. Instead, use $CATALINA_BASE/setenv.sh to set the UMASK environment variable (which should work). > I’ve also tried setting the UMASK to 0022 in the setenv.sh with same > results. Good. Well, not good. But I mean, good that you are using setenv.sh. > I’m hesitant to comment out the loading of the security listener in > catalina.sh as I don’t want to disable anything else important that it > may be doing from a security standpoint. It's verifying the minimum umask and that you aren't running as any of the configured OS usernames (default: "root"). I suspect if you disable the SecurityListener you will find that nothing changesL: your umask will still be ignored for some reason. > Does anyone have any ideas as to a workaround? How are you launching Tomcat? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: OpenSSL prompts for key password
Michael, On 10/14/20 12:46, Michael Osipov wrote: > Folks, > > I have recently upgrade a cert and left out the last char of the key > password by accident. > >> # /sbin/init.d/tomcat-smartld start >> Starting Apache Tomcat 8.5... >> Using CATALINA_BASE: /var/opt/tomcat-smartld >> Using CATALINA_HOME: /opt/ports/apache-tomcat-8.5.57 >> Using CATALINA_TMPDIR: /var/opt/tomcat-smartld/temp >> Using JRE_HOME: /opt/java8 >> Using CLASSPATH: >> /opt/ports/apache-tomcat-8.5.57/bin/bootstrap.jar:/opt/ports/apache-tomcat-8.5.57/bin/tomcat-juli.jar >> >> Tomcat started. >> Apache Tomcat 8.5 started. >> # Some of your private key files are encrypted for security reasons. >> In order to read them you have to provide the pass phrases. >> Enter password : >> > > I have seen similar with HTTPd in the past. Since the start is async I > have no option to react on that and it will block the entire config. I > looked briefly in the OpenSSL API, but wasn't really able to find a flag > to inhibit the interactive prompt. > > Does someone know whether we can make this better with libtcnative? What kind of behavior were you hoping for? I'm assuming that some kind of exception would be best for this case (incorrect password). Suppressing the interactive prompt is likely to simply cause the connector to fail to initialize; basically the same thing as throwing an exception in the above case. I searched the Tomcat code and I don't see that sting anywhere, so I suspect it's coming directly from OpenSSL (which is very weird IMHO). mod_ssl has a configurable way to gather this passphrase, presumably to pass it into OpenSSL's read-key function. It would surprise me greatly if an incorrect passphrase would cause the same kind of prompt in httpd. What version of OpenSSL are you using? Have you tried any other versions? -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: FW: HTTP2: memory filled up fast on increasing the connections to 1000/2000 (Embedded tomcat 9.0.38)
Arshiya, On 10/14/20 01:23, Arshiya Shariff wrote: > Please find the answers in-line Mark. > > Http2 requests with message payload of 34KB are pumped from JMeter > at 20 TPS with 700 connections to an application with Embedded tomcat > - 9.0.39 (max-Threads : 200, all other values are the tomcat > defaults) > >> What does that URL do with the POSTed content? Ignore it? Read it >> from an InputStream? Read it via getParameter()? > > The posted content is read via BufferedReader reader = > request.getReader() and processed asynchronously How... exactly? > Is JMeter run on the same machine as Tomcat? > JMeter is run from a different machine. > > Do you use the JMeter GUI or the command line? > Launched via Command line (JMeter heap increased to 10 GB ) > > What are the specs of the server(s) being used? > The server is a VM with 12 CPUs and 120 GB RAM > > Please let us know if you require more details. This would probabyl be easier if you'd just provide a test-case: a sample (simple!) web application which reproduces what you are reporting. -chris > -Original Message- > From: Mark Thomas > Sent: Monday, October 12, 2020 7:28 PM > To: users@tomcat.apache.org > Subject: Re: HTTP2: memory filled up fast on increasing the connections to > 1000/2000 (Embedded tomcat 9.0.38) > > On 12/10/2020 08:02, Arshiya Shariff wrote: >> Hi Mark , >> >> The issue is reproduced with version 9.0.39 as well. Max threads in Tomcat >> is 200. >> >> Please find the case: >> Client:JMeter 5.2.1 (With http2 plugin) >> TPS: around 20 >> No of users from JMeter : 700 >> Message payload size: 6 KB to 34 KB >> Loop: Infinite >> We let the loop run infinitely and see the java.lang.StackOverflowError >> trace printed multiple times in the log within few minutes of starting the >> test. > > POSTing to what URL? > > What does that URL do with the POSTed content? Ignore it? Read it from an > InputStream? Read it via getParameter()? > > Is JMeter run on the same machine as Tomcat? > > Do you use the JMeter GUI or the command line? > > What are the specs of the server(s) being used? > > You need to provide the exact steps to recreate this issue on a clean install > of Tomcat 9.0.39 as provided by the ASF. > > Mark > > >> Please help us with this . What is the impact of StackOverflowError ? >> >> Thanks and Regards >> Arshiya Shariff >> >> -Original Message- >> From: Mark Thomas >> Sent: Friday, October 9, 2020 5:31 PM >> To: users@tomcat.apache.org >> Subject: Re: HTTP2: memory filled up fast on increasing the >> connections to 1000/2000 (Embedded tomcat 9.0.38) >> >> On 09/10/2020 12:32, Arshiya Shariff wrote: >>> Hi, >>> >>> Mark , with the test runs that I performed over clean 9.0.x branch I was >>> not able to reproduce this. >> >> Good. But I'd really like to understand why... >> >>> But with 9.0.38 and the jars built from 9.0.x with hash: >>> c8ec2d4cde3a31b0e9df9a30e7915d77ba725545 , with 700 or 1000 users >>> (connections) and on sending 1000 Requests per second (or even lesser) , >>> payload of 16K from JMeter I can see that this Exception occurs within few >>> minutes of starting the test . The maxThreads configured in tomcat is 200 . >>> >>> How often do you see these errors in your test run? >>> Randomly, at times 2 or 3 such traces. >> >> OK. Definitely a timing issue then. >> >>> Do you have the other end of that stack trace? >>> It is only the two lines that is recursively printed till the end about >>> ~500 times in one trace : >>> at >>> org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper$NioOperationState.run(NioEndpoint.java:1511) >>> at >>> org.apache.tomcat.util.net.SocketWrapperBase$VectoredIOCompletionHand >>> l >>> er.completed(SocketWrapperBase.java:1100) >> >> Doesn't tell me much unfortunately. >> >>> I see the trace starting with : >>> Exception in thread "http-nio-x.y.z-1090-exec-107" >>> java.lang.StackOverflowError >>> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:446) >>> at org.apache.tomcat.util.net.NioChannel.read(NioChannel.java:174) >>> at >>> org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper$NioOperationState.run(NioEndpoint.java:1468) >>> at >>> org.apache.tomcat.util.net.SocketWrapperBase$VectoredIOCompletionHand >>> l >>> er.completed(SocketWrapperBase.java:1100) >>> >>> (OR) >>> >>> Exception in thread "http-nio-x.y.z-1090-exec-87" >>> java.lang.StackOverflowError >>> at sun.nio.ch.IOVecWrapper.get(IOVecWrapper.java:96) >>> at sun.nio.ch.IOUtil.read(IOUtil.java:240) >>> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:440) >>> at org.apache.tomcat.util.net.NioChannel.read(NioChannel.java:174) >>> at >>> org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper$NioOperationState.run(NioEndpoint.java:1468) >>> at >>>
OpenSSL prompts for key password
Folks, I have recently upgrade a cert and left out the last char of the key password by accident. # /sbin/init.d/tomcat-smartld start Starting Apache Tomcat 8.5... Using CATALINA_BASE: /var/opt/tomcat-smartld Using CATALINA_HOME: /opt/ports/apache-tomcat-8.5.57 Using CATALINA_TMPDIR: /var/opt/tomcat-smartld/temp Using JRE_HOME:/opt/java8 Using CLASSPATH: /opt/ports/apache-tomcat-8.5.57/bin/bootstrap.jar:/opt/ports/apache-tomcat-8.5.57/bin/tomcat-juli.jar Tomcat started. Apache Tomcat 8.5 started. # Some of your private key files are encrypted for security reasons. In order to read them you have to provide the pass phrases. Enter password : I have seen similar with HTTPd in the past. Since the start is async I have no option to react on that and it will block the entire config. I looked briefly in the OpenSSL API, but wasn't really able to find a flag to inhibit the interactive prompt. Does someone know whether we can make this better with libtcnative? Michael - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Custom protocol implementation not found
Op 14-10-2020 om 14:10 schreef Rémy Maucherat: On Wed, Oct 14, 2020 at 11:38 AM Maarten van den Broek < mbr...@messagedesign.nl> wrote: I use tomcat 9.0.33 with windows10 home and amazon corretto jdk1.8.0_212. Below a snapshot of two different Connector definitions in server.xml Using the first Connector everything is working fine. Debugging the setKeystorePass method of the class nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the protocol attribute shows that the encrypted password gets decrypted. Using the second connector with the SSLHostConfig element instead of the deprecated attributes debugging shows that the setKeystorePass method is not called and I get errors for the incorrect password of the keystore. What am I doing wrong in migrating to the configuration with the SSLHostConfig element? Sincerely yours,Maarten van den Broek If you simply want to obfuscate server.xml attributes, you should look into the digester property sources instead of engaging in this sort of stuff. One such property source out there: https://github.com/web-servers/tomcat-vault Note: This capability is not included directly into Tomcat itself because it provides no actual extra security. Rémy Dear Rémy, Thank you for your swift response. Customers are happy with this solution because they only need to provide these passwords during the first installation and it can be done by the owner of the certificate. The key for the en/decryption is in a keystore with a password, that can only be obtained by debugging the code. In a production environment this is in most cases impossible. This mechanism is also used to encrypt columns in the database and it is easy to reuse for the encryption of the keystore passwords. I prefer this over the use of a tomcat vault, because there is increased complexity installing tomcat. Alas, it may break with major releases and then you have to fix it again, but with your swift help it is no problem. Maarten - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Custom protocol implementation not found
On Wed, Oct 14, 2020 at 11:38 AM Maarten van den Broek < mbr...@messagedesign.nl> wrote: > I use tomcat 9.0.33 with windows10 home and amazon corretto jdk1.8.0_212. > > Below a snapshot of two different Connector definitions in server.xml > > maxThreads="150" SSLEnabled="true" scheme="https" > secure="true" > protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" > clientAuth="false" sslEnabledProtocols="TLSv1.2" > minSpareThreads="5" > enableLookups="true" disableUploadTimeout="true" > keystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" > keystorePass="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" > /> > > disableUploadTimeout="true" enableLookups="true" maxThreads="150" > minSpareThreads="5" port="443" > protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" > SSLEnabled="true" scheme="https" secure="true"> > > certificateKeystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" > > certificateKeystorePassword="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" > certificateKeystoreType="JKS"/> > > > > Using the first Connector everything is working fine. Debugging the > setKeystorePass method of the class > nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the > protocol attribute shows that the encrypted password gets decrypted. > > Using the second connector with the SSLHostConfig element instead of the > deprecated attributes debugging shows that the setKeystorePass method is > not called and I get errors for the incorrect password of the keystore. > > What am I doing wrong in migrating to the configuration with the > SSLHostConfig element? > > Sincerely yours,Maarten van den Broek > If you simply want to obfuscate server.xml attributes, you should look into the digester property sources instead of engaging in this sort of stuff. One such property source out there: https://github.com/web-servers/tomcat-vault Note: This capability is not included directly into Tomcat itself because it provides no actual extra security. Rémy
Re: Custom protocol implementation not found
On 14/10/2020 10:38, Maarten van den Broek wrote: > I use tomcat 9.0.33 with windows10 home and amazon corretto jdk1.8.0_212. > Using the first Connector everything is working fine. Debugging the > setKeystorePass method of the class > nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the > protocol attribute shows that the encrypted password gets decrypted. > > Using the second connector with the SSLHostConfig element instead of the > deprecated attributes debugging shows that the setKeystorePass method is > not called and I get errors for the incorrect password of the keystore. > > What am I doing wrong in migrating to the configuration with the > SSLHostConfig element? I do wonder a) what risk(s) you are attempting to mitigate with this and b) where that custom connector obtains the necessary pass-phrase to decrypt the supplied value. I am assuming you have extended the existing Http11Nio2Protocol implementation and over-ridden setKeystorePass() as that won't get called when an SSLHostConfig element is explicitly configured. Based on the assumptions above, the following approach should work: - override init() - iterate over the results of findSslHostConfigs() - for each SSLHostConfig instance - call getCertificateKeystorePassword() - decrypt it - call setCertificateKeystorePassword() If you have multiple certificates per host you'll need to iterate over the nested SSLHostConfigCertificate instances rather than use the short-cut methods above that work with the default certificate instance. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Custom protocol implementation not found
I use tomcat 9.0.33 with windows10 home and amazon corretto jdk1.8.0_212. Below a snapshot of two different Connector definitions in server.xml maxThreads="150" SSLEnabled="true" scheme="https" secure="true" protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" clientAuth="false" sslEnabledProtocols="TLSv1.2" minSpareThreads="5" enableLookups="true" disableUploadTimeout="true" keystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" keystorePass="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" /> disableUploadTimeout="true" enableLookups="true" maxThreads="150" minSpareThreads="5" port="443" protocol="nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol" SSLEnabled="true" scheme="https" secure="true"> certificateKeystoreFile="C:/Users/Maarten/Certificaten/gm_messagedesign_nl2020.jks" certificateKeystorePassword="ZURV/6aoh/mLRxJGFhnvEpVZ7PoL72h3" certificateKeystoreType="JKS"/> Using the first Connector everything is working fine. Debugging the setKeystorePass method of the class nl.messagedesign.tomcatlib.EncryptedPassword_Http11Nio2Protocol in the protocol attribute shows that the encrypted password gets decrypted. Using the second connector with the SSLHostConfig element instead of the deprecated attributes debugging shows that the setKeystorePass method is not called and I get errors for the incorrect password of the keystore. What am I doing wrong in migrating to the configuration with the SSLHostConfig element? Sincerely yours, Maarten van den Broek - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
