RE: How to encrypt db password in tomcat context.xml

[Pesonen, Harri]

I have implemented a Tomcat vault as well, it is basically a simplified version 
of

https://github.com/web-servers/tomcat-vault

My version does not have keystore, so it is much easier to use.
It would be great if Tomcat would have this functionality built-in somehow.

-Harri

-Original Message-
From: jonmcalexan...@wellsfargo.com.INVALID 
 
Sent: maanantai 29. kesäkuuta 2020 17.25
To: users@tomcat.apache.org
Subject: RE: How to encrypt db password in tomcat context.xml

-Original Message-
From: Rémy Maucherat  
Sent: Monday, June 29, 2020 1:58 AM
To: Tomcat Users List 
Subject: Re: How to encrypt db password in tomcat context.xml

On Mon, Jun 29, 2020 at 8:03 AM Carsten Klein  wrote:

> Hi Jürgen and Olaf,
>
> I can really understand Jürgen's intentions. The core problem is not 
> security but administrators and so called security panels in 
> "professional" (non-open source related) companies. I really know this 
> from my own experiences. Maybe that's a German problem, since Germans 
> are said to be over-correct? Sometimes, that turns into paranoia...
> (I'm from Germany, so I know this circumstances quite well, sounds 
> like Jürgen is German as well...)
>
> True is, that there are administrators, which have very little 
> knowledge of software in general and security. Those tend to stick to 
> their personal categorical rules, which often are far off from what is 
> considered sensible by real IT and software professionals.
> Furthermore, there are "security" panels, working out policies for a 
> company. These often only consist of people with very little *real* IT 
> an security knowledge.
>
> The (sad) point is, that the policies passed by such a council are 
> actually valid and no one ever again asks whether these make sense or 
> are *correct* from a security professional's point of view. Changing 
> user passwords on a regular basis (e.g. after 90 days) still today is 
> one prominent example of that.
>
> So, in order to make Tomcat fit into such "professional" company 
> environments easily (w/o requiring people to implement their own data 
> source class), it may be a good idea to add such a "encrypted 
> password" feature to Tomcat. Consider this as pure "syntactic sugar"
> and keep in mind, that it's NOT a security feature (need to emphasis 
> that in the docs).
>
> My idea is, that *all* passwords read by Tomcat MAY be 
> encrypted/obfuscated with a small number of algorithms. The algorithm 
> applied to the password could be prefixed like Jürgen suggested:
>
> password="+duk6<7v@LD#"(plain, no encryption)
> password="base64:K2R1azY8N3ZATEQj" (base64 obfuscation)
> password="3des:hkph5ewjEwv70CvTB16v/w=="   (3DES with hard-coded key,
> expressed as base64 string)
>
> The decoding algorithm could be implemented in a common util method 
> String decodePassword(String password) in Tomcat, so it could easily 
> be called from all those places at which Tomcat actually reads a 
> password.
>
> Also, a small separate tool should be provided, which encodes such 
> passwords (like htpasswd does for httpd). However, it should be 
> sufficient to just print the encoded password to standard out. Then, 
> the user is responsible for copy and pasting it into the config file.
>
> I offer my help for writing such an enhancement, since I believe that 
> it's a way to make Tomcat more out-of-the-box usable in such 
> "professional" company's environments (for some people it may be the 
> only way).
>
> Again, I know this is NOT a security feature as it adds no extra 
> security to Tomcat. However, I may make some administrators and CEOs 
> happy, that are solely guided by questionable policies they don't 
> understand.
>
> Some ideas on that?
>

> The Tomcat committers' decision has always been to block inclusion of such a 
> feature, for the reasons explained in the wiki page 
> here https://cwiki.apache.org/confluence/display/TOMCAT/Password
> As a result, your proposal will not be considered.

> If you want a ready to use tool, go here:
> https://github.com/web-servers/tomcat-vault

> Rémy

I have been working with the tomcat-vault and so far am finding it promising. 
One caveat with this, is you can't set variables in the catalina.properties 
that pull the values from the vault, it only works in xml files, so you have to 
reference the vault item in server.xml, context.xml, etc.

>
> Carsten
>
>
> On 28.06.2020 23:49, Jürgen Weber wrote:
> > I'd just put some nice password as byte[] into Tomcat's source code 
> > and provide a way to have passwords in the configs encrypted with 
> > that nice password.
> >
> >> Use properties replacement so that in the xml config you have 
> >> ${db.password} and in conf/catalina.properties you put the password 
> >> there.
> >
> > So one could have samething like db.pass=3des: in 
> > catalina.properties
> >
> > Greetings, Juergen
> >
> > Am So., 28. Juni 2020 um 21:19 Uhr schrieb Olaf Kock 
> > >:
> >>
> >>
> >> On 28.06.2

8.5.20 - java.security.KeyStoreException: Cannot get key bytes, not PKCS#8 encoded

[Pesonen, Harri]

Hello, there is this problem in 8.5.20, that is already fixed for the coming 
8.5.21:

https://bz.apache.org/bugzilla/show_bug.cgi?id=61451

What is the latest version that does not have this problem? Is it 8.5.16? If I 
understand correctly, this problem was introduced in 8.5.18 (not released):

"Enable TLS connectors to use Java key stores that contain multiple keys where 
each key has a separate password. Based on a patch by Frank Taffelt. (markt)"

-Harri



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: How to stop Tomcat if servlet fails to initialize

[Pesonen, Harri]

8.5.13. Maybe it is supported in 9 only.

-Harri

-Original Message-
From: Mark Thomas [mailto:ma...@apache.org] 
Sent: 22. kesäkuuta 2017 13:46
To: Tomcat Users List 
Subject: Re: How to stop Tomcat if servlet fails to initialize

On 22/06/17 11:44, Pesonen, Harri wrote:
> Hello, I tried adding throwOnFailure to conf/context.xml, but it did not 
> work, I only got this:
> 
> 10:34:23.582 (14) WRN> 
> (org.apache.catalina.startup.SetContextPropertiesRule#begin) 
> [SetContextPropertiesRule]{Context} Setting property 'throwOnFailure' to 
> 'true' did not find a matching property.
> 
> Maybe I did it wrong:
> 
> 

I was working from the code rather than testing but this should work.
What version are you using?

Mark


> 
> -Harri
> 
> -Original Message-
> From: Mark Thomas [mailto:ma...@apache.org] 
> Sent: 21. kesäkuuta 2017 17:28
> To: Tomcat Users List 
> Subject: Re: How to stop Tomcat if servlet fails to initialize
> 
> On 21/06/17 13:33, Pesonen, Harri wrote:
>> Hello, if one servlet fails to initialize, meaning that 
>> ServletContextListener.contextInitialized() throws exception, then Tomcat 
>> still starts (but without the servlet). 
> 
> If ServletContextListener.contextInitialized(), the web application will
> not start.
> 
> Servlets are not initialised until first used so they can't prevent an
> application from starting. The one exception is Servlets configured for
> "load on startup". Any failure of these servlets will cause the
> application to fail to start.
> 
>> I found out that if servlet calls System.exit(1), then Tomcat dies. Is this 
>> the only way to make this happen?
>> This does not work:
>> -Dorg.apache.catalina.startup.EXIT_ON_INIT_FAILURE=true
> 
> You can try setting the (completely undocumented) throwOnFailure
> attribute for the Context, Host, Engine, Service and Server.
> 
> The Context you can set in conf/context.xml so it applies to all web
> applications.
> 
> Mark
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: How to stop Tomcat if servlet fails to initialize

[Pesonen, Harri]

Hello, I tried adding throwOnFailure to conf/context.xml, but it did not work, 
I only got this:

10:34:23.582 (14) WRN> 
(org.apache.catalina.startup.SetContextPropertiesRule#begin) 
[SetContextPropertiesRule]{Context} Setting property 'throwOnFailure' to 'true' 
did not find a matching property.

Maybe I did it wrong:



-Harri

-Original Message-
From: Mark Thomas [mailto:ma...@apache.org] 
Sent: 21. kesäkuuta 2017 17:28
To: Tomcat Users List 
Subject: Re: How to stop Tomcat if servlet fails to initialize

On 21/06/17 13:33, Pesonen, Harri wrote:
> Hello, if one servlet fails to initialize, meaning that 
> ServletContextListener.contextInitialized() throws exception, then Tomcat 
> still starts (but without the servlet). 

If ServletContextListener.contextInitialized(), the web application will
not start.

Servlets are not initialised until first used so they can't prevent an
application from starting. The one exception is Servlets configured for
"load on startup". Any failure of these servlets will cause the
application to fail to start.

> I found out that if servlet calls System.exit(1), then Tomcat dies. Is this 
> the only way to make this happen?
> This does not work:
> -Dorg.apache.catalina.startup.EXIT_ON_INIT_FAILURE=true

You can try setting the (completely undocumented) throwOnFailure
attribute for the Context, Host, Engine, Service and Server.

The Context you can set in conf/context.xml so it applies to all web
applications.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

How to stop Tomcat if servlet fails to initialize

[Pesonen, Harri]

Hello, if one servlet fails to initialize, meaning that 
ServletContextListener.contextInitialized() throws exception, then Tomcat still 
starts (but without the servlet). 
I found out that if servlet calls System.exit(1), then Tomcat dies. Is this the 
only way to make this happen?
This does not work:
-Dorg.apache.catalina.startup.EXIT_ON_INIT_FAILURE=true

-Harri

RE: server.xml password encryption instead of plain text

[Pesonen, Harri]

It is possible to use Windows certificate store like this:



You have to enter keyAlias that matches the subject of the certificate in 
Windows user's personal certificates. Then you don't need to enter password at 
all.

-Harri

-Original Message-
From: John Palmer [mailto:johnpalm...@gmail.com] 
Sent: 25. toukokuuta 2017 17:01
To: Tomcat Users List 
Subject: Re: server.xml password encryption instead of plain text

I haven't tested it yet, but if you're on a Windows platform you MAY be
able to tell Tomcat to use the Windows Certificate Store (an thus NOT have
a password in server.xml) by adding something like this to the Java Options:
-Djavax.net.ssl.trustStoreProvider=SunMSCAPI
-Djavax.net.ssl.trustStoreType=Windows-ROOT
-Djavax.net.ssl.trustStore=NONE
-Djavax.net.ssl.keyStoreProvider=SunMSCAPI
-Djavax.net.ssl.keyStoreType=Windows-MY
-Djavax.net.ssl.keyStore=NONE

.. and this may not work at all..


On Thu, May 25, 2017 at 7:46 AM, Vidyadhar  wrote:

> On Thu, 25 May 2017 at 6:01 PM, Dhaval Jaiswal 
> wrote:
>
> > How can we avoid defining plain text password in server.xml​ or is there
> a
> > way i can encrypt the password in server.xml. ​
> >
> There are couple of examples on https://wiki.apache.org/
> tomcat/FAQ/Password
> --
> Regards,
> Vidyadhar
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: How to monitor Tomcat connectors?

[Pesonen, Harri]

I looked at Tomcat code, and it seems that it is not possible. Bootstrap has 
private static instance:

public final class Bootstrap {

/**
 * Daemon object used by main.
 */
private static Bootstrap daemon = null;

And it is not possible to get this.
If it was, then still it would not possible to get the Catalina instance, which 
is also private:

private Object catalinaDaemon = null;

But if I would get the Catalina instance somehow, then I probably could get 
Server and connectors and everything.

I wonder if it would be possible to add such static method to Bootstrap:

public static Object getCatalina() {
if (daemon != null) {
return daemon.catalinaDaemon;
}
return null;
}

But I have a feeling that there must be some better way of doing this.

-Harri

-Original Message-
From: Pesonen, Harri [mailto:harri.peso...@sap.com] 
Sent: 15. toukokuuta 2017 18:10
To: Tomcat Users List 
Subject: How to monitor Tomcat connectors?

Hello, what would be the best way to find out if any of Tomcat connectors have 
failed to initialize at startup?
They could fail for many reasons, like when the port is already in use, or 
keystore is missing etc.
Now Tomcat prints the error in log, but I would like to find out 
programmatically if any of the connectors have failed.
Or alternatively, fail the Tomcat and shutdown it if any connectors fail.

I see that Connector has getState():

https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/util/LifecycleBase.html#getState()

public LifecycleState getState()
Obtain the current state of the source component.
Specified by:
getState in interface Lifecycle
Returns:
The current state of the source component.

DESTROYED
DESTROYING
FAILED
INITIALIZED
INITIALIZING
NEW
STARTED
STARTING
STARTING_PREP
STOPPED
STOPPING
STOPPING_PREP

If the state is any of (DESTROYED, DESTROYING, FAILED) then I think that it has 
failed.
Then question is, how to get the connectors? Is there some static method to get 
Tomcat server instance and then connectors?

I am starting Tomcat using static Bootstrap method main:
https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/startup/Bootstrap.html#main(java.lang.String[])

I wonder how to get the started Tomcat instance?

-Harri

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

How to monitor Tomcat connectors?

[Pesonen, Harri]

Hello, what would be the best way to find out if any of Tomcat connectors have 
failed to initialize at startup?
They could fail for many reasons, like when the port is already in use, or 
keystore is missing etc.
Now Tomcat prints the error in log, but I would like to find out 
programmatically if any of the connectors have failed.
Or alternatively, fail the Tomcat and shutdown it if any connectors fail.

I see that Connector has getState():

https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/util/LifecycleBase.html#getState()

public LifecycleState getState()
Obtain the current state of the source component.
Specified by:
getState in interface Lifecycle
Returns:
The current state of the source component.

DESTROYED
DESTROYING
FAILED
INITIALIZED
INITIALIZING
NEW
STARTED
STARTING
STARTING_PREP
STOPPED
STOPPING
STOPPING_PREP

If the state is any of (DESTROYED, DESTROYING, FAILED) then I think that it has 
failed.
Then question is, how to get the connectors? Is there some static method to get 
Tomcat server instance and then connectors?

I am starting Tomcat using static Bootstrap method main:
https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/startup/Bootstrap.html#main(java.lang.String[])

I wonder how to get the started Tomcat instance?

-Harri

Security question

[Pesonen, Harri]

Hello,
the following lists Tomcat versions 8.5.0 - 8.5.12, does it mean that the 
problem has been fixed in 8.5.13 and later?

https://nvd.nist.gov/vuln/detail/CVE-2017-5651

I assume that it has been fixed, as 8.5.13 readme has:


  *   [Fix:]  60918: Fix 
sendfile processing error that could lead to subsequent requests experiencing 
an IllegalStateException. (markt)
  *   [Fix:]  Improve sendfile handling when requests are pipelined. (markt)

-Harri

RE: Tomcat WebSocket does not always send asynchronous messages

[Pesonen, Harri]

andler implements SendHandler {

private final Semaphore semaphore;

private SemaphoreSendHandler(Semaphore semaphore) {
this.semaphore = semaphore;
}

@Override
public void onResult(SendResult result) {
semaphore.release();
}
}
}
}

-Harri

-Original Message-
From: Pesonen, Harri [mailto:harri.peso...@sap.com] 
Sent: 8. maaliskuuta 2017 16:32
To: Tomcat Users List 
Subject: RE: Tomcat WebSocket does not always send asynchronous messages

Hello, and sorry for top-posting, I don't know how to configure Outlook to do 
it differently.

I was finally able to run your test. I had a lot of trouble doing it:
* did not have SVN, downloaded TortoiseSVN
* tried to open the project in IDEA, but failed miserably, I really hope that 
there was pom.xml
* was able to build whole Tomcat and test using ant command line, but it took 
so long, had to abort
* was not able to run this single test with ant:

Testsuite: org.apache.tomcat.websocket.server.TestAsyncMessages.java
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec

Caused an ERROR
org.apache.tomcat.websocket.server.TestAsyncMessages.java
java.lang.ClassNotFoundException: 
org.apache.tomcat.websocket.server.TestAsyncMessages.java

* but was able to make the Eclipse project with "ant ide-eclipse"
* was able to run the unit test in Eclipse:

08-Mar-2017 14:14:40.538 INFO [main] 
org.apache.catalina.startup.LoggingBaseTest.setUp Starting test case 
[testAsyncTiming]
08-Mar-2017 14:14:42.676 INFO [main] org.apache.coyote.AbstractProtocol.init 
Initializing ProtocolHandler ["http-nio-127.0.0.1-auto-1"]
08-Mar-2017 14:14:42.778 INFO [main] 
org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared 
selector for servlet write/read
08-Mar-2017 14:14:42.800 INFO [main] 
org.apache.catalina.core.StandardService.startInternal Starting service Tomcat
08-Mar-2017 14:14:42.802 INFO [main] 
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: 
Apache Tomcat/@VERSION@
08-Mar-2017 14:14:43.213 INFO [main] org.apache.coyote.AbstractProtocol.start 
Starting ProtocolHandler [http-nio-127.0.0.1-auto-1-54783]
Sent Hello message, waiting for data
Expected diff < 500,000 but was [6054390], count [2]
Expected diff < 500,000 but was [1015710], count [14]
Expected diff < 500,000 but was [642270], count [25]
Expected diff < 500,000 but was [1712852], count [26]
Expected diff < 500,000 but was [595293], count [41]
Expected diff < 500,000 but was [792673], count [61]
Expected diff < 500,000 but was [799777], count [62]
Expected diff < 500,000 but was [531738], count [68]
Expected diff < 500,000 but was [532922], count [76]
Expected diff < 500,000 but was [673851], count [98]
Expected diff < 500,000 but was [538054], count [133]
Expected diff < 500,000 but was [747276], count [158]
Expected diff < 500,000 but was [794646], count [262]
Expected diff < 500,000 but was [1290461], count [263]
Expected diff < 500,000 but was [1013341], count [296]
Expected diff < 500,000 but was [582267], count [311]
Expected diff < 500,000 but was [1377703], count [337]
Expected diff < 500,000 but was [1698245], count [338]
Expected diff < 500,000 but was [1303488], count [424]
Expected diff < 500,000 but was [965181], count [425]
Expected diff < 500,000 but was [534896], count [455]
Expected diff < 500,000 but was [847938], count [458]
Expected diff < 500,000 but was [883862], count [473]
Expected diff < 500,000 but was [1026368], count [475]
Expected diff < 500,000 but was [1096241], count [476]
Expected diff < 500,000 but was [518710], count [481]
Expected diff < 500,000 but was [1053607], count [482]
Expected diff < 500,000 but was [641481], count [500]
Expected diff < 500,000 but was [565292], count [512]
Expected diff < 500,000 but was [808857], count [556]
Expected diff < 500,000 but was [643455], count [653]
Expected diff < 500,000 but was [508447], count [670]
Expected diff < 500,000 but was [960839], count [671]
Expected diff < 500,000 but was [954918], count [683]
Expected diff < 500,000 but was [601215], count [749]
Expected diff < 500,000 but was [561345], count [752]
Expected diff < 500,000 but was [688062], count [935]
Expected diff < 500,000 but was [1405730], count [937]
Expected diff < 500,000 but was [1414415], count [938]
Expected diff < 500,000 but was [1284935], count [941]
Expected diff < 500,000 but was [516737], count [995]
Expected diff < 500,000 but was [587398], count [1067]
Expected diff < 500,000 but was [946233], count [1079]
Expected diff < 500,000 but was [5403041], count [1114]
Expected diff < 500,000 but was [1181114], count [1115]
Expected diff < 500,000 but was [554239], count [1118]
Expected diff < 500,000 but was [1437706], count 

RE: Tomcat WebSocket does not always send asynchronous messages

[Pesonen, Harri]

0 but was [525816], count [1394]
08-Mar-2017 14:15:09.251 INFO [main] org.apache.coyote.AbstractProtocol.pause 
Pausing ProtocolHandler ["http-nio-127.0.0.1-auto-1-54783"]
08-Mar-2017 14:15:09.266 INFO [main] 
org.apache.catalina.core.StandardService.stopInternal Stopping service Tomcat

Then I was able to run the test also in IDEA, by importing Eclipse project and 
modifying the libraries.

I changed the big message size to 16384 bytes and small size to 1904 bytes (the 
most common problem that we have seen).
So need to add the following to servlet configuration:



org.apache.tomcat.websocket.binaryBufferSize
16384



org.apache.tomcat.websocket.textBufferSize
16384


Like this:

ctx.addParameter(Constants.
BINARY_BUFFER_SIZE_SERVLET_CONTEXT_INIT_PARAM, "16384");
ctx.addParameter(org.apache.tomcat.websocket.server.Constants.
TEXT_BUFFER_SIZE_SERVLET_CONTEXT_INIT_PARAM, "16384");
wsContainer.setDefaultMaxBinaryMessageBufferSize(16384);
wsContainer.setDefaultMaxTextMessageBufferSize(16384);

Then I changed the test logic so that there are only 2 messages, one big and 
one small, but it fails randomly like the original test.
Also added test for maximum delay for the big message:

} else if (diff > 6000) {
System.out.println("Expected diff < 60ms but was [" + 
diff + "], count [" + count + "]");
fail = true;
}

Sent Hello message, waiting for data
Expected diff < 500,000 but was [718458], count [9]
Expected diff < 500,000 but was [1224142], count [17]
Expected diff < 500,000 but was [1102952], count [19]
Expected diff < 500,000 but was [663587], count [21]
Expected diff < 500,000 but was [537659], count [43]
Expected diff < 500,000 but was [3158452], count [73]
Expected diff < 500,000 but was [582662], count [75]
Expected diff < 500,000 but was [825437], count [121]
Expected diff < 500,000 but was [605953], count [129]
Expected diff < 500,000 but was [1092293], count [143]
Expected diff < 500,000 but was [552660], count [145]
Expected diff < 500,000 but was [7926329], count [177]
Expected diff < 500,000 but was [507657], count [189]
Expected diff < 500,000 but was [913468], count [193]
Expected diff < 500,000 but was [560160], count [229]
Expected diff < 500,000 but was [1748774], count [319]
Expected diff < 500,000 but was [529764], count [321]
Expected diff < 500,000 but was [692009], count [381]
Expected diff < 500,000 but was [556213], count [389]
Expected diff < 500,000 but was [2727772], count [403]
Expected diff < 500,000 but was [937154], count [469]
Expected diff < 500,000 but was [544370], count [513]
Expected diff < 500,000 but was [1018473], count [575]
Expected diff < 500,000 but was [518711], count [601]
Expected diff < 500,000 but was [885441], count [613]
Expected diff < 500,000 but was [2075633], count [645]
Expected diff < 500,000 but was [606347], count [715]
Expected diff < 500,000 but was [561345], count [721]
Expected diff < 500,000 but was [947812], count [889]
Expected diff < 500,000 but was [513974], count [923]
Expected diff < 60ms but was [65475211], count [960]
Expected diff > 40ms but was [35346555], count [962]

So the delay is at most 2 milliseconds, which is acceptable.

I think that you understood my problem quite well. It seems that the problem 
can't be reproduced in this test.
I also tried adding sendPing() there but it did not have any effect.
We do not use compression extension.
Thanks for the test! :-)

-Harri

-Original Message-
From: Mark Thomas [mailto:ma...@apache.org] 
Sent: 7. maaliskuuta 2017 23:58
To: Tomcat Users List 
Subject: Re: Tomcat WebSocket does not always send asynchronous messages

On 07/03/17 14:55, Mark Thomas wrote:
> On 07/03/17 11:03, Mark Thomas wrote:
>> On 07/03/17 08:28, Pesonen, Harri wrote:
>>> Hello, we have a problem that Tomcat WebSocket does not always send 
>>> asynchronous messages. This problem is random, and it has been reproduced 
>>> in Tomcat 8.5.6 and 8.5.11. Synchronized operations work fine, and also the 
>>> asynchronous operations work except in one special case. When there is a 
>>> big message that we want to send to client, we split it into 16 kB packets 
>>> for technical reasons, and then we send them very quickly after each other 
>>> using
>>>
>>> /**
>>> * Initiates the asynchronous transmission of a binary message. This method 
>>> returns before the message
>>> * is transmitted. Developers provide a callback to be notified when the 
>>> message has been
>>> * tra

Tomcat WebSocket does not always send asynchronous messages

[Pesonen, Harri]

Hello, we have a problem that Tomcat WebSocket does not always send 
asynchronous messages. This problem is random, and it has been reproduced in 
Tomcat 8.5.6 and 8.5.11. Synchronized operations work fine, and also the 
asynchronous operations work except in one special case. When there is a big 
message that we want to send to client, we split it into 16 kB packets for 
technical reasons, and then we send them very quickly after each other using

/**
* Initiates the asynchronous transmission of a binary message. This method 
returns before the message
* is transmitted. Developers provide a callback to be notified when the message 
has been
* transmitted. Errors in transmission are given to the developer in the 
SendResult object.
*
* @param data   the data being sent, must not be {@code null}.
* @param handler the handler that will be notified of progress, must not be 
{@code null}.
* @throws IllegalArgumentException if either the data or the handler are {@code 
null}.
*/
void sendBinary(ByteBuffer data, SendHandler handler);

Because there can be only one ongoing write to socket, we use Semaphore that is 
released on the SendHandler callback:

public void onResult(javax.websocket.SendResult result) {
semaphore.release();

So the code to send is actually:

semaphore.acquireUninterruptibly();
async.sendBinary(buf, asyncHandler);

This works fine in most cases. But when we send one 16 kB packet and then 
immediately one smaller packet (4 kB), then randomly the smaller packet is not 
actually sent, but only after we call

async.sendPing(new byte[0])

in another thread. sendPing() is called every 20 seconds to keep the WebSocket 
connection alive. This means that the last packet gets extra delay on client, 
which varies between 0 - 20 seconds.

We have an easy workaround to the problem. If we call flushBatch() after each 
sendBinary(), then it works great, but this means that the sending is not 
actually asynchronous, because flushBatch() is synchronous.
Also we should not be forced to call flushBatch(), because we are not enabling 
batching. Instead we make sure that it is disabled:

if (async.getBatchingAllowed()) {
async.setBatchingAllowed(false);

So the working code is:

semaphore.acquireUninterruptibly();
async.sendBinary(buf, asyncHandler);
async.flushBatch();

Normally the code works fine without flushBatch(), if there is delay between 
the messages, but when we send the messages right after each other, then the 
last small message is not always sent immediately.
I looked at the Apache WebSocket code, but it was not clear to me what is 
happening there.
Any ideas what is going on here? Any ideas how I could troubleshoot this more?
Thanks,

-Harri