[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #35 from Christopher Schultz  ---
(In reply to Michael Osipov from comment #34)
> Awesome, can you retry with Java 8?

This is Oracle 1.8.0_181.
Selector Provider: sun.nio.ch.KQueueSelectorProvider@16e15b65

16,10461
32,5245
64,2696
128,1403
256,728
512,400
1024,223
2048,131
4096,71
8192,50
16384,28
146988,27

Curve is the same: https://jsfiddle.net/mrw3o96t/

> But the results are quite surprising because I have expected similar results
> as with FreeBSD due to the BSD userland and kernel parts in macOS.

The Macos kernel is quite a bit different than the BSD kernel(s). It's really a
mach kernel with a BSD interface slapped on top so they didn't have to write
all their own custom userland.

Note that this is running the code from Mark's test source from github. All I
did was replace the logging and add an optional command-line argument to make
it easier to run repeatedly. In light of Mark's comment #27, I'd like to
confirm that this is the correct code to run. It's setting the socket's send
buffer size and not the appWriteBufSize.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63875] Tomcat 8.5.46, APR/libtcnative crashes

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63875

Christopher Schultz  changed:

   What|Removed |Added

 Resolution|FIXED   |---
 Status|RESOLVED|REOPENED

--- Comment #18 from Christopher Schultz  ---
Was this caused by custom native code of custom Java code?

I would argue that, unless you went into the C stack/heap with a hatchet,
(ab)using the tcnative code shouldn't cause a JVM crash. Exceptions, dropped
connections, etc., sure. But the tcnative code should be protecting itself
against improper usage.

Can you please share your custom code surrounding your use of
AprLifecycleListener? Maybe there really is a bug, here.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63875] Tomcat 8.5.46, APR/libtcnative crashes

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63875

sachin.pip...@globallogic.com changed:

   What|Removed |Added

 Status|NEEDINFO|RESOLVED
 Resolution|--- |FIXED

--- Comment #17 from sachin.pip...@globallogic.com ---
Marking resolved this issue, as the issue was with the custom code while
creating the AprLifeccycleListner the reference was getting overridden.

Thanks to all who supported during the time.

Thanks
Sachin

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch 8.5.x updated: Properly calculate dynamic parts of ErrorReportValve response for TestHttp2InitialConnection

2019-11-13 Thread michaelo
This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this push:
 new f056ef1  Properly calculate dynamic parts of ErrorReportValve response 
for TestHttp2InitialConnection
f056ef1 is described below

commit f056ef130fd629fe82f34b52fc9688d74324d7e7
Author: Michael Osipov 
AuthorDate: Wed Nov 13 23:12:09 2019 +0100

Properly calculate dynamic parts of ErrorReportValve response for 
TestHttp2InitialConnection
---
 .../coyote/http2/TestHttp2InitialConnection.java | 20 +---
 webapps/docs/changelog.xml   |  6 ++
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java 
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
index a362a19..1a99c3a 100644
--- a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
+++ b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
@@ -115,7 +115,7 @@ public class TestHttp2InitialConnection extends 
Http2TestBase {
  * This will vary depending on where the test is run due to:
  * - The length of the version string that appears once in the 
error
  *   page
- * - The status header uses a UTF-8 EM dash. When running in an IDE
+ * - The status header uses a UTF-8 EN dash. When running in an IDE
  *   the UTF-8 properties files will be used directly rather than
  *   after native2ascii conversion.
  *
@@ -124,10 +124,24 @@ public class TestHttp2InitialConnection extends 
Http2TestBase {
 int serverInfoLength = 
ServerInfo.getServerInfo().getBytes().length;
 StringManager sm = StringManager.getManager(
 ErrorReportValve.class.getPackage().getName(), 
Locale.ENGLISH);
+String reason = sm.getString("http." + 
testData.getExpectedStatus() + ".reason");
+int descriptionLength = sm.getString("http." + 
testData.getExpectedStatus() + ".desc")
+.getBytes(StandardCharsets.UTF_8).length;
 int statusHeaderLength = sm
-.getString("errorReportValve.statusHeader", "", "")
+.getString("errorReportValve.statusHeader",
+String.valueOf(testData.getExpectedStatus()), 
reason)
+.getBytes(StandardCharsets.UTF_8).length;
+int typeLabelLength = sm.getString("errorReportValve.type")
+.getBytes(StandardCharsets.UTF_8).length;
+int statusReportLabelLength = 
sm.getString("errorReportValve.statusReport")
+.getBytes(StandardCharsets.UTF_8).length;
+int descriptionLabelLength = 
sm.getString("errorReportValve.description")
 .getBytes(StandardCharsets.UTF_8).length;
-int len = 708 + serverInfoLength + statusHeaderLength * 2;
+// 196 bytes is the static length of the pure HTML code from the 
ErrorReportValve
+int len = 196 + org.apache.catalina.util.TomcatCSS.TOMCAT_CSS
+.getBytes(StandardCharsets.UTF_8).length +
+typeLabelLength + statusReportLabelLength + 
descriptionLabelLength +
+descriptionLength + serverInfoLength + statusHeaderLength 
* 2;
 String contentLength = String.valueOf(len);
 return getResponseBodyFrameTrace(streamId,
 testData.getExpectedStatus(), "text/html;charset=utf-8",
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index 8a5e592..a2f9c89 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -135,6 +135,12 @@
 with ping_mode=I, the CPING
 message will not always be followed by the start of a request. (markt)
   
+  
+Properly calculate all dynamic parts of the ErrorReportValve response
+on the fly in
+org.apache.coyote.http2.TestHttp2InitialConnection.
+(michaelo)
+  
 
   
   


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [tomcat] branch master updated: Fix test fallout of BZ 63905

2019-11-13 Thread Michael Osipov

Am 2019-11-13 um 17:48 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 11/9/19 05:58, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git
repository.

michaelo pushed a commit to branch master in repository
https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this
push: new ccd6518  Fix test fallout of BZ 63905 ccd6518 is
described below

commit ccd6518024ab4f8c0add42eb2b20941a6b237f5d Author: Michael
Osipov  AuthorDate: Sat Nov 9 11:35:22 2019
+0100

Fix test fallout of BZ 63905 ---
test/org/apache/coyote/http2/TestHttp2InitialConnection.java | 2
+- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git
a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
index 2157eae..a362a19 100644 ---
a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java +++
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java @@
-127,7 +127,7 @@ public class TestHttp2InitialConnection extends
Http2TestBase { int statusHeaderLength = sm
.getString("errorReportValve.statusHeader", "", "")
.getBytes(StandardCharsets.UTF_8).length; -int len =
1073 + serverInfoLength + statusHeaderLength * 2; +int
len = 708 + serverInfoLength + statusHeaderLength * 2;


Why not fix this properly with:

int len =
org.apache.catalina.util.TomcatCSS.TOMCAT_CSS.getBytes(StandardCharsets.
UTF_8).length
+ serverInfoLength + statusHeaderLength * 2;

??

It's just going to happen again when someone changes the CSS.


I have now pushed a change which should address this properly.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Properly calculate dynamic parts of ErrorReportValve response for TestHttp2InitialConnection

2019-11-13 Thread michaelo
This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new df08451  Properly calculate dynamic parts of ErrorReportValve response 
for TestHttp2InitialConnection
df08451 is described below

commit df084515c34328f2186f5d43a993c798ff00b729
Author: Michael Osipov 
AuthorDate: Wed Nov 13 23:12:09 2019 +0100

Properly calculate dynamic parts of ErrorReportValve response for 
TestHttp2InitialConnection
---
 .../coyote/http2/TestHttp2InitialConnection.java | 20 +---
 webapps/docs/changelog.xml   |  6 ++
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java 
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
index a362a19..1a99c3a 100644
--- a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
+++ b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
@@ -115,7 +115,7 @@ public class TestHttp2InitialConnection extends 
Http2TestBase {
  * This will vary depending on where the test is run due to:
  * - The length of the version string that appears once in the 
error
  *   page
- * - The status header uses a UTF-8 EM dash. When running in an IDE
+ * - The status header uses a UTF-8 EN dash. When running in an IDE
  *   the UTF-8 properties files will be used directly rather than
  *   after native2ascii conversion.
  *
@@ -124,10 +124,24 @@ public class TestHttp2InitialConnection extends 
Http2TestBase {
 int serverInfoLength = 
ServerInfo.getServerInfo().getBytes().length;
 StringManager sm = StringManager.getManager(
 ErrorReportValve.class.getPackage().getName(), 
Locale.ENGLISH);
+String reason = sm.getString("http." + 
testData.getExpectedStatus() + ".reason");
+int descriptionLength = sm.getString("http." + 
testData.getExpectedStatus() + ".desc")
+.getBytes(StandardCharsets.UTF_8).length;
 int statusHeaderLength = sm
-.getString("errorReportValve.statusHeader", "", "")
+.getString("errorReportValve.statusHeader",
+String.valueOf(testData.getExpectedStatus()), 
reason)
+.getBytes(StandardCharsets.UTF_8).length;
+int typeLabelLength = sm.getString("errorReportValve.type")
+.getBytes(StandardCharsets.UTF_8).length;
+int statusReportLabelLength = 
sm.getString("errorReportValve.statusReport")
+.getBytes(StandardCharsets.UTF_8).length;
+int descriptionLabelLength = 
sm.getString("errorReportValve.description")
 .getBytes(StandardCharsets.UTF_8).length;
-int len = 708 + serverInfoLength + statusHeaderLength * 2;
+// 196 bytes is the static length of the pure HTML code from the 
ErrorReportValve
+int len = 196 + org.apache.catalina.util.TomcatCSS.TOMCAT_CSS
+.getBytes(StandardCharsets.UTF_8).length +
+typeLabelLength + statusReportLabelLength + 
descriptionLabelLength +
+descriptionLength + serverInfoLength + statusHeaderLength 
* 2;
 String contentLength = String.valueOf(len);
 return getResponseBodyFrameTrace(streamId,
 testData.getExpectedStatus(), "text/html;charset=utf-8",
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index 08598fc..bf9dacc 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -141,6 +141,12 @@
 with ping_mode=I, the CPING
 message will not always be followed by the start of a request. (markt)
   
+  
+Properly calculate all dynamic parts of the ErrorReportValve response
+on the fly in
+org.apache.coyote.http2.TestHttp2InitialConnection.
+(michaelo)
+  
 
   
   


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Polish. Trivial clean-up while looking at 9.0.x/8.5.x/7.0.x diff

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 335502a  Polish. Trivial clean-up while looking at 9.0.x/8.5.x/7.0.x 
diff
335502a is described below

commit 335502a6ebd53fcee123f0cc099c2ec369d9468c
Author: Mark Thomas 
AuthorDate: Wed Nov 13 22:03:12 2019 +

Polish. Trivial clean-up while looking at 9.0.x/8.5.x/7.0.x diff
---
 java/org/apache/catalina/startup/Bootstrap.java  | 1 -
 java/org/apache/catalina/tribes/transport/nio/ParallelNioSender.java | 3 +--
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/startup/Bootstrap.java 
b/java/org/apache/catalina/startup/Bootstrap.java
index ac2bfe5..177a59d 100644
--- a/java/org/apache/catalina/startup/Bootstrap.java
+++ b/java/org/apache/catalina/startup/Bootstrap.java
@@ -34,7 +34,6 @@ import 
org.apache.catalina.startup.ClassLoaderFactory.RepositoryType;
 import org.apache.juli.logging.Log;
 import org.apache.juli.logging.LogFactory;
 
-
 /**
  * Bootstrap loader for Catalina.  This application constructs a class loader
  * for use in loading the Catalina internal classes (by accumulating all of the
diff --git 
a/java/org/apache/catalina/tribes/transport/nio/ParallelNioSender.java 
b/java/org/apache/catalina/tribes/transport/nio/ParallelNioSender.java
index 7eb47b4..585d2cd 100644
--- a/java/org/apache/catalina/tribes/transport/nio/ParallelNioSender.java
+++ b/java/org/apache/catalina/tribes/transport/nio/ParallelNioSender.java
@@ -209,7 +209,6 @@ public class ParallelNioSender extends AbstractSender 
implements MultiPointSende
 }
 }
 return result;
-
 }
 
 private static class SendResult {
@@ -372,4 +371,4 @@ public class ParallelNioSender extends AbstractSender 
implements MultiPointSende
 if ( result ) try { selector.selectNow(); }catch (Exception 
e){/*Ignore*/}
 return result;
 }
-}
\ No newline at end of file
+}


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [tomcat] branch master updated: Fix test fallout of BZ 63905

2019-11-13 Thread Michael Osipov

Am 2019-11-13 um 17:48 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 11/9/19 05:58, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git
repository.

michaelo pushed a commit to branch master in repository
https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this
push: new ccd6518  Fix test fallout of BZ 63905 ccd6518 is
described below

commit ccd6518024ab4f8c0add42eb2b20941a6b237f5d Author: Michael
Osipov  AuthorDate: Sat Nov 9 11:35:22 2019
+0100

Fix test fallout of BZ 63905 ---
test/org/apache/coyote/http2/TestHttp2InitialConnection.java | 2
+- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git
a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
index 2157eae..a362a19 100644 ---
a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java +++
b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java @@
-127,7 +127,7 @@ public class TestHttp2InitialConnection extends
Http2TestBase { int statusHeaderLength = sm
.getString("errorReportValve.statusHeader", "", "")
.getBytes(StandardCharsets.UTF_8).length; -int len =
1073 + serverInfoLength + statusHeaderLength * 2; +int
len = 708 + serverInfoLength + statusHeaderLength * 2;


Why not fix this properly with:

int len =
org.apache.catalina.util.TomcatCSS.TOMCAT_CSS.getBytes(StandardCharsets.
UTF_8).length
+ serverInfoLength + statusHeaderLength * 2;

??


Thanks for raising, but simply this won't work. You have to calculate 
the length of:



HTTP Status 400 – Bad Requestbody {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 
{font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}HTTP Status 400 – Bad RequestType Status ReportDescription The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, 
invalid request message framing, or deceptive request routing).Apache Tomcat/9.0.x-dev


What confuses me here for the test, why is

> sm.getString("errorReportValve.statusHeader", "", "")

performed instead of getting the real strings for HTTP 400?!
If you remove all calculatable bits and add them on the code, you have left:


 
 


So one should add 196 static bytes.

I will look into it.

Michael




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [tomcat] branch master updated: BZ 63905: Clean up Tomcat CSS

2019-11-13 Thread Michael Osipov

Am 2019-11-13 um 17:41 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 11/8/19 17:49, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git
repository.

michaelo pushed a commit to branch master in repository
https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this
push: new 69602cb  BZ 63905: Clean up Tomcat CSS 69602cb is
described below

commit 69602cb1ef0dc3aa2188a1b3be9fa3a1440cd1b1 Author: Michael
Osipov  AuthorDate: Thu Nov 7 22:21:27 2019
+0100

BZ 63905: Clean up Tomcat CSS

Remove duplication in Tomcat CSS and align its usage which applies
to ErrorReportValve, and several servlets. Also update update
incorrect documentation as well. ---
java/org/apache/catalina/servlets/DefaultServlet.java | 16
++-- java/org/apache/catalina/util/TomcatCSS.java
| 16 +++- webapps/docs/changelog.xml
|  3 +++ webapps/docs/config/valve.xml |  5
++--- 4 files changed, 22 insertions(+), 18 deletions(-)

diff --git a/java/org/apache/catalina/servlets/DefaultServlet.java
b/java/org/apache/catalina/servlets/DefaultServlet.java index
c217cf6..2883f5c 100644 ---
a/java/org/apache/catalina/servlets/DefaultServlet.java +++
b/java/org/apache/catalina/servlets/DefaultServlet.java @@ -1815,14
+1815,18 @@ public class DefaultServlet extends HttpServlet {
String rewrittenContextPath =  rewriteUrl(contextPath);

// Render the page header -sb.append("\r\n"); +
sb.append("\r\n"); +/* TODO Activate
this as soon as we use smClient with the request locales +
sb.append("\r\n"); +
*/


Thank you for this. Even though it's not yet active, It's good to have
it in there as a reminder for the future.

Do you think it's worth having any \r\n characters in here? Seems like
just a waste of space on the wire.


I would consider two options:

* Remove line breaks, the browser does not need that
* Use a StringWriter/PrintWriter to produce consistent line breaks on 
the platforms Tomcat runs


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63867] Add option for reason phrase

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63867

--- Comment #19 from Michael Osipov  ---
The problem also is that the only would fix the symptom and push the remedy
even more into the future where the problem will get worse.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #34 from Michael Osipov  ---
(In reply to Christopher Schultz from comment #33)
> (In reply to Rainer Jung from comment #23)
> > Plus: performance behavior plus provider string would also be interesting
> > for MacOS.
> 
> Some info from MacOS:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [146988]
> Writing 10MB took [35] milliseconds
> 
> With 16k buffer:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller 16384
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [16384]
> Writing 10MB took [42] milliseconds
> 
> With 8k buffer:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller 8192
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [8192]
> Writing 10MB took [63] milliseconds
> 
> With 4k buffer:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller 4096
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [4096]
> Writing 10MB took [91] milliseconds
> 
> With 1k buffer:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller 1024
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [1024]
> Writing 10MB took [248] milliseconds
> 
> With 32 byte buffer:
> 
> $ java -showversion org/apache/tomcat/Bug63916NioPoller 32
> openjdk version "13.0.1" 2019-10-15
> OpenJDK Runtime Environment (build 13.0.1+9)
> OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
> Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
> Default send buffer size is [146988]
> Using send buffer size [32]
> Writing 10MB took [5279] milliseconds
> 
> More data (buffer size, duration):
> 16,10383
> 32,5206
> 64,2695
> 128,1413
> 256,744
> 512,435
> 1024,238
> 2048,149
> 4096,85
> 8192,61
> 16384,40
> 146988,35
> 
> Graphed: https://jsfiddle.net/etp8sa5f/
> 
> Interestingly, it's almost a perfectly straight trend line when graphed on a
> logarithmic scale. I think that means there aren't any inconsistencies on
> this platform: buffer size is hugely important but there aren't any
> particularly bad buffer sizes (other than "smaller is worse").

Awesome, can you retry with Java 8?

But the results are quite surprising because I have expected similar results as
with FreeBSD due to the BSD userland and kernel parts in macOS.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Fix SpotBugs warnings

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 35e4aaa  Fix SpotBugs warnings
35e4aaa is described below

commit 35e4aaa02b2587a77d303e8e46fda2ba5ccb835e
Author: Mark Thomas 
AuthorDate: Wed Nov 13 17:59:04 2019 +

Fix SpotBugs warnings
---
 .../org/apache/catalina/startup/ContextConfig.java |  2 +-
 .../apache/tomcat/buildutil/translate/Utils.java   |  6 ++-
 res/findbugs/filter-false-positives.xml| 62 +-
 .../apache/catalina/core/TestAsyncContextImpl.java | 16 +++---
 4 files changed, 75 insertions(+), 11 deletions(-)

diff --git a/java/org/apache/catalina/startup/ContextConfig.java 
b/java/org/apache/catalina/startup/ContextConfig.java
index c35c354..39fb25d 100644
--- a/java/org/apache/catalina/startup/ContextConfig.java
+++ b/java/org/apache/catalina/startup/ContextConfig.java
@@ -2692,7 +2692,7 @@ public class ContextConfig implements LifecycleListener {
 @Override
 public void lifecycleEvent(LifecycleEvent event) {
 
-if (event.getType() == Lifecycle.AFTER_DESTROY_EVENT) {
+if (Lifecycle.AFTER_DESTROY_EVENT.equals(event.getType())) {
 Host host = (Host) event.getSource();
 hostWebXmlCache.remove(host);
 }
diff --git a/java/org/apache/tomcat/buildutil/translate/Utils.java 
b/java/org/apache/tomcat/buildutil/translate/Utils.java
index 8b69c79..5eb015f 100644
--- a/java/org/apache/tomcat/buildutil/translate/Utils.java
+++ b/java/org/apache/tomcat/buildutil/translate/Utils.java
@@ -78,7 +78,11 @@ public class Utils {
 
 
 static void processDirectory(File root, File dir, Map 
translations) throws IOException {
-for (File f : dir.listFiles()) {
+File[] files = dir.listFiles();
+if (files == null) {
+throw new IllegalArgumentException("Not a directory [" + 
dir.getAbsolutePath() + "]");
+}
+for (File f : files) {
 if (f.isDirectory()) {
 processDirectory(root, f, translations);
 } else if (f.isFile()) {
diff --git a/res/findbugs/filter-false-positives.xml 
b/res/findbugs/filter-false-positives.xml
index 1c290b2..7fcc6bd 100644
--- a/res/findbugs/filter-false-positives.xml
+++ b/res/findbugs/filter-false-positives.xml
@@ -836,6 +836,18 @@
 
   
   
+
+
+
+
+  
+  
+
+
+
+
+  
+  
 
 
 
@@ -1389,6 +1401,13 @@
 
   
   
+
+
+
+
+
+  
+  
 
 
 
@@ -1440,6 +1459,21 @@
 
   
   
+  
+
+
+
+  
+  
+  
+
+
+  
+  
+
+
+  
+  
 
 
 
@@ -1552,9 +1586,17 @@
 
   
   
+
+ 
+
+  
+  
 
 
-
+
+  
+  
+
 
   
   
@@ -1606,11 +1648,23 @@
   
   
 
+
+
+
+  
+  
+
 
 
 
   
   
+
+
+
+
+  
+  
 
 
 
@@ -1688,6 +1742,12 @@
 
   
   
+
+
+
+
+  
+  
 
 
 
diff --git a/test/org/apache/catalina/core/TestAsyncContextImpl.java 
b/test/org/apache/catalina/core/TestAsyncContextImpl.java
index 56e86ae..ea156ba 100644
--- a/test/org/apache/catalina/core/TestAsyncContextImpl.java
+++ b/test/org/apache/catalina/core/TestAsyncContextImpl.java
@@ -564,7 +564,7 @@ public class TestAsyncContextImpl extends TomcatBaseTest {
 private static final long serialVersionUID = 1L;
 
 private final Boolean completeOnTimeout;
-private final TrackingListener trackingListener;
+private final transient TrackingListener trackingListener;
 
 public static final long ASYNC_TIMEOUT = 100;
 
@@ -694,7 +694,7 @@ public class TestAsyncContextImpl extends TomcatBaseTest {
 private static final long serialVersionUID = 1L;
 private static final String ITER_PARAM = "iter";
 private static final String DISPATCH_CHECK = "check";
-private final TrackingListener trackingListener;
+private final transient TrackingListener trackingListener;
 
 public DispatchingServlet(boolean addTrackingListener,
 boolean completeOnError) {
@@ -1905,7 +1905,7 @@ public class TestAsyncContextImpl extends TomcatBaseTest {
 
 private static final long serialVersionUID = 1L;
 
-private final TrackingListener trackingListener = new 
TrackingListener(true, false, "/async");
+private final transient TrackingListener trackingListener = new 
TrackingListener(true, false, "/async");
 @Override
 protected void doGet(HttpServletRequest req, HttpServletResponse resp)
 throws ServletException, IOException {
@@ -2756,7 +2756,7 @@ public class TestAsyncContextImpl extends TomcatBaseTest {
 
 private final boolean useThread;
 private final 

[Bug 63865] Cookie Attribute SameSite=None is default to unset in Chrome browser

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63865

--- Comment #5 from Remy Maucherat  ---
There's a release batch almost every month, so please stop asking "when" ...

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Add explicit list of missing Graal features for Tomcat

2019-11-13 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 27d82e2  Add explicit list of missing Graal features for Tomcat
27d82e2 is described below

commit 27d82e20bc112df7be063d9bd7398f7afd848ead
Author: remm 
AuthorDate: Wed Nov 13 18:56:28 2019 +0100

Add explicit list of missing Graal features for Tomcat

And what's broken because of them.
---
 webapps/docs/graal.xml | 13 +
 1 file changed, 13 insertions(+)

diff --git a/webapps/docs/graal.xml b/webapps/docs/graal.xml
index c751344..852de70 100644
--- a/webapps/docs/graal.xml
+++ b/webapps/docs/graal.xml
@@ -176,6 +176,19 @@ ant -Dwebapp.name=somewebapp -f 
graal-webapp.ant.xml
 listeners (use of internal code that does not exist in Graal).
   
 
+  
+Missing items for better Tomcat functionality:
+
+  Java 11: ALPN for TLS, TLS 1.3, etc
+  BeanInfo support: BeanInfo use in JSP taglibs must be configured
+manually in descriptors
+  Java serialization: Clustering, session persistence
+  Static linking configuration: JSSE elliptic curve support,
+tomcat-native support when creating an image with static linking
+  JMX: Tomcat monitoring and management
+
+  
+
   
 
 


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63865] Cookie Attribute SameSite=None is default to unset in Chrome browser

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63865

--- Comment #4 from Christopher Schultz  ---
(In reply to Pushyami Gundala from comment #3)
> Thanks for the fix. When will 8.5.48 going to be released? as Chrome 80 is
> scheduled to release Feb 4, 2020
> https://www.chromestatus.com/features/schedule

Looks like markt has just proposed a round of releases, including what will
(likely) be 8.5.48. You should certainly have plenty of time before Chrome 80
is released.

If you have further questions, please post to the users' mailing list as
Bugzilla is not a support forum but a bug-tracker.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63867] Add option for reason phrase

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63867

--- Comment #18 from Christopher Schultz  ---
(In reply to Nathan Clement from comment #17)
> We operate a REST API and thousands of corporate/government clients connect
> to our API. We know that some of these clients rely on the reason phrase,
> but we have no way to determine how many.
>
> We currently use Tomcat 8.5, but we will need to upgrade to 9.0 at some
> point. That upgrade will cause clients to break, and the clients will come
> to us and demand that we "fix it". They won't accept arguments that it's
> good for them to have to change their system. All they will see is that we
> upgraded our system and their integration stopped working.

So announce a new version of your REST API which does not include a reason
phrase and ask everyone to move to it. Anyone who starts getting errors can
fall-back to the older version and update their clients. Give them a timeline
for when the old service will no longer be available.

Tomcat can't remain static forever.

> The reason phrase is a feature of HTTP/1.1 and I don't see any downside for
> Tomcat supporting it long term.

And yet we, the actual maintainers, *do* see downsides to supporting it long
term.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #33 from Christopher Schultz  ---
(In reply to Rainer Jung from comment #23)
> Plus: performance behavior plus provider string would also be interesting
> for MacOS.

Some info from MacOS:

$ java -showversion org/apache/tomcat/Bug63916NioPoller
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [146988]
Writing 10MB took [35] milliseconds

With 16k buffer:

$ java -showversion org/apache/tomcat/Bug63916NioPoller 16384
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [16384]
Writing 10MB took [42] milliseconds

With 8k buffer:

$ java -showversion org/apache/tomcat/Bug63916NioPoller 8192
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [8192]
Writing 10MB took [63] milliseconds

With 4k buffer:

$ java -showversion org/apache/tomcat/Bug63916NioPoller 4096
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [4096]
Writing 10MB took [91] milliseconds

With 1k buffer:

$ java -showversion org/apache/tomcat/Bug63916NioPoller 1024
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [1024]
Writing 10MB took [248] milliseconds

With 32 byte buffer:

$ java -showversion org/apache/tomcat/Bug63916NioPoller 32
openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
Selector Provider: sun.nio.ch.KQueueSelectorProvider@1ac1a5f6
Default send buffer size is [146988]
Using send buffer size [32]
Writing 10MB took [5279] milliseconds

More data (buffer size, duration):
16,10383
32,5206
64,2695
128,1413
256,744
512,435
1024,238
2048,149
4096,85
8192,61
16384,40
146988,35

Graphed: https://jsfiddle.net/etp8sa5f/

Interestingly, it's almost a perfectly straight trend line when graphed on a
logarithmic scale. I think that means there aren't any inconsistencies on this
platform: buffer size is hugely important but there aren't any particularly bad
buffer sizes (other than "smaller is worse").

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [PROPOSAL] Tomcat 10: Drop APR Connector

2019-11-13 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Rémy,

On 11/11/19 14:20, Rémy Maucherat wrote:
> On Mon, Nov 11, 2019 at 7:47 PM Michael Osipov
> mailto:micha...@apache.org>> wrote:
> 
> To revive this, why APR is stil important:
> 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=63916
> 
> There is some severe bug making NIO performing very bad.
> 
> 
> We're making long term plans here, a bug report filed yesterday is 
> rather irrelevant.

I tend to agree, especially due to the subsequent resolution of this
problem.

The BIO connector had better performance than NIO, too, with less CPU
usage and yet we still removed it.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3MNsIACgkQHPApP6U8
pFiagA/+MV29Y6W2GTfvAVDrfsv87jlB63BOqPbw7cNtUCA7XEH/iiK9kVSiOuqJ
0s9rBHMo0QtMAHEUixfgHRQFMeoAsEwSyJK4vAF3nN7kBBSq8fL7DGIczNEDmKf1
0p20DFTRGHJBxPptNCJJvSSgYn9MiUAXhiDaPomJ4daL/5dlLincWS7fo+kiP36L
cTglITbILknMyrGsYS9AvYQYEpA2epfJFiWHxDYO3ZgB4MCSazr9nmUky/Pscrir
ew8aBcCm0ArTB65v6729zB3f7UJxNo74FtkbSUtdR28WTvpVAcSVC9eiEFnivXR9
gyO9w9sZYAt0X5jZbV4SA23wnBLx63tWwNm3GNKBV+v2hOZYd5H8cF6iMJ20I/x4
MhqaLnUSQZyu2YvMXtwVAYkGGEnEmUagBU5quII2lSdyH6idsI70I01cxCMd8bEc
/64a81zriaQK20IlAEm6zAClC9dS3s8Qugx0hW+l+Gs73BTLOSzXK6Qm7XGH+uUV
hYMoK31i4lZKsctWKtuV1r9sOSNOSfDc91FjBnkasuXD/YO3K4ZoilYQwWvKpsoH
biLOMeA5PFGSi1ptaaIF/ij6BmHRwYaU75CXd2BCb9lDgGHbYge+8fhgU9Usip1b
8m8WGWtn6W6Cgs4ix4BdYSrvxqJaj37L5F8aZTQZlBoe/boXYzg=
=DCXE
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [tomcat] branch master updated: Fix test fallout of BZ 63905

2019-11-13 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 11/9/19 05:58, micha...@apache.org wrote:
> This is an automated email from the ASF dual-hosted git
> repository.
> 
> michaelo pushed a commit to branch master in repository
> https://gitbox.apache.org/repos/asf/tomcat.git
> 
> 
> The following commit(s) were added to refs/heads/master by this
> push: new ccd6518  Fix test fallout of BZ 63905 ccd6518 is
> described below
> 
> commit ccd6518024ab4f8c0add42eb2b20941a6b237f5d Author: Michael
> Osipov  AuthorDate: Sat Nov 9 11:35:22 2019
> +0100
> 
> Fix test fallout of BZ 63905 --- 
> test/org/apache/coyote/http2/TestHttp2InitialConnection.java | 2
> +- 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git
> a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java
> b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java 
> index 2157eae..a362a19 100644 ---
> a/test/org/apache/coyote/http2/TestHttp2InitialConnection.java +++
> b/test/org/apache/coyote/http2/TestHttp2InitialConnection.java @@
> -127,7 +127,7 @@ public class TestHttp2InitialConnection extends
> Http2TestBase { int statusHeaderLength = sm 
> .getString("errorReportValve.statusHeader", "", "") 
> .getBytes(StandardCharsets.UTF_8).length; -int len =
> 1073 + serverInfoLength + statusHeaderLength * 2; +int
> len = 708 + serverInfoLength + statusHeaderLength * 2;

Why not fix this properly with:

int len =
org.apache.catalina.util.TomcatCSS.TOMCAT_CSS.getBytes(StandardCharsets.
UTF_8).length
+ serverInfoLength + statusHeaderLength * 2;

??

It's just going to happen again when someone changes the CSS.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3MM/kACgkQHPApP6U8
pFhkUw//VSKrL5DFR+mtxdDlzsPmnMWYJQ6hKNfuuq6borShwI9smOIBaUmnBWWu
jmA+qWJ0zoQaWwOsc0mmixDzMO1Ck5/B5HVdWcwtSlif1YUK7MSEqlCIELkwvqgA
TYxiA6CLf4SQc7X3cipZvhqJQ+5tVBjS9Irc3tQ86p6Ojly3h83c3gAAKncYXLWO
nI07wIUI5uC8jh/s5lmq0fUcIvmWgwzU4vOEH2ZvCPDnfja5HYfp3JOMPkwUXD43
/FP/dewd/trDy2vYBu0StKn6Ft0fhVQzXCvMDzWUVdyMJGHUQ+gh3D4rCwpRs0p4
X1lI6pogK0GNE1mtokNcqtkIg6W4z8QWGyKK0i2zFsk3gaVLoV5K0Bhoao3LUvZ/
oFSJnQaYdhe7QQrftkXw0NlEdMKkt6kHVMbiV/GSlvcL8ERH8mBlyIUfEKLIb43n
3iXxaJqTnFXYjYqiRiZezZxHhB2OyEs3eCshxP4lxSXmL8HQOEHnAvwadW16xwA4
WDYaL31AeYqxl206ku2fQFCm5Lb+Gt6EEI9wHXc9g+oLgzcw8aLo4zr/0FgWq5US
TxcjgNfJqSF65a86oLvwx1BE545FPFctgck/lXVcKTVYgeDsfgr6V8S2cif0812H
3Tf5RtHVsmUEsML2+Ge3kIfObZioqCCaeBwdQxLE4NNkKHgfKzs=
=RvAv
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63905] ErrorReportValve adds CSS even if both showReport and showServerInfo are set to false

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63905

--- Comment #19 from Christopher Schultz  ---
(In reply to Michael Osipov from comment #18)
> (In reply to Christopher Schultz from comment #17)
> > (In reply to Michael Osipov from comment #15)
> > > (In reply to Christopher Schultz from comment #14)
> > > > This should all really be replaced by external stylesheets, for a few
> > > > reasons:
> > > > 
> > > > 1. They are trivially changed by administrators instead of hacking Java 
> > > > code
> > > > 2. They can be completely blocked, replaced, etc. by a reverse proxy if
> > > > desired
> > > > 3. They are more compatible with a desire to reduce response entity byte
> > > > count
> > > > 4. They can be used with a "safe" CSF policies[1]
> > > 
> > > I agree here, but this is far more work than I do now at the moment.
> > 
> > Honestly, I see this as easy:
> > 
> > 1. Move CSS definitions to [whatever.css]
> > 1. Change all CSS to 
> 
> It is not that easy. TOMCAT_CSS is used in multiple locations. You have to
> make sure that they are present for several components.

So use TOMCAT_EXTERNAL_CSS as a replacement, and only implement it in e.g.
DefaultServlet. No need to re-write all of Tomcat's use of CSS at once.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [tomcat] branch master updated: BZ 63905: Clean up Tomcat CSS

2019-11-13 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 11/8/19 17:49, micha...@apache.org wrote:
> This is an automated email from the ASF dual-hosted git
> repository.
> 
> michaelo pushed a commit to branch master in repository
> https://gitbox.apache.org/repos/asf/tomcat.git
> 
> 
> The following commit(s) were added to refs/heads/master by this
> push: new 69602cb  BZ 63905: Clean up Tomcat CSS 69602cb is
> described below
> 
> commit 69602cb1ef0dc3aa2188a1b3be9fa3a1440cd1b1 Author: Michael
> Osipov  AuthorDate: Thu Nov 7 22:21:27 2019
> +0100
> 
> BZ 63905: Clean up Tomcat CSS
> 
> Remove duplication in Tomcat CSS and align its usage which applies
> to ErrorReportValve, and several servlets. Also update update
> incorrect documentation as well. --- 
> java/org/apache/catalina/servlets/DefaultServlet.java | 16
> ++-- java/org/apache/catalina/util/TomcatCSS.java
> | 16 +++- webapps/docs/changelog.xml
> |  3 +++ webapps/docs/config/valve.xml |  5
> ++--- 4 files changed, 22 insertions(+), 18 deletions(-)
> 
> diff --git a/java/org/apache/catalina/servlets/DefaultServlet.java
> b/java/org/apache/catalina/servlets/DefaultServlet.java index
> c217cf6..2883f5c 100644 ---
> a/java/org/apache/catalina/servlets/DefaultServlet.java +++
> b/java/org/apache/catalina/servlets/DefaultServlet.java @@ -1815,14
> +1815,18 @@ public class DefaultServlet extends HttpServlet { 
> String rewrittenContextPath =  rewriteUrl(contextPath);
> 
> // Render the page header -sb.append("\r\n"); +
> sb.append("\r\n"); +/* TODO Activate
> this as soon as we use smClient with the request locales +
> sb.append(" sb.append(smClient.getLocale().getLanguage()).append("\">\r\n"); +
> */

Thank you for this. Even though it's not yet active, It's good to have
it in there as a reminder for the future.

Do you think it's worth having any \r\n characters in here? Seems like
just a waste of space on the wire.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3MMisACgkQHPApP6U8
pFikpBAAkolAV7dx+erYxAYAjs6jIsRMIdZhjGkRJ+K57b5f1A0UlaimBT5embIW
bYrCDZEpV+1yYW3Ii61ZNZD59gkCVh5eyCxE72s+M3AWvWvgAOepCoj9nbiw3kXq
ac0LCQ/bB2qmLIBc2NGCACYxRNRtmB3aGAC1j9Jnr6MsYGRLIWOyueVKRDVhJjvm
F/F3PDQIy6TWvMXKfgu+JeE38ArB7qVg2UFI7SYTRVw6gvvi/XnngOio4Io8uFv2
Q2iAkeWu0maeCbjDtKOq3YVLiiKXbGsThx/tLITKYk4V12x/TFflpYd5fYFUuMoJ
Mw9HdID3BaIBHOwq3Q4J5t9ocZfT01jsIbjLhD9QIrIqrLCp7PbwzE2DPD4tQlDV
mFk6UI3FI+4SbtJMEy3QdwB4Oj44xogIclwVwUfDx07WrfSLwUmOOwhceLyDkgO+
9azj71HggvovDx0ROQXLzzved3qZIP23N4nNgpDJcj+kP/+0laR6crcfLIGLWR2f
z0ejIjwB0lhTxeheYOPCnGjACcN6h4Dm5+uu08g/QxmCObnzZpORdPJuW2PSlaYy
Dn0h0cuz/vRg/Z+4nEve9GBWX7COZNiuOqXwEwoO5OKro2Fzvwj5o3ZqbM7MGRmj
2WViKkuhedYEHFRlfEeZ182/z7OczDX8irbPdYl2k8Vn41O/00Y=
=sEMq
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #32 from Michael Osipov  ---
I am down now to:

$ for file in output/build/logs/*TestNonBlockingAPI* ; do head -2 $file; done
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 35,434 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 41,08 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 36,316 sec

All tests run now with 18 threads in 10 minutes on that server. What a
tremendous improvement.

Mark, thank you very much looking into this. Shouldn't the findings for buffer
sizes be documented somewhere on the connectors page for our users?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63865] Cookie Attribute SameSite=None is default to unset in Chrome browser

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63865

--- Comment #3 from Pushyami Gundala  ---
Thanks for the fix. When will 8.5.48 going to be released? as Chrome 80 is
scheduled to release Feb 4, 2020 https://www.chromestatus.com/features/schedule

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63922] The Java NMT (Native Memory Trackingú cannot be activated by Tomcat 7, 8 or 9 windows service

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63922

--- Comment #2 from Artur Linhart  ---
OK, thank you for correcting me. See the issue
https://issues.apache.org/jira/browse/DAEMON-412

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Tagging 9.0.28 and 8.5.48

2019-11-13 Thread Mark Thomas
Hi all,

Rather later than I had hoped, it looks like we are in a position to tag
9.0.28 and 8.5.48. As usual, I'll run the unit tests on various
platforms and, assuming all is well, tag shortly afterwards.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63922] The Java NMT (Native Memory Trackingú cannot be activated by Tomcat 7, 8 or 9 windows service

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63922

Mark Thomas  changed:

   What|Removed |Added

 Resolution|--- |INVALID
 Status|NEW |RESOLVED

--- Comment #1 from Mark Thomas  ---
Procrun is part of the Commons Daemon project. Bugs against procrun should be
raised at:
https://issues.apache.org/jira/projects/DAEMON

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #31 from Mark Thomas  ---
Thanks. Missed that. Fixed now.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch 8.5.x updated: Additional fix for https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this push:
 new 9df8cae  Additional fix for 
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916
9df8cae is described below

commit 9df8caed051dc0934b530309fee2ccf6829d06c3
Author: Mark Thomas 
AuthorDate: Wed Nov 13 15:16:11 2019 +

Additional fix for https://bz.apache.org/bugzilla/show_bug.cgi?id=63916
---
 test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
index e9e9059..a1a9a4a 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -313,7 +313,10 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
 String servletName = NBWriteServlet.class.getName();
 Tomcat.addServlet(ctx, servletName, servlet);
 ctx.addServletMappingDecoded("/", servletName);
-tomcat.getConnector().setProperty("socket.txBufSize", "1024");
+// Note: Low values of socket.txBufSize can trigger very poor
+//   performance. Set it just low enough to ensure that the
+//   non-blocking write servlet will see isReady() == false
+tomcat.getConnector().setProperty("socket.txBufSize", "1048576");
 tomcat.start();
 
 SocketFactory factory = SocketFactory.getDefault();


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Additional fix for https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 4f38e7f  Additional fix for 
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916
4f38e7f is described below

commit 4f38e7f8201401083985a30d45db271b543dae41
Author: Mark Thomas 
AuthorDate: Wed Nov 13 15:16:11 2019 +

Additional fix for https://bz.apache.org/bugzilla/show_bug.cgi?id=63916
---
 test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
index f93b277..0ac987c 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -326,7 +326,10 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
 String servletName = NBWriteServlet.class.getName();
 Tomcat.addServlet(ctx, servletName, servlet);
 ctx.addServletMappingDecoded("/", servletName);
-tomcat.getConnector().setProperty("socket.txBufSize", "1024");
+// Note: Low values of socket.txBufSize can trigger very poor
+//   performance. Set it just low enough to ensure that the
+//   non-blocking write servlet will see isReady() == false
+tomcat.getConnector().setProperty("socket.txBufSize", "1048576");
 tomcat.start();
 
 SocketFactory factory = SocketFactory.getDefault();


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #30 from Michael Osipov  ---
(In reply to Michael Osipov from comment #29)
> (In reply to Mark Thomas from comment #28)
> > Fixed in:
> > - master for 9.0.28 onwards
> > - 8.5.x for 8.5.48 onwards
> 
> I see now
> 
> $ for file in output/build/logs/*TestNonBlockingAPI* ; do head -2 $file; done
> Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 36,975 sec
> Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
> Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 69,567 sec
> Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
> Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 65,669 sec
> 
> Is there a reason why you did not change the second socket.txBufSize? It
> would cut the time in half.

The failures are timing issues for 200 ms which can be ignored as I have
learned two weeks ago.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #29 from Michael Osipov  ---
(In reply to Mark Thomas from comment #28)
> Fixed in:
> - master for 9.0.28 onwards
> - 8.5.x for 8.5.48 onwards

I see now

$ for file in output/build/logs/*TestNonBlockingAPI* ; do head -2 $file; done
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 36,975 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 69,567 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 65,669 sec

Is there a reason why you did not change the second socket.txBufSize? It would
cut the time in half.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63922] The Java NMT (Native Memory Trackingú cannot be activated by Tomcat 7, 8 or 9 windows service

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63922

Artur Linhart  changed:

   What|Removed |Added

Summary|The Native Memory Tracking  |The Java NMT (Native Memory
   |cannot be activated by  |Trackingú cannot be
   |Tomcat 7, 8 or 9 windows|activated by Tomcat 7, 8 or
   |service |9 windows service

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63922] New: The Native Memory Tracking cannot be activated by Tomcat 7, 8 or 9 windows service

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63922

Bug ID: 63922
   Summary: The Native Memory Tracking cannot be activated by
Tomcat 7, 8 or 9 windows service
   Product: Tomcat 9
   Version: unspecified
  Hardware: PC
OS: All
Status: NEW
  Severity: major
  Priority: P2
 Component: Packaging
  Assignee: dev@tomcat.apache.org
  Reporter: artur.linh...@algites.eu
  Target Milestone: -

In the case the user specifies some from the the NativeMemoryTracking
parameters (see
https://docs.oracle.com/en/java/javase/11/vm/native-memory-tracking.html#GUID-0DB3B8D1-8D7D-447F-B6FF-15620103EE47
) like 

-XX:NativeMemoryTracking=summary

in Windows service registry, restarts the service, so the option can be seen in
the log file by the startup options, then teh Native memory tracking is still
disabled and does not work. If you try with

jcmd  VM.native_memory summary

to get the information about the native JVM memory in the tomcats JVM, the jcmd
call fails with the message

:
Native memory tracking is not enabled

and in the Tomcat's log file can be found following error message:

Java HotSpot(TM) 64-Bit Server VM warning: Native Memory Tracking did not setup
properly, using wrong launcher? 

The problem seems to be in the JVM launcher Procrun, which dies not initialize
the JVM propertly - as far as I have understood from this blog:

https://blogs.oracle.com/poonam/using-nmt-with-custom-jvm-launcher

The Procrun.exe has to define some environmetn variable before the start of
JVM, etc. to get the native memory tracking to work.

Known workaround is the start of Tomcat from the command line, but this is no
option in the productive environment, where you have to let the service run for
longer time to collect the proper result and maybe also as some different user
because of security concerns, then such option is simply out of possible
scenarios. So the Tomcat service JVM cannot be in such circumstances tracked by
NMT at all, what is the problem in the mission-critical applications.

The Procrun should be modified in the way it enables the NMT in the windows
service by the startup, by setting the environment variable 
NMT_LEVEL_ to the value "detail" or "summary" (or "off" if unspecified) in
the dependency on the specified or unspecified option of 
-XX:NativeMemoryTracking
like done normally by java.exe
Remark: Also change of the startup mode from "jvm" to "exe" or "java" does not
help,. then the windows service does not start at all.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

Mark Thomas  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|NEW |RESOLVED

--- Comment #28 from Mark Thomas  ---
Fixed in:
- master for 9.0.28 onwards
- 8.5.x for 8.5.48 onwards

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch 8.5.x updated: Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 speed up test

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this push:
 new 8475c0f  Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 
speed up test
8475c0f is described below

commit 8475c0fdeb39de2110d3659653fe54cdc35f572d
Author: Mark Thomas 
AuthorDate: Wed Nov 13 14:54:57 2019 +

Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 speed up test
---
 test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
index 8acffa7..e9e9059 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -154,7 +154,10 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
 String servletName = NBWriteServlet.class.getName();
 Tomcat.addServlet(ctx, servletName, servlet);
 ctx.addServletMappingDecoded("/", servletName);
-tomcat.getConnector().setProperty("socket.txBufSize", "1024");
+// Note: Low values of socket.txBufSize can trigger very poor
+//   performance. Set it just low enough to ensure that the
+//   non-blocking write servlet will see isReady() == false
+tomcat.getConnector().setProperty("socket.txBufSize", "1048576");
 tomcat.start();
 
 SocketFactory factory = SocketFactory.getDefault();


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 speed up test

2019-11-13 Thread markt
This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 38e7f28  Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 
speed up test
38e7f28 is described below

commit 38e7f28b290f62ae1f4bae7aa562b59f218a7f94
Author: Mark Thomas 
AuthorDate: Wed Nov 13 14:54:57 2019 +

Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63916 speed up test
---
 test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
index 7f16f26..f93b277 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -157,7 +157,10 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
 String servletName = NBWriteServlet.class.getName();
 Tomcat.addServlet(ctx, servletName, servlet);
 ctx.addServletMappingDecoded("/", servletName);
-tomcat.getConnector().setProperty("socket.txBufSize", "1024");
+// Note: Low values of socket.txBufSize can trigger very poor
+//   performance. Set it just low enough to ensure that the
+//   non-blocking write servlet will see isReady() == false
+tomcat.getConnector().setProperty("socket.txBufSize", "1048576");
 tomcat.start();
 
 SocketFactory factory = SocketFactory.getDefault();


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #27 from Mark Thomas  ---
I have been mixing up socket.txBufSize and socket.appWriteBufSize.

My testing shows that socket.appWriteBufSize has no measurable impact on
performance of the stand-alone test.

Performance of the stand-alone test is tightly linked to socket.txBufSize. The
smaller the buffer, the more likely it is the writing thread will fill it
before the OS thread  writes some data. That will cause the socket to be placed
in the Poller and polling does appear to be relatively slow.

The log messages are a significant factor as writing them to stdout slows
things down. I've commented them out for now in my stand-alone test.

The Tomcat unit test in question deliberately sets the txBufSize small to
trigger the use of the Poller. It appears that that setting is far too small.
I'll do some tests and increase it - probably to around 1MB.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #26 from Michael Osipov  ---
A local change to Tomcat master:

> diff --git a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
> b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> index 7f16f26c1e..c8ec7a7eb8 100644
> --- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> +++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
> @@ -157,7 +157,6 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
>  String servletName = NBWriteServlet.class.getName();
>  Tomcat.addServlet(ctx, servletName, servlet);
>  ctx.addServletMappingDecoded("/", servletName);
> -tomcat.getConnector().setProperty("socket.txBufSize", "1024");
>  tomcat.start();
> 
>  SocketFactory factory = SocketFactory.getDefault();
> @@ -323,7 +322,6 @@ public class TestNonBlockingAPI extends TomcatBaseTest {
>  String servletName = NBWriteServlet.class.getName();
>  Tomcat.addServlet(ctx, servletName, servlet);
>  ctx.addServletMappingDecoded("/", servletName);
> -tomcat.getConnector().setProperty("socket.txBufSize", "1024");
>  tomcat.start();
> 
>  SocketFactory factory = SocketFactory.getDefault();

Gives me:

osipovmi@deblndw011x:~/var/Projekte/tomcat (master *=)
$ for file in output/build/logs/*TestNonBlockingAPI* ; do head -2 $file; done
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 37,629 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 36,315 sec
Testsuite: org.apache.catalina.nonblocking.TestNonBlockingAPI
Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 33,279 sec

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #25 from Michael Osipov  ---
Here are new test results, is the terminal/consumer (telnet) too slow?

For both: telnet localhost  > /dev/null

FreeBSD:
All telnet output has been redirected to /dev/null.

default: Writing 10MB took [256] milliseconds
8 KiB: as slow as before
16 KiB: ""
20 KiB: ""
22 KiB: ""
23,5 KiB: ""
bisected to 24 KiB..
24 KiB: Writing 10MB took [288] milliseconds
32 KiB: Writing 10MB took [252] milliseconds
64 KiB: Writing 10MB took [256] milliseconds

Anything below 24 KiB is unusable.


HP-UX:
All telnet output has been redirected to /dev/null.

default: Writing 10MB took [283] milliseconds
8 KiB: as slow as before
8193 B: Writing 10MB took [279] milliseconds
8,5 KiB: Writing 10MB took [305] milliseconds
9 KiB: Writing 10MB took [284] milliseconds
10 KiB: Writing 10MB took [270] milliseconds
12 KiB: Writing 10MB took [278] milliseconds
16 KiB: Writing 10MB took [274] milliseconds
32 KiB: ""

Anything below 8193 B is unusable.


What does that mean now for Tomcat and the tests?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #24 from Michael Osipov  ---
(In reply to Rainer Jung from comment #22)
> According to the Javadoc, the Selector has various impls and is encapsulated
> via a SelectorProvider. The chosen provider decides about which system api
> is being used (poll, epoll, event ports, kqueue, ...).
> 
> I added
> 
> System.out.println("Selector Provider: " + selector.provider());
> 
> to the beginning of the run() method of the Poller and got:
> 
> Linux:
> Selector Provider: sun.nio.ch.EPollSelectorProvider@5ff0e400
> 
> Solaris:
> Selector Provider: sun.nio.ch.DevPollSelectorProvider@3927e6ad
> 
> Maybe one can check the output for FreeBSD? Maybe it falls bac to poll
> instead of kqueue on FreeBSD? If this would be true, we could check, whther
> there is a better provider for FreeBSD and try to choose it explicitly.
> 
> Just my 0.01 cents.

Selector Provider: sun.nio.ch.KQueueSelectorProvider@626007aa

Should be the same as on macOS, imho.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #23 from Rainer Jung  ---
Plus: performance behavior plus provider string would also be interesting for
MacOS.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #22 from Rainer Jung  ---
According to the Javadoc, the Selector has various impls and is encapsulated
via a SelectorProvider. The chosen provider decides about which system api is
being used (poll, epoll, event ports, kqueue, ...).

I added

System.out.println("Selector Provider: " + selector.provider());

to the beginning of the run() method of the Poller and got:

Linux:
Selector Provider: sun.nio.ch.EPollSelectorProvider@5ff0e400

Solaris:
Selector Provider: sun.nio.ch.DevPollSelectorProvider@3927e6ad

Maybe one can check the output for FreeBSD? Maybe it falls bac to poll instead
of kqueue on FreeBSD? If this would be true, we could check, whther there is a
better provider for FreeBSD and try to choose it explicitly.

Just my 0.01 cents.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #21 from Michael Osipov  ---
New test results:

RHEL:
8 KiB: Writing 10MB took [1601] milliseconds; nothing added
default: Writing 10MB took [1176] milliseconds

HP-UX:
8 KiB: Writing 10MB took [89460] milliseconds; add 68 ms for each select()
default: Writing 10MB took [2712] milliseconds

FreeBSD:
8 KiB: Writing 10MB took [134837] milliseconds ; adds 102 ms for each select()
default: Writing 10MB took [2733] milliseconds

I am trying to understand what this really means and needs to be documented for
users and what I need to report with bugs.freebsd.org.

I will performs another test where telnet will redirect to /dev/null and see
how this effects performance.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #20 from Remy Maucherat  ---
With NIO2, when I set a really low send buffer (I tried as low as 32 bytes) it
does change what the read/write logs from TestNonBlockingAPI look like as
expected, but the "performance" does not go down meaningfully. Just a little.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #19 from Mark Thomas  ---
Hmm. No change at all with a Executor. The cut-off for buffer size between good
and bad performance is still exactly 4537 bytes. That doesn't match with what I
thought was going on.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 48392] jdbc-pool is not returning the proxied connection in resultSet and statement

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=48392

--- Comment #9 from KMI  ---
Hi,


I'm using ddlutils 1.0 and tomcat jdbc pool 1.0.7.1 and I getting an error
due to a connection is closed and the pool is not aware of that.
Basically the issue is that ddlutils has a resultset iterator and when it
finishes it closes the connection by getting it from the *
resultSet.preparedStatement.connection* and the connection returned is not
the proxy that the pool has created.


So the issue happens when another client retrieves a connection from the
pool because the pool returns a connection that was actually closed.
Why tomcat jdbc pool is not creating proxies for preparedStatements and
resultSets like commons-dbcp?


Is there any other way to address this issue?

Thanks

Antonio - https://kmidata.es/

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #18 from Mark Thomas  ---
Tomcat is calling write until 0 bytes are written before adding to the poller
so that isn't what is going on here.

(In reply to Remy Maucherat from comment #12)
> By default on my Linux the socket send buffer size is 2304. For NIO to have
> the same "performance" and behavior running the test [as seen with the debug
> logs] as NIO2, the send buffer size needs to be exactly >= to 4737.

On my Linux box the default send buffer size is 1313280. With the stand-alone
test, I see slowness if the buffer size < 4537 and no slowness when it is
>=4537.

It does seem that performance is related to how many times we have to call
select(). With a larger buffer size, the network write seems to be more
efficient so there are far fewer calls to select().

I'd expect the stand-alone test to be writing faster than Tomcat and therefore
need a larger buffer to avoid the slowness. I want to add use of a executor for
the writes to see if that gives more realistic results.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #17 from Michael Osipov  ---
(In reply to Mark Thomas from comment #14)
> I think I have a fix. I've updated the test to keep writing until the
> non-blocking write returns 0 bytes written and only then add the connection
> to the Poller. Performance is back to normal for me even with the
> (relatively) small buffer.
> 
> Can you test this on your platforms as well?
> 
> Meanwhile, I'm going to be looking at implementing a similar change in the
> Tomcat code.

The file did not change, did you push your changes to remote?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #16 from Michael Osipov  ---
(In reply to Remy Maucherat from comment #15)
> The performance numbers would still indicate FreeBSD has a specific poller
> issue of some kind though, the difference is too large there.

I will inquire this with the port maintainer afterwards.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #15 from Remy Maucherat  ---
The performance numbers would still indicate FreeBSD has a specific poller
issue of some kind though, the difference is too large there.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #14 from Mark Thomas  ---
I think I have a fix. I've updated the test to keep writing until the
non-blocking write returns 0 bytes written and only then add the connection to
the Poller. Performance is back to normal for me even with the (relatively)
small buffer.

Can you test this on your platforms as well?

Meanwhile, I'm going to be looking at implementing a similar change in the
Tomcat code.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #13 from Michael Osipov  ---
(In reply to Mark Thomas from comment #11)
> Simpler test case that reproduces the issue
> 
> https://github.com/markt-asf/tomcat-bugs/blob/master/src/java/org/apache/
> tomcat/Bug63916NioPoller.java
> 
> Run the test case and connect with telnet to localhost 8080.
> Uncomment the line that sets the send buffer to 8k and watch the time taken
> to write the 10MB jump from ~400ms to ~50s.
> 
> Tomorrow's task is to experiment to see if I can figure out the root cause
> of the time difference.

Ran the test on the same FreeBSD server:

Without explicit buffer size:
> Default send buffer size is [49032]
> Writing 10MB took [2601] milliseconds
With explicit buffer size of 8 KiB:
> Writing 10MB took [135095] milliseconds

That's insane!

On HP-UX with Java 8 for comparison:
Without explicit buffer size:
> Default send buffer size is [32768]
> Writing 10MB took [2851] milliseconds
With explicit buffer size of 8 KiB:
> Writing 10MB took [89457] milliseconds

and on RHEL 7, Java 8u232:
Without explicit buffer size:
> Default send buffer size is [1313280]
> Writing 10MB took [661] milliseconds
With explicit buffer size of 8 KiB:
Writing 10MB took [50696] milliseconds

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63916] org.apache.catalina.nonblocking.TestNonBlockingAPI abysmally slow with NIO/NIO2 on FreeBSD

2019-11-13 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63916

--- Comment #12 from Remy Maucherat  ---
By default on my Linux the socket send buffer size is 2304. For NIO to have the
same "performance" and behavior running the test [as seen with the debug logs]
as NIO2, the send buffer size needs to be exactly >= to 4737.

Note: CI used to show this behavior but that's no longer the case, now NIO and
NIO2 are equal and APR is quite a bit behind.
https://ci.apache.org/projects/tomcat/tomcat9/logs/4721/TEST-org.apache.catalina.nonblocking.TestNonBlockingAPI.NIO.txt
https://ci.apache.org/projects/tomcat/tomcat9/logs/4721/TEST-org.apache.catalina.nonblocking.TestNonBlockingAPI.NIO2.txt
https://ci.apache.org/projects/tomcat/tomcat9/logs/4721/TEST-org.apache.catalina.nonblocking.TestNonBlockingAPI.APR.txt

I have yet to notice any performance impact at all in a "real" test, NIO async
with HTTP/2 is very fine performance wise compared to NIO2, same as sendfile,
etc.

Also at this point given the difference and most importantly that NIO2 is also
affected, I would say FreeBSD's Java IO might be broken. AFAIK it is not a
platform directly supported by OpenJDK.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org