Test TestNonBlockingAPI (testNonBlockingWriteError02NoSwallow, testNonBlockingWriteError02Swallow) fail on Solaris 11

[Rainer Jung]

The tests testNonBlockingWriteError02NoSwallow() and 
testNonBlockingWriteError02Swallow() of TestNonBlockingAPI fail for me 
on Solaris 11. I am using Java 11 and TC 10.1.4 but it seems it is not 
specific to these. The test loops unterminated doing flush on Solaris 
until after 60 seconds the test server gets shutdown.


The access log entry for Solaris is

127.0.0.1 - - [12/Dec/2022:23:36:18 +0100] "POST / HTTP/1.1" 500 - 
http-nio-127.0.0.1-auto-1-exec-2 1012864


for Linux

127.0.0.1 - - [12/Dec/2022:23:28:29 +0100] "POST / HTTP/1.1" 500 - 
http-nio-127.0.0.1-auto-1-exec-2 208015


so the Solaris line does not reflect this looping for 60 seconds.

I added some debug logging to the TestWriteListener02.onWritePossible(), 
which follows. If there is more I can do to analyze please let me know. 
I assume a BZ would be good?


First the patch that generates the output lines:

diff --git 
a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java 
b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java

index 0db6faa900..cc8ba0d0b0 100644
--- a/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
+++ b/test/org/apache/catalina/nonblocking/TestNonBlockingAPI.java
@@ -1659,27 +1659,36 @@ public class TestNonBlockingAPI extends 
TomcatBaseTest {

 @Override
 public void onWritePossible() throws IOException {
 try {
+log.info("TestWriteListener02 onWritePossible");
 ServletOutputStream sos = 
ac.getResponse().getOutputStream();

 do {
+log.info("sos.isReady() stage.get()==" + stage.get());
 if (stage.get() == 0) {
+log.info("Committing");
 // Commit the response
 ac.getResponse().flushBuffer();
 responseCommitLatch.countDown();
 stage.incrementAndGet();
 } else if (stage.get() == 1) {
+log.info("Waiting for client drop");
 // Wait for the client to drop the connection
 try {
+log.info("Calling clientCloseLatch.await()");
 clientCloseLatch.await();
 } catch (InterruptedException e) {
+log.info("Got InterruptedException", e);
 // Ignore
 }
+log.info("Writing TEST");
 sos.print("TEST");
 stage.incrementAndGet();
 } else if (stage.get() == 2) {
+log.info("Flushing");
 sos.flush();
 }
 } while (sos.isReady());
 } catch (IOException ioe) {
+log.info("IOException", ioe);
 if (!swallowIoException) {
 throw ioe;
 }

This shows the following behavior on Solaris:

[junit] 12-Dec-2022 23:36:22.753 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.AbstractProtocol$ConnectionHandler.process Created new 
processor [org.apache.coyote.http11.Http11Processor@71be62ce]
[junit] 12-Dec-2022 23:36:22.754 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.http11.Http11InputBuffer.fill Before fill(): 
parsingHeader: [true], parsingRequestLine: [true], 
parsingRequestLinePhase: [0], parsingRequestLineStart: [0], 
byteBuffer.position(): [0], byteBuffer.limit(): [0], end: [0]
[junit] 12-Dec-2022 23:36:22.755 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.net.SocketWrapperBase.populateReadBuffer Socket: 
[org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@308368f5:org.apache.tomcat.util.net.NioChannel@7f798351:java.nio.channels.SocketChannel[connected 
local=/127.0.0.1:48778 remote=/127.0.0.1:52071]], Read from buffer: [0]
[junit] 12-Dec-2022 23:36:22.757 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.read Socket: 
[org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@308368f5:org.apache.tomcat.util.net.NioChannel@7f798351:java.nio.channels.SocketChannel[connected 
local=/127.0.0.1:48778 remote=/127.0.0.1:52071]], Read direct from 
socket: [36]
[junit] 12-Dec-2022 23:36:22.757 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.coyote.http11.Http11InputBuffer.fill Received [GET / HTTP/1.1

[junit] Host: localhost:
[junit]
[junit] ]
[junit] 12-Dec-2022 23:36:22.759 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.tomcat.util.http.Parameters.setQueryStringCharset Set query 
string encoding to UTF-8
[junit] 12-Dec-2022 23:36:22.762 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 
org.apache.catalina.authenticator.AuthenticatorBase.invoke Security 
checking request GET /
[junit] 12-Dec-2022 23:36:22.763 FINE 
[http-nio-127.0.0.1-auto-3-exec-1] 

[tomcat] branch main updated: Add support for Servlet read/write via ByteBuffer

[markt]

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
 new 2d931eed6c Add support for Servlet read/write via ByteBuffer
2d931eed6c is described below

commit 2d931eed6c35702ef6541c0fd6c067b79e5eb371
Author: Mark Thomas 
AuthorDate: Mon Dec 12 15:43:28 2022 +

Add support for Servlet read/write via ByteBuffer
---
 java/jakarta/servlet/ServletInputStream.java   | 141 -
 java/jakarta/servlet/ServletOutputStream.java  | 101 ++-
 .../catalina/connector/CoyoteInputStream.java  |  14 +-
 .../catalina/connector/CoyoteOutputStream.java |   6 +
 .../org/apache/catalina/connector/InputBuffer.java |   4 +
 .../catalina/connector/LocalStrings.properties |   1 +
 java/org/apache/coyote/LocalStrings.properties |   1 -
 java/org/apache/coyote/Response.java   |   5 +-
 .../apache/coyote/http11/Http11OutputBuffer.java   |   7 +
 webapps/docs/changelog.xml |  11 ++
 10 files changed, 264 insertions(+), 27 deletions(-)

diff --git a/java/jakarta/servlet/ServletInputStream.java 
b/java/jakarta/servlet/ServletInputStream.java
index 0c8280c2e9..10048fbb88 100644
--- a/java/jakarta/servlet/ServletInputStream.java
+++ b/java/jakarta/servlet/ServletInputStream.java
@@ -18,6 +18,8 @@ package jakarta.servlet;
 
 import java.io.IOException;
 import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.util.Objects;
 
 /**
  * Provides an input stream for reading binary data from a client request,
@@ -43,6 +45,80 @@ public abstract class ServletInputStream extends InputStream 
{
 // NOOP
 }
 
+/**
+ * Reads from the input stream into the given buffer.
+ * 
+ * If the input stream is in non-blocking mode, before each invocation of
+ * this method {@link #isReady()} must be called and must return
+ * {@code true} or the {@link ReadListener#onDataAvailable()} call back 
must
+ * indicate that data is available to read else an
+ * {@link IllegalStateException} must be thrown.
+ * 
+ * Otherwise, if this method is called when {@code buffer} has no space
+ * remaining, the method returns {@code 0} immediately and {@code buffer} 
is
+ * unchanged.
+ * 
+ * If the input stream is in blocking mode and {@code buffer} has space
+ * remaining, this method blocks until at least one byte has been read, end
+ * of stream is reached or an exception is thrown.
+ * 
+ * Returns the number of bytes read or {@code -1} if the end of stream is
+ * reached without reading any data.
+ * 
+ * When the method returns, and if data has been read, the buffer's 
position
+ * will be unchanged from the value when passed to this method and the 
limit
+ * will be the position incremented by the number of bytes read.
+ * 
+ * Subclasses are strongly encouraged to override this method and provide a
+ * more efficient implementation.
+ *
+ * @param buffer The buffer into which the data is read.
+ *
+ * @return The number of bytes read or {@code -1} if the end of the stream
+ * has been reached.
+ *
+ * @exception IllegalStateException If the input stream is in non-blocking
+ * mode and this method is called without first calling {@link #isReady()}
+ * and that method has returned {@code true} or
+ * {@link ReadListener#onDataAvailable()} has not signalled that data is
+ * available to read.
+ *
+ * @exception IOException If data cannot be read for any reason other than
+ * the end of stream being reached, the input stream has been closed or if
+ * some other I/O error occurs.
+ *
+ * @exception NullPointerException If buffer is null.
+ *
+ * @since Servlet 6.1
+ */
+public int read(ByteBuffer buffer) throws IOException {
+Objects.requireNonNull(buffer);
+
+if (!isReady()) {
+throw new IllegalStateException();
+}
+
+if (buffer.remaining() == 0) {
+return 0;
+}
+
+byte[] b = new byte[buffer.remaining()];
+
+int result = read(b);
+if (result == -1) {
+return -1;
+}
+
+int position = buffer.position();
+
+buffer.put(b, 0, result);
+
+buffer.position(position);
+buffer.limit(position + result);
+
+return result;
+}
+
 /**
  * Reads the input stream, one line at a time. Starting at an offset, reads
  * bytes into an array, until it reads a certain number of bytes or reaches
@@ -50,6 +126,8 @@ public abstract class ServletInputStream extends InputStream 
{
  * 
  * This method returns -1 if it reaches the end of the input stream before
  * reading the maximum number of bytes.
+ * 
+ * This method may only be 

Re: Extended access log valve merged with AbstractAccessLogValve?

[Romain Manni-Bucau]

Basically it seems a few "accessor" (access log element) are only in
extended flavor and not the abstract one, for example
servletContext.getAttribute accessor.
The issue is then that abstract access log valve was created to enable to
extend and forward the access log to anything else than the built-in file
writer (a custom logger for ex).
Now, with these differences you don't get this abstraction and either need
to reimplement the pattern parsing or override a lot of the extended valve
since it is quite bound to the writer (open for ex).
So I'm looking to unify both worlds if possible.
I don't see a big negative effect to support tomcat extensions in the
abstract impl so I'm proposing it as an option.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le lun. 12 déc. 2022 à 11:25, Mark Thomas  a écrit :

> Can you expand on what problem you are trying to solve that you can't
> solve with the current code?
>
> Mark
>
>
> On 12/12/2022 09:51, Romain Manni-Bucau wrote:
> > Hi all,
> >
> > Wonder if there is any reason to not push supported element of
> > ExtendedAccessLogValve in the abstract log valve.
> > Rational would be to leverage "extensions" in a custom log sink impl
> (protected
> > abstract void log(CharArrayWriter message);).
> >
> > Wdyt?
> > Let me know if you are interested and I need to do a PR, would be more
> than
> > happy to help if relevant.
> >
> > Romain Manni-Bucau
> > @rmannibucau  |  Blog
> >  | Old Blog
> >  | Github <
> https://github.com/rmannibucau> |
> > LinkedIn  | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>

Re: Extended access log valve merged with AbstractAccessLogValve?

[Mark Thomas]

Can you expand on what problem you are trying to solve that you can't 
solve with the current code?


Mark


On 12/12/2022 09:51, Romain Manni-Bucau wrote:

Hi all,

Wonder if there is any reason to not push supported element of
ExtendedAccessLogValve in the abstract log valve.
Rational would be to leverage "extensions" in a custom log sink impl (protected
abstract void log(CharArrayWriter message);).

Wdyt?
Let me know if you are interested and I need to do a PR, would be more than
happy to help if relevant.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

JDK 20 Rampdown Phase 1 & Valhalla LW4 Early-Access builds

[David Delabassee]

Welcome to the final OpenJDK Quality Outreach update for 2022!

JDK 20, scheduled for General Availability on March 21 2023, is now in 
Rampdown Phase One (RDP1) [1]. At this point, the overall JDK 20 [2] 
feature set is frozen (see below the final list of JEPs integrated into 
JDK 20) and only low-risk enhancements might still be considered. The 
coming weeks should be used to identify and resolve as many issues as 
possible, i.e. before JDK 20 enters the Release Candidates phase in 
early February 2023.



## JDK 20 Early-Access builds

The latest Early-Access (builds 27) are available [2] with the Release 
Notes here [3]. Those builds are provided under the GNU GPL v2, with the 
Classpath Exception.


### JEPs integrated into JDK 20:

JEP 429: Scoped Values (Incubator)
JEP 432: Record Patterns (2nd Preview)
JEP 433: Pattern Matching for switch (4th Preview)
JEP 434: Foreign Function & Memory API (2nd Preview)
JEP 436: Virtual Threads (2nd Preview)
JEP 437: Structured Concurrency (2nd Incubator)

[1] https://mail.openjdk.org/pipermail/jdk-dev/2022-December/007233.html
[2] https://jdk.java.net/20/
[3] https://jdk.java.net/20/release-notes


### Changes in recent JDK 20 builds that may be of interest:

 Build 27:
- JDK-8297794: Deprecate JMX Management Applets for Removal
- JDK-8297118: Change IncompatibleClassChangeError to MatchException for 
exhaustive switch statements and switch expressions

- JDK-8294047: HttpResponseInputStream swallows interrupts
- JDK-8281236: (D)TLS key exchange named groups
- JDK-8280798: com.sun.jdi.ObjectReference::setValue spec should 
prohibit any final field modification

- JDK-8295350: JFR: Add stop methods for recording streams
- JDK-8295044: Implementation of Foreign Function and Memory API (2nd 
Preview)

- JDK-8296896: Change virtual Thread.yield to use external submit
- JDK-8297804: (tz) Update Timezone Data to 2022g
- JDK-8295803: Console should be usable in jshell and other environments
- JDK-828: Implementation of Scoped Values (Incubator)
- JDK-8296672: Implementation of Virtual Threads (2nd Preview)

 Build 26:
- JDK-8297276: Remove thread text from Subject.current
- JDK-8297030: Reduce Default Keep-Alive Timeout Value for httpclient
- JDK-8247645: ChaCha20 Intrinsics

 Build 25:
- JDK-8296472: Remove ObjectLocker around 
appendToClassPathForInstrumentation call
- JDK-8290313: Produce warning when user specified java.io.tmpdir 
directory doesn't exist
- JDK-8288717: Add a means to close idle connections in HTTP/2 
connection pool

- JDK-8288047: Accelerate Poly1305 on x86_64 using AVX512 instructions
- JDK-8059632: Method reference compilation uses incorrect qualifying type
- JDK-8297161: Add additional Service Attributes to Standard Algorithm 
Names guide

- JDK-8294073: Performance improvement for message digest implementations

 Build 24:
- JDK-8294731: Improve multiplicative inverse for secp256r1 implementation
- JDK-8296715: CLDR v42 update for tzdata 2022f
- JDK-8296958: [JVMCI] add API for retrieving ConstantValue attributes

 Build 23:
- JDK-8296226: Add constructors (String,Throwable) and (Throwable) to 
InvalidParameterException
- JDK-8295673: Deprecate and disable legacy parallel class loading 
workaround for non-parallel-capable class loaders

- JDK-8294241: Deprecate URL public constructors
- JDK-8289689: (fs) Re-examine the need for normalization to Unicode 
Normalization Format D (macOS)

- JDK-8279164: Disable TLS_ECDH_* cipher suites
- JDK-8178355: IdentityHashMap uses identity-based comparison for values 
everywhere except remove(K,V) and replace(K,V,V)

- JDK-8296108: (tz) Update Timezone Data to 2022f


## Heads-up - JDK 21: First Early-Access Builds

When JDK 20 entered RDP1 [4], the JDK mainline [5] was (a) forked into a 
JDK 20 stabilization repository [6], and (b) set to JDK 21. As a 
consequence, the first JDK 21 Early-Access builds have been published [7].


[4] https://mail.openjdk.org/pipermail/jdk-dev/2022-December/007233.html
[5] https://github.com/openjdk/jdk
[6] https://github.com/openjdk/jdk20
[7] https://jdk.java.net/21/


## Heads-up - Valhalla: LW4 Early-Access Builds

Valhalla LW4 early-access builds have been published [8], those builds 
are primarily focused on implementing the Value Objects JEP draft [9]. 
For additional details on those EA builds, make sure to read these LW4 
release notes [10]. For a more hands-on introduction to Value Object, 
you can watch the latest JEP Café: Java Value Objects in Action [11]. 
Interested developers are encouraged to explore the performance and 
migration impact of value objects on their applications, and to provide 
feedback to the valhalla-dev [12] mailing list.


[8] https://jdk.java.net/valhalla/
[9] https://openjdk.org/jeps/8277163
[10] https://openjdk.org/projects/valhalla/early-access
[11] https://inside.java/2022/12/06/jepcafe15/
[12] https://mail.openjdk.org/pipermail/valhalla-dev/


## Heads-up - Generational ZGC: New Early-Access Builds

New 

Extended access log valve merged with AbstractAccessLogValve?

[Romain Manni-Bucau]

Hi all,

Wonder if there is any reason to not push supported element of
ExtendedAccessLogValve in the abstract log valve.
Rational would be to leverage "extensions" in a custom log sink impl (protected
abstract void log(CharArrayWriter message);).

Wdyt?
Let me know if you are interested and I need to do a PR, would be more than
happy to help if relevant.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book