Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 31/03/2020 14:36, Manuel Dominguez Sarmiento wrote:
> Thanks, so we would be able to log this with AccessLogValve using the
> following patterns, right?
> %{org.apache.coyote.connectionID}r
> %{org.apache.coyote.streamID}r
Correct.
Mark
>
> *Manuel Dominguez Sarmiento*
>
> On 31/03/2020 10:28, Mark Thomas wrote:
>> On 29/03/2020 16:16, Mark Thomas wrote:
>>> On 29/03/2020 15:58, Manuel Dominguez Sarmiento wrote:
> Do the streams with errors all share the same connection ID? I suspect
> an error on one stream is triggering the closure of the connection and
> what you are seeing is the fallout of closing the connection.
That makes sense and is kind of what we're suspecting. How can we check
the connection/stream id? Can this be logged by AccessLogValve?
>>> Not easily at the moment. That looks like something we should fix. I'll
>>> see what I can do for 9.0.34.
>> From 8.5.54, 9.0.34 and 10.0.0-M4 the following request attributes will
>> be available for HTTP/2
>>
>> org.apache.coyote.connectionID
>> org.apache.coyote.streamID
>>
>> Mark
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Thanks, so we would be able to log this with AccessLogValve using the
following patterns, right?
%{org.apache.coyote.connectionID}r
%{org.apache.coyote.streamID}r
*Manuel Dominguez Sarmiento*
On 31/03/2020 10:28, Mark Thomas wrote:
On 29/03/2020 16:16, Mark Thomas wrote:
On 29/03/2020 15:58, Manuel Dominguez Sarmiento wrote:
Do the streams with errors all share the same connection ID? I suspect
an error on one stream is triggering the closure of the connection and
what you are seeing is the fallout of closing the connection.
That makes sense and is kind of what we're suspecting. How can we check
the connection/stream id? Can this be logged by AccessLogValve?
Not easily at the moment. That looks like something we should fix. I'll
see what I can do for 9.0.34.
From 8.5.54, 9.0.34 and 10.0.0-M4 the following request attributes will
be available for HTTP/2
org.apache.coyote.connectionID
org.apache.coyote.streamID
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 29/03/2020 16:16, Mark Thomas wrote: > On 29/03/2020 15:58, Manuel Dominguez Sarmiento wrote: >> >>> Do the streams with errors all share the same connection ID? I suspect >>> an error on one stream is triggering the closure of the connection and >>> what you are seeing is the fallout of closing the connection. >> That makes sense and is kind of what we're suspecting. How can we check >> the connection/stream id? Can this be logged by AccessLogValve? > > Not easily at the moment. That looks like something we should fix. I'll > see what I can do for 9.0.34. >From 8.5.54, 9.0.34 and 10.0.0-M4 the following request attributes will be available for HTTP/2 org.apache.coyote.connectionID org.apache.coyote.streamID Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 29/03/2020 15:58, Manuel Dominguez Sarmiento wrote: > >> Do the streams with errors all share the same connection ID? I suspect >> an error on one stream is triggering the closure of the connection and >> what you are seeing is the fallout of closing the connection. > That makes sense and is kind of what we're suspecting. How can we check > the connection/stream id? Can this be logged by AccessLogValve? Not easily at the moment. That looks like something we should fix. I'll see what I can do for 9.0.34. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Do the streams with errors all share the same connection ID? I suspect an error on one stream is triggering the closure of the connection and what you are seeing is the fallout of closing the connection. That makes sense and is kind of what we're suspecting. How can we check the connection/stream id? Can this be logged by AccessLogValve? *Manuel Dominguez Sarmiento *
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 28/03/2020 14:00, Manuel Dominguez Sarmiento wrote:
> Hi Mark, we're now on the latest 9.0.33 release and we still see this
> issue intermittently in our logs. Only on HTTP/2 secure requests.
>
> Please see the attached access logs (these represent all the cases for
> one whole day in one single high-volume server).
> Some of the following request fields are NULL (or -1) in these examples:
> - remoteAddr
> - remotePort
> - serverPort
> - requestURI
> - User-Agent
>
> Some requests are missing some of the fields, some of the requests are
> missing others. What is particularly interesting is that the errors are
> clustered around particular timestamps, pointing to some likely issue
> regarding object sharing across several requests.
Do the streams with errors all share the same connection ID? I suspect
an error on one stream is triggering the closure of the connection and
what you are seeing is the fallout of closing the connection.
Mark
>
> Please note that this is not just an issue at the AccessLogValve level.
> These fields contain invalid data while the request is being processed,
> so that is causing unexpected exceptions in our production code. The
> cases are few and isolated, but still this should be looked into.
>
> Any thoughts?
>
> *Manuel Dominguez Sarmiento*
>
> On 05/02/2020 14:12, Manuel Dominguez Sarmiento wrote:
>> Our filter is not doing anything fancy (and it has always worked
>> correctly before we ran into this bug). In pseudo-code:
>>
>> public doFilter(request, response) {
>>
>> String ip = request.getRemoteAddr();
>> boolean isProxy = isProxy(ip);
>> if (isProxy) {
>> String unwrappedIP = unwrapXForwardedFor(request);
>> chain.doFilter(new
>> MobileProxyHidingServletRequestWrapper(request, unwrappedIP), response);
>> } else {
>> chain.doFilter(request, response);
>> }
>> }
>>
>> All that MobileProxyHidingServletRequestWrapper is override
>> getRemoteAddr() returning unwrappedIP instead of delegating to the
>> actual request, while unwrapXForwardedFor() does what the name
>> suggests, which is processing X-Forwarded-For to obtain the
>> originating IP before it hit the detected proxy.
>>
>> *Manuel Dominguez Sarmiento*
>>
>> On 05/02/2020 10:28, Mark Thomas wrote:
>>> On 04/02/2020 22:27, Manuel Dominguez Sarmiento wrote:
We are getting the NPEs in a top-of-the-chain servlet filter which
decorates HttpServletRequest.getRemoteAddr() before actual servlet
processing. Only on HTTP/2 and in a very small number of cases. Perhaps
we should test 9.0.31 and see what happens. When is this new version due
for release?
>>> I'm just working through back-porting some changes and then I'll be
>>> starting the release process. It 9.0.31 should be available towards the
>>> beginning of next week.
>>>
>>> Can you expand on what your filter is doing? When is the call made to
>>> HttpServletRequest.getRemoteAddr() on the original request?
>>>
>>> Mark
>>>
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Hi Mark, we're now on the latest 9.0.33 release and we still see this
issue intermittently in our logs. Only on HTTP/2 secure requests.
Please see the attached access logs (these represent all the cases for
one whole day in one single high-volume server).
Some of the following request fields are NULL (or -1) in these examples:
- remoteAddr
- remotePort
- serverPort
- requestURI
- User-Agent
Some requests are missing some of the fields, some of the requests are
missing others. What is particularly interesting is that the errors are
clustered around particular timestamps, pointing to some likely issue
regarding object sharing across several requests.
Please note that this is not just an issue at the AccessLogValve level.
These fields contain invalid data while the request is being processed,
so that is causing unexpected exceptions in our production code. The
cases are few and isolated, but still this should be looked into.
Any thoughts?
*Manuel Dominguez Sarmiento*
On 05/02/2020 14:12, Manuel Dominguez Sarmiento wrote:
Our filter is not doing anything fancy (and it has always worked
correctly before we ran into this bug). In pseudo-code:
public doFilter(request, response) {
String ip = request.getRemoteAddr();
boolean isProxy = isProxy(ip);
if (isProxy) {
String unwrappedIP = unwrapXForwardedFor(request);
chain.doFilter(new
MobileProxyHidingServletRequestWrapper(request, unwrappedIP), response);
} else {
chain.doFilter(request, response);
}
}
All that MobileProxyHidingServletRequestWrapper is override
getRemoteAddr() returning unwrappedIP instead of delegating to the
actual request, while unwrapXForwardedFor() does what the name
suggests, which is processing X-Forwarded-For to obtain the
originating IP before it hit the detected proxy.
*Manuel Dominguez Sarmiento*
On 05/02/2020 10:28, Mark Thomas wrote:
On 04/02/2020 22:27, Manuel Dominguez Sarmiento wrote:
We are getting the NPEs in a top-of-the-chain servlet filter which
decorates HttpServletRequest.getRemoteAddr() before actual servlet
processing. Only on HTTP/2 and in a very small number of cases. Perhaps
we should test 9.0.31 and see what happens. When is this new version due
for release?
I'm just working through back-porting some changes and then I'll be
starting the release process. It 9.0.31 should be available towards the
beginning of next week.
Can you expand on what your filter is doing? When is the call made to
HttpServletRequest.getRemoteAddr() on the original request?
Mark
LOOKING FOR ALL ISSUE INSTANCES:
[root@optimus ~]# cat /home/wap/logs/access.2020-03-27.log | grep "^-"
- -1 443 [27/Mar/2020:07:53:12 -0300] "GET /us/en/country.do?method=list
HTTP/2.0" 400 762 "-" "Mozilla/5.0 (Linux; Android 6.0; vivo 1609)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.93 Mobile Safari/537.36"
- -1 443 [27/Mar/2020:10:48:12 -0300] "GET
/pe/es/subscriptionPlanDetail.do?id=4483=false=2181=46045=true=419634618870==ojo.pe=d=EAIaIQobChMIif6cyOW66AIVKAa5Bh3eRgI6EAEYASAAEgJuRPD_BwE
HTTP/2.0" 400 637 "https://ojo.pe/; "Mozilla/5.0 (Linux; Android 9; LM-X520
Build/PKQ1.190223.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0
Chrome/80.0.3987.119 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/260.0.0.42.118;]"
- -1 443 [27/Mar/2020:14:39:36 -0300] "GET
/cl/es/subscriptionPlanDetail.do?id=4120=false=2131=17450=false=45663=true=380011499904==mobileapp%3A%3A2-com.appstar.callrecorder=EAIaIQobChMI-aa0_8uN6AIVZga5Bh1UBwQ5EAEYASAAEgKAxPD_BwE_cl_smd_ok=32320413=32320413=705b26c82e98b8401b74a463a68180d6=1584044911681=CELLULAR=EFFECTIVE_4G=true
HTTP/2.0" 400 637
"https://wap.renxo.com/cl/es/subscriptionPlanDetail.do?id=4120=false=2131=17450=false=45663=true=380011499904==mobileapp%3A%3A2-com.appstar.callrecorder=EAIaIQobChMI-aa0_8uN6AIVZga5Bh1UBwQ5EAEYASAAEgKAxPD_BwE_cl_smd_ok=32320413=32320413=705b26c82e98b8401b74a463a68180d6=1584044911681=CELLULAR=EFFECTIVE_4G;
"Mozilla/5.0 (Linux; Android 8.1.0; SAMSUNG SM-J710MN Build/M1AJQ)
AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/9.4 Chrome/67.0.3396.87
Mobile Safari/537.36"
- -1 443 [27/Mar/2020:17:18:55 -0300] "GET
/ar/es/subscriptionPlanDetail.do?id=4328=16242=2403=48008=true=409370554249=%2Farts%20%26%20entertainment=cuttsite.website=d=EAIaIQobChMI98zj67y76AIVT4p3Ch3riAVXEAEYASAAEgLGavD_BwE
HTTP/2.0" 400 637
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Our filter is not doing anything fancy (and it has always worked
correctly before we ran into this bug). In pseudo-code:
public doFilter(request, response) {
String ip = request.getRemoteAddr();
boolean isProxy = isProxy(ip);
if (isProxy) {
String unwrappedIP = unwrapXForwardedFor(request);
chain.doFilter(new
MobileProxyHidingServletRequestWrapper(request, unwrappedIP), response);
} else {
chain.doFilter(request, response);
}
}
All that MobileProxyHidingServletRequestWrapper is override
getRemoteAddr() returning unwrappedIP instead of delegating to the
actual request, while unwrapXForwardedFor() does what the name suggests,
which is processing X-Forwarded-For to obtain the originating IP before
it hit the detected proxy.
*Manuel Dominguez Sarmiento*
On 05/02/2020 10:28, Mark Thomas wrote:
On 04/02/2020 22:27, Manuel Dominguez Sarmiento wrote:
We are getting the NPEs in a top-of-the-chain servlet filter which
decorates HttpServletRequest.getRemoteAddr() before actual servlet
processing. Only on HTTP/2 and in a very small number of cases. Perhaps
we should test 9.0.31 and see what happens. When is this new version due
for release?
I'm just working through back-porting some changes and then I'll be
starting the release process. It 9.0.31 should be available towards the
beginning of next week.
Can you expand on what your filter is doing? When is the call made to
HttpServletRequest.getRemoteAddr() on the original request?
Mark
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 04/02/2020 22:27, Manuel Dominguez Sarmiento wrote: > We are getting the NPEs in a top-of-the-chain servlet filter which > decorates HttpServletRequest.getRemoteAddr() before actual servlet > processing. Only on HTTP/2 and in a very small number of cases. Perhaps > we should test 9.0.31 and see what happens. When is this new version due > for release? I'm just working through back-porting some changes and then I'll be starting the release process. It 9.0.31 should be available towards the beginning of next week. Can you expand on what your filter is doing? When is the call made to HttpServletRequest.getRemoteAddr() on the original request? Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
We are getting the NPEs in a top-of-the-chain servlet filter which decorates HttpServletRequest.getRemoteAddr() before actual servlet processing. Only on HTTP/2 and in a very small number of cases. Perhaps we should test 9.0.31 and see what happens. When is this new version due for release? *- **Manuel Dominguez Sarmiento* Thanks Mark. I just wanted to clarify that the issue is not only present when the request arrives at AccessLogValve, but while the request is being serviced as well. We noticed this bug because we were getting random NullPointerExceptions when trying to do anything with the result from HttpServletRequest.getRemoteAddr() in our servlets. Should the fix solve this as well? It depends. The only way I can see this happening is if the application swallows an IOException and continues. It probably won't fix that and I don't think we'd consider an NPE an error in those circumstances. Mark
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 04/02/2020 15:37, Manuel Dominguez Sarmiento wrote: > Thanks Mark. I just wanted to clarify that the issue is not only present > when the request arrives at AccessLogValve, but while the request is > being serviced as well. > We noticed this bug because we were getting random NullPointerExceptions > when trying to do anything with the result from > HttpServletRequest.getRemoteAddr() in our servlets. > > Should the fix solve this as well? It depends. The only way I can see this happening is if the application swallows an IOException and continues. It probably won't fix that and I don't think we'd consider an NPE an error in those circumstances. Mark > > - Manuel Dominguez Sarmiento >> On 03/02/2020 19:24, Mark Thomas wrote: >> >> >> >>> I haven't fixed this but I can reproduce it easily with the h2spec test >>> suite. As I have a reproducible test case I'm hopeful I'll be able to >>> find a fix fairly quickly. >> I've just committed a fix. It is in: >> - master for 10.0.0.0-M1 onwards >> - 9.0.x for 9.0.31 onwards >> - 8.5.x for 8.5.51 onwards >> >> 7.0.x was not affected. >> >> The issue was that by the time the AccessLogValve tried to obtain the >> remote host/addr/port the connection had been closed and the information >> was not available. This was a result of the change below which removed a >> potential path for a request to see incorrect remote client information. >> >> This entry in the changelog: >> >> Decouple the socket wrapper, which is not recycled, from the NIOx >> channel after close, and replace it with a dummy static object. >> >> >> Mark >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Thanks Mark. I just wanted to clarify that the issue is not only present when the request arrives at AccessLogValve, but while the request is being serviced as well. We noticed this bug because we were getting random NullPointerExceptions when trying to do anything with the result from HttpServletRequest.getRemoteAddr() in our servlets. Should the fix solve this as well? - Manuel Dominguez Sarmiento On 03/02/2020 19:24, Mark Thomas wrote: I haven't fixed this but I can reproduce it easily with the h2spec test suite. As I have a reproducible test case I'm hopeful I'll be able to find a fix fairly quickly. I've just committed a fix. It is in: - master for 10.0.0.0-M1 onwards - 9.0.x for 9.0.31 onwards - 8.5.x for 8.5.51 onwards 7.0.x was not affected. The issue was that by the time the AccessLogValve tried to obtain the remote host/addr/port the connection had been closed and the information was not available. This was a result of the change below which removed a potential path for a request to see incorrect remote client information. This entry in the changelog: Decouple the socket wrapper, which is not recycled, from the NIOx channel after close, and replace it with a dummy static object. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 03/02/2020 19:24, Mark Thomas wrote: > I haven't fixed this but I can reproduce it easily with the h2spec test > suite. As I have a reproducible test case I'm hopeful I'll be able to > find a fix fairly quickly. I've just committed a fix. It is in: - master for 10.0.0.0-M1 onwards - 9.0.x for 9.0.31 onwards - 8.5.x for 8.5.51 onwards 7.0.x was not affected. The issue was that by the time the AccessLogValve tried to obtain the remote host/addr/port the connection had been closed and the information was not available. This was a result of the change below which removed a potential path for a request to see incorrect remote client information. This entry in the changelog: Decouple the socket wrapper, which is not recycled, from the NIOx channel after close, and replace it with a dummy static object. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 24/01/2020 12:26, Manuel Dominguez Sarmiento wrote: > Hi Mark, thanks for your feedback. Please see below: >> On 23/01/2020 13:40, Manuel Dominguez Sarmiento wrote: >>> Hi, we started >>> noticing that HttpServletRequest.getRemoteAddr() was >>> sometimes returning NULL (which is invalid according to the servlet >>> spec), about 20-50 times per day (we have high-load servers which >>> routinely handle over 100 requests per second). This only happens on >>> secure HTTP/2 requests. >>> Our server.xml is below (confidential data has been modified). We are >>> using the NIO connector, Tomcat Native + APR libraries (but NOT the APR >>> connector). The useAsyncIO flag is disabled because of server lockups >>> we've experienced with this flag enabled on our production servers. Any >>> ideas? >> Nothing comes to mind immediately. >> >> Mark > Please let me know if there's any diagnostic info that might help with > this and we'll produce it. Sorry it has taken me so long to get back to this. I haven't fixed this but I can reproduce it easily with the h2spec test suite. As I have a reproducible test case I'm hopeful I'll be able to find a fix fairly quickly. Kind regards, Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
Hi Mark, thanks for your feedback. Please see below: On 23/01/2020 13:40, Manuel Dominguez Sarmiento wrote:> Hi, we started noticing that HttpServletRequest.getRemoteAddr() was sometimes returning NULL (which is invalid according to the servlet spec), about 20-50 times per day (we have high-load servers which routinely handle over 100 requests per second). This only happens on secure HTTP/2 requests. We noticed this while running the latest 9.0.30, but looking at our logs for the past few months, this was also happening on 9.0.21 as well as 9.0.22 which are the previous versions we had deployed. This issue was not present in 9.0.17, which was the version we were using before 9.0.21. Before anyone cries out "there is a bug in your code", Tomcat itself reports a NULL remoteAddr via AccessLogValve in these cases. It is also interesting that the remotePort is reported as "-1". We looked through the changelog, but we can't figure out what change might have triggered this (presumably some change between 9.0.17 and 9.0.21). It looks like some kind of async race condition during the HTTP/2 upgrade, but this is only an educated guess. I'll take a look. You mention async. Are these requests using the Servlet async API? No, we're not using the async API in any capacity. Just the regular servlet API: doGet() / doPost() / etc. Perhaps my usage of "async" was confusing (as that can refer to several different things in Tomcat). I just meant that it feels like a race condition between different threads. Our server.xml is below (confidential data has been modified). We are using the NIO connector, Tomcat Native + APR libraries (but NOT the APR connector). The useAsyncIO flag is disabled because of server lockups we've experienced with this flag enabled on our production servers. Any ideas? Nothing comes to mind immediately. Mark Please let me know if there's any diagnostic info that might help with this and we'll produce it. Thanks again, - Manuel Dominguez Sarmiento
Re: HttpServletRequest.getRemoteAddr() sometimes returns NULL on Tomcat 9.0.30 and HTTP/2 secure requests
On 23/01/2020 13:40, Manuel Dominguez Sarmiento wrote:> Hi, we started noticing that HttpServletRequest.getRemoteAddr() was > sometimes returning NULL (which is invalid according to the servlet > spec), about 20-50 times per day (we have high-load servers which > routinely handle over 100 requests per second). This only happens on > secure HTTP/2 requests. > > We noticed this while running the latest 9.0.30, but looking at our logs > for the past few months, this was also happening on 9.0.21 as well as > 9.0.22 which are the previous versions we had deployed. This issue was > not present in 9.0.17, which was the version we were using before 9.0.21. > > Before anyone cries out "there is a bug in your code", Tomcat itself > reports a NULL remoteAddr via AccessLogValve in these cases. It is also > interesting that the remotePort is reported as "-1". > > We looked through the changelog, but we can't figure out what change > might have triggered this (presumably some change between 9.0.17 and > 9.0.21). It looks like some kind of async race condition during the > HTTP/2 upgrade, but this is only an educated guess. I'll take a look. You mention async. Are these requests using the Servlet async API? > Our server.xml is below (confidential data has been modified). We are > using the NIO connector, Tomcat Native + APR libraries (but NOT the APR > connector). The useAsyncIO flag is disabled because of server lockups > we've experienced with this flag enabled on our production servers. Any > ideas? Nothing comes to mind immediately. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
