Hi, On Tue, Sep 27, 2022 at 12:35 AM Glen Peterson <glen.k.peter...@gmail.com> wrote: > > Thank you for Jetty. It is wonderful! I especially like configuring it in > code. > > Running Jetty 11.0.11 (possibly also saw this with 11.0.9??) > > Our server seems to have partially or completely hung on Aug 19 and again > today (Sept 26 2022). I don't fully understand why. Right before the "hang" > I see a bunch of errors in the logs like: > > o.e.j.s.HttpChannel [HttpChannel.java:802] handleException /cov/25.png > java.io.EOFException: reset
Are you using HTTP/2? This is typically reported by the server when the client aborts the request. > We had adequate disk space (only 16% full), CPUs mostly idle. I forgot to > check memory consumption. Docker container (adoptopenjdk/openjdk11:alpine) > seemed to be running fine. > > I was able to reproduce this on my own machine (Ubuntu 18.04) running through > IntelliJ, I think with Java 11. I did it by requesting maybe 3 or 4 thousand > images from a singe HTML page. Sometimes I'd have to click on the page 2 or > 3 times to get the hang. > > Setting Jetty to DEBUG produced a lot of output, but nothing obviously > useful. EXCEPT: it slowed down the system significantly and wouldn't hang! > > Tried connecting jconsole to the running app. First time loading the Images > screen for block-images, all good, but went up very quickly to 146 live > threads. Second time, it looked like a garbage collection happened, and the > CPU usage dropped to zero. See attachment from 4:40PM. Out of ideas on that > for now. > > Using Chrome dev tools, in the network tab I can see a request sent , but no > response received from the server. This is normal if the client sent a RST_STREAM for that request, which appears to be your case. > On the server side, I see the request sent and even processed (meaning the > end of my servlet handle() method, but somehow, nothing is returned by Jetty. > The request should have been written to buffer, and maybe even some of that > buffer returned by Jetty by then, but nothing is returned. Hmm... It looks > like maybe the server is hung? > > After a minute or two, Chrome's console gave me over 1K errors along the > lines of: > GET https://my.site.com:8443/img/3024.png net::ERR_FAILED Images:3049 > Chrome appears to have crashed too, which doesn't help. > > I thought I could do various things on my end to lighten the load, or not > goof things up, but nothing I tried made any difference. Can you take a thread dump via JVM tools (jstack), or better a server dump as explained here: https://www.eclipse.org/jetty/documentation/jetty-11/operations-guide/index.html#og-troubleshooting-dump -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz _______________________________________________ jetty-users mailing list jetty-users@eclipse.org To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jetty-users
