I have isolated the problem to text/.* Content-types, but only as these apply to the /geoserver/mapml/ controller; /geoserver/openlayers/*.css or text/html responses are not affected by the following problem:
some part of the call stack is setting the Content-length header to the size of the ungzipped response. As a result, the client is waiting for the keep-alive timeout value (20 seconds) then closing and rendering the response. I can't figure out how to prevent the header from being set, even by commenting out the many places in the GZIPResponseWrapper where it may be set. It gets set outside the call into the GZIPFilter. Nor can I figure out how to set the header value to the correct figure which would be the size of the compressed response. I can operationally work around this by not compressing HTML and CSs, but it will bug me, and be trouble for users. I will continue to try to find a fix. Cheers Peter On Tue, Mar 21, 2023, 5:13 a.m. Andrea Aime < andrea.a...@geosolutionsgroup.com> wrote: > If that's the case, then the changes to GSIPResponseOutputStream might be > the cause? > But I don't see how: > https://github.com/geoserver/geoserver/pull/6515/files#diff-40e2171fa39da85e5fd855ddfab206e350b990149834a4f16c1e64e0ee72cb24 > > The upgrade of Spring version required an update to the servet api itself, > which caused the two extra methods > to be added. Maybe there is some behavioral change there? > > I've also grepped the various jstacks attached for any trace of GeoServer > activity, but came out empty, e.g: > > grep -i "org.geoserver" * > > returns nothing... It looks like GeoServer was completely inactive while > they were taken. > I've also checked out other compressible output formats, like HTML or > GeoJSON, and I'm seeing no slowdown, e.g. (demo server on the main branch > here): > > > https://gs-main.geosolutionsgroup.com/geoserver/ogc/features/v1/collections/topp:states/items?f=text%2Fhtml > > https://gs-main.geosolutionsgroup.com/geoserver/ogc/features/v1/collections/topp:states/items?f=json > > Is there anything else you have going on in the demo server? Changes in > web.xml? Other plugins? > Anything special about how mapml writes onto the output? > > Cheers > Andrea > > > > On Mon, Mar 20, 2023 at 9:41 PM Peter Rushforth <peter.rushfo...@gmail.com> > wrote: > >> Hi >> >> Sorry I didn't copy the list in my last email, my mistake. >> >> Anyway, I looked at the Spring changes and nothing really stands out to >> me, although there is some logging on main that didn't occur in 2.22.x. It >> might still have something to do with that, but because I noticed in >> web.xml that text/.* was being compressed by >> the org.geoserver.filters.GZIPFilter filter, but .js was not, I tried >> removing the compression on text.* and sure enough the delay disappears >> from main / problem is gone for mapml previews. >> >> I don't think it's desirable to not compress html and css etc, so I will >> keep looking, but please let me know if anything occurs to you. >> >> Peter >> >> On Mon, Mar 20, 2023 at 12:24 PM Peter Rushforth < >> peter.rushfo...@gmail.com> wrote: >> >>> A couple of logs with GEOSERVER_DEVELOPER_LOGGING enabled, the first >>> with a (custom) build from the 2.22.x branch. The customization is that I >>> include some geoserver.mapml logging statements. The second is from main >>> today. >>> They compare pretty closely, except for the big pause. One difference >>> I've noticed is the 2.22.x branch's: >>> >>> "[filter.GeoServerSecurityContextPersistenceFilter$1] - >>> SecurityContextHolder now cleared, as request processing completed" >>> >>> log statements don't have any parallel in today's 2.24.SNAPSHOT (main) >>> log, so I'm wondering if there is some resource that is being withheld for >>> many / all the resources. >>> >>> Anyway, will keep looking. >>> >>> Thanks >>> Peter >>> >>> On Mon, Mar 20, 2023 at 11:14 AM Peter Rushforth < >>> peter.rushfo...@gmail.com> wrote: >>> >>>> Hi Andrea, and Jody, >>>> >>>> I am running a build off main today, as it has the same behaviour as >>>> 2.23-RC1. >>>> >>>> I ran a few jstack commands during execution of the tomcat process id, >>>> first at idle (jstack-idle.out), then during a mapml preview request >>>> (jstack-mapml-out to jstack-mapml.out12) and a couple during just the >>>> mapml.css request at the URL bar (jstack-css.out). >>>> >>>> I attach these files, hopefully the list will allow them. They don't >>>> tell me much, but if you can spot something, please let me know. >>>> >>>> I will see if I can profile the app during the same events, for >>>> comparison. >>>> >>>> Thanks for your help! >>>> Peter >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Mar 20, 2023 at 6:48 AM Andrea Aime < >>>> andrea.a...@geosolutionsgroup.com> wrote: >>>> >>>>> Hi Peter, >>>>> can you run a few jstack during the slowdown? Would help us figure out >>>>> what is slowing it down. >>>>> >>>>> Has anyone else noticed slowdowns in other parts of GeoServer yet? >>>>> >>>>> Cheers >>>>> Andrea >>>>> >>>>> On Mon, Mar 20, 2023 at 8:33 AM Jody Garnett <jody.garn...@gmail.com> >>>>> wrote: >>>>> >>>>>> Peter: >>>>>> >>>>>> Thanks for noticing the regression; the spring upgrade is indeed the >>>>>> big internal change this release cycle. Are you in position to do any >>>>>> profiling to determine where the slowdown is occurring? You may be able >>>>>> to >>>>>> use git bisect to determine the exact change that introduced the >>>>>> slowdown. >>>>>> >>>>>> I would like to schedule the 2.23.0 release later this month. >>>>>> -- >>>>>> Jody Garnett >>>>>> >>>>>> >>>>>> On Fri, Mar 17, 2023 at 7:38 AM Peter Rushforth < >>>>>> peter.rushfo...@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am the maintainer of the MapML module. In testing 2.23-RC1, I >>>>>>> noticed a long delay that occurs during processing of pretty simple >>>>>>> requests to the module. I have checked out and built the HEAD of the >>>>>>> 2.22.x branch, and the delay does not occur there. Since the MapML >>>>>>> module >>>>>>> is a Spring REST controller, I was wondering if it might have been >>>>>>> negatively impacted by a spring upgrade? >>>>>>> >>>>>>> Definitely willing to change the code to play nice with the new >>>>>>> spring. >>>>>>> >>>>>>> Any advice on how to test / evaluate this is welcome. >>>>>>> >>>>>>> Thank you >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Geoserver-devel mailing list >>>>>>> Geoserver-devel@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>>>>>> >>>>>> _______________________________________________ >>>>>> Geoserver-devel mailing list >>>>>> Geoserver-devel@lists.sourceforge.net >>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Regards, >>>>> >>>>> Andrea Aime >>>>> >>>>> == >>>>> GeoServer Professional Services from the experts! >>>>> >>>>> Visit http://bit.ly/gs-services-us for more information. >>>>> == >>>>> >>>>> Ing. Andrea Aime >>>>> @geowolf >>>>> Technical Lead >>>>> >>>>> GeoSolutions Group >>>>> phone: +39 0584 962313 >>>>> >>>>> fax: +39 0584 1660272 >>>>> >>>>> mob: +39 339 8844549 >>>>> >>>>> https://www.geosolutionsgroup.com/ >>>>> >>>>> http://twitter.com/geosolutions_it >>>>> >>>>> ------------------------------------------------------- >>>>> >>>>> Con riferimento alla normativa sul trattamento dei dati personali >>>>> (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati >>>>> “GDPR”), >>>>> si precisa che ogni circostanza inerente alla presente email (il suo >>>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >>>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >>>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >>>>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia. >>>>> >>>>> This email is intended only for the person or entity to which it is >>>>> addressed and may contain information that is privileged, confidential or >>>>> otherwise protected from disclosure. We remind that - as provided by >>>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of >>>>> this >>>>> e-mail or the information herein by anyone other than the intended >>>>> recipient is prohibited. If you have received this email by mistake, >>>>> please >>>>> notify us immediately by telephone or e-mail >>>>> >>>> > > -- > > Regards, > > Andrea Aime > > == > GeoServer Professional Services from the experts! > > Visit http://bit.ly/gs-services-us for more information. > == > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions Group > phone: +39 0584 962313 > > fax: +39 0584 1660272 > > mob: +39 339 8844549 > > https://www.geosolutionsgroup.com/ > > http://twitter.com/geosolutions_it > > ------------------------------------------------------- > > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE > 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si > precisa che ogni circostanza inerente alla presente email (il suo > contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è > riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il > messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra > operazione è illecita. Le sarei comunque grato se potesse darmene notizia. > > This email is intended only for the person or entity to which it is > addressed and may contain information that is privileged, confidential or > otherwise protected from disclosure. We remind that - as provided by > European Regulation 2016/679 “GDPR” - copying, dissemination or use of this > e-mail or the information herein by anyone other than the intended > recipient is prohibited. If you have received this email by mistake, please > notify us immediately by telephone or e-mail >
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
