Hi Peter, had a quick look using the built-in Jetty container... I cannot reproduce the issue, the viewer is always fast. Debugging, I've set breakspoints in the Jetty Response objects dealing with headers, I've observed the following:
- GZIPResponseWrapper/AlternativesResponseStream is holding onto an eventual content-length attribute, by overriding all methods that allowing setting one (addHeader, setHeader, addIntHeader, setIntHeader). - It eventually calls the response.setContentLength only if the mime type is found to be non compressible, otherwise, the header won't be set. I could think that maybe a newer version of the Servlet API has other ways to set the header... I cannot find any straight method there, but this one caught my eye, and it's available only in Servlet 4: https://javadoc.io/static/javax.servlet/javax.servlet-api/4.0.1/javax/servlet/http/HttpServletResponse.html#setTrailerFields-java.util.function.Supplier- That allows setting headers at the end of response, Content-lenght seems like it would be a good candidate for it. Now, GeoServer uses Servlet 3.1 where that method is not available, but if you were to deploy GeoServer in Tomcat 10 with the automatic on the fly servlet API transformation (not recommended!) then I can see how that would escape the current management. Either that, or I've missed places where the content-length can be set, in general, if there is a method that escapes the wrapping, that would cause the header to be set while compression is activated. Just thinking out loud, mind!! Cheers Andrea On Thu, Mar 23, 2023 at 7:22 PM Peter Rushforth <peter.rushfo...@gmail.com> wrote: > Hi Andrea, > > yep, it's really odd. It seems that no matter what I do to the GZIPFilter > and it's components, the Content-Length header gets set, and committed, > outside of the actions of the filter itself. Also, although the web.xml > pattern for text/.* should match the text/mapml media type, and I do see it > enter the filter, what gets received by the client is not encoded. All of > this only happens, as far as I can tell, in URLs within the > /geoserver/mapml controller paths. Outside that, as you noticed, the > gzipping works / doesn't work as specified by the filter config in web.xml. > > I have to give up the chase at this point, because I'm out of ideas and > time. I will make an amendment to the user guide that notes to users that > they must remove the gzip filter from web.xml if they want to use the > extension. Eventually I'm certain more information will arrive and I'll be > able to fix it. > > Thanks for your help!! > Peter > > On Thu, Mar 23, 2023 at 5:00 AM Andrea Aime < > andrea.a...@geosolutionsgroup.com> wrote: > >> Hi Peter, >> it's odd, we have other parts of GeoServer that return HTML out of Spring >> MVC controllers without setting >> the content-length, e.g.: >> >> >> https://gs-main.geosolutionsgroup.com/geoserver/ogc/features/v1/collections?f=text%2Fhtml >> >> It is HTML, gzip compressed, and yet does not have a content-length >> header. Same goes for the administrative >> REST API (I don't have a link for that one, for obvious reasons). >> >> Checking the controllers set up, the only difference I see is that the >> MapML ones have the CrossOrigin annotation, >> but that seems unrelated... >> >> Cheers >> Andrea >> >> On Thu, Mar 23, 2023 at 4:47 AM Peter Rushforth < >> peter.rushfo...@gmail.com> wrote: >> >>> I have isolated the problem to text/.* Content-types, but only as these >>> apply to the /geoserver/mapml/ controller; /geoserver/openlayers/*.css or >>> text/html responses are not affected by the following problem: >>> >>> some part of the call stack is setting the Content-length header to the >>> size of the ungzipped response. As a result, the client is waiting for the >>> keep-alive timeout value (20 seconds) then closing and rendering the >>> response. >>> >>> I can't figure out how to prevent the header from being set, even by >>> commenting out the many places in the GZIPResponseWrapper where it may be >>> set. It gets set outside the call into the GZIPFilter. >>> >>> Nor can I figure out how to set the header value to the correct figure >>> which would be the size of the compressed response. >>> >>> I can operationally work around this by not compressing HTML and CSs, >>> but it will bug me, and be trouble for users. >>> >>> I will continue to try to find a fix. >>> >>> Cheers >>> Peter >>> >>> On Tue, Mar 21, 2023, 5:13 a.m. Andrea Aime < >>> andrea.a...@geosolutionsgroup.com> wrote: >>> >>>> If that's the case, then the changes to GSIPResponseOutputStream might >>>> be the cause? >>>> But I don't see how: >>>> https://github.com/geoserver/geoserver/pull/6515/files#diff-40e2171fa39da85e5fd855ddfab206e350b990149834a4f16c1e64e0ee72cb24 >>>> >>>> The upgrade of Spring version required an update to the servet api >>>> itself, which caused the two extra methods >>>> to be added. Maybe there is some behavioral change there? >>>> >>>> I've also grepped the various jstacks attached for any trace of >>>> GeoServer activity, but came out empty, e.g: >>>> >>>> grep -i "org.geoserver" * >>>> >>>> returns nothing... It looks like GeoServer was completely inactive >>>> while they were taken. >>>> I've also checked out other compressible output formats, like HTML or >>>> GeoJSON, and I'm seeing no slowdown, e.g. (demo server on the main branch >>>> here): >>>> >>>> >>>> https://gs-main.geosolutionsgroup.com/geoserver/ogc/features/v1/collections/topp:states/items?f=text%2Fhtml >>>> >>>> https://gs-main.geosolutionsgroup.com/geoserver/ogc/features/v1/collections/topp:states/items?f=json >>>> >>>> Is there anything else you have going on in the demo server? Changes in >>>> web.xml? Other plugins? >>>> Anything special about how mapml writes onto the output? >>>> >>>> Cheers >>>> Andrea >>>> >>>> >>>> >>>> On Mon, Mar 20, 2023 at 9:41 PM Peter Rushforth < >>>> peter.rushfo...@gmail.com> wrote: >>>> >>>>> Hi >>>>> >>>>> Sorry I didn't copy the list in my last email, my mistake. >>>>> >>>>> Anyway, I looked at the Spring changes and nothing really stands out >>>>> to me, although there is some logging on main that didn't occur in 2.22.x. >>>>> It might still have something to do with that, but because I noticed in >>>>> web.xml that text/.* was being compressed by >>>>> the org.geoserver.filters.GZIPFilter filter, but .js was not, I tried >>>>> removing the compression on text.* and sure enough the delay disappears >>>>> from main / problem is gone for mapml previews. >>>>> >>>>> I don't think it's desirable to not compress html and css etc, so I >>>>> will keep looking, but please let me know if anything occurs to you. >>>>> >>>>> Peter >>>>> >>>>> On Mon, Mar 20, 2023 at 12:24 PM Peter Rushforth < >>>>> peter.rushfo...@gmail.com> wrote: >>>>> >>>>>> A couple of logs with GEOSERVER_DEVELOPER_LOGGING enabled, the first >>>>>> with a (custom) build from the 2.22.x branch. The customization is that I >>>>>> include some geoserver.mapml logging statements. The second is from main >>>>>> today. >>>>>> They compare pretty closely, except for the big pause. One >>>>>> difference I've noticed is the 2.22.x branch's: >>>>>> >>>>>> "[filter.GeoServerSecurityContextPersistenceFilter$1] - >>>>>> SecurityContextHolder now cleared, as request processing completed" >>>>>> >>>>>> log statements don't have any parallel in today's 2.24.SNAPSHOT >>>>>> (main) log, so I'm wondering if there is some resource that is being >>>>>> withheld for many / all the resources. >>>>>> >>>>>> Anyway, will keep looking. >>>>>> >>>>>> Thanks >>>>>> Peter >>>>>> >>>>>> On Mon, Mar 20, 2023 at 11:14 AM Peter Rushforth < >>>>>> peter.rushfo...@gmail.com> wrote: >>>>>> >>>>>>> Hi Andrea, and Jody, >>>>>>> >>>>>>> I am running a build off main today, as it has the same behaviour as >>>>>>> 2.23-RC1. >>>>>>> >>>>>>> I ran a few jstack commands during execution of the tomcat process >>>>>>> id, first at idle (jstack-idle.out), then during a mapml preview request >>>>>>> (jstack-mapml-out to jstack-mapml.out12) and a couple during just the >>>>>>> mapml.css request at the URL bar (jstack-css.out). >>>>>>> >>>>>>> I attach these files, hopefully the list will allow them. They >>>>>>> don't tell me much, but if you can spot something, please let me know. >>>>>>> >>>>>>> I will see if I can profile the app during the same events, for >>>>>>> comparison. >>>>>>> >>>>>>> Thanks for your help! >>>>>>> Peter >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Mar 20, 2023 at 6:48 AM Andrea Aime < >>>>>>> andrea.a...@geosolutionsgroup.com> wrote: >>>>>>> >>>>>>>> Hi Peter, >>>>>>>> can you run a few jstack during the slowdown? Would help us figure >>>>>>>> out what is slowing it down. >>>>>>>> >>>>>>>> Has anyone else noticed slowdowns in other parts of GeoServer yet? >>>>>>>> >>>>>>>> Cheers >>>>>>>> Andrea >>>>>>>> >>>>>>>> On Mon, Mar 20, 2023 at 8:33 AM Jody Garnett < >>>>>>>> jody.garn...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Peter: >>>>>>>>> >>>>>>>>> Thanks for noticing the regression; the spring upgrade is indeed >>>>>>>>> the big internal change this release cycle. Are you in position to do >>>>>>>>> any >>>>>>>>> profiling to determine where the slowdown is occurring? You may be >>>>>>>>> able to >>>>>>>>> use git bisect to determine the exact change that introduced the >>>>>>>>> slowdown. >>>>>>>>> >>>>>>>>> I would like to schedule the 2.23.0 release later this month. >>>>>>>>> -- >>>>>>>>> Jody Garnett >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Mar 17, 2023 at 7:38 AM Peter Rushforth < >>>>>>>>> peter.rushfo...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I am the maintainer of the MapML module. In testing 2.23-RC1, I >>>>>>>>>> noticed a long delay that occurs during processing of pretty simple >>>>>>>>>> requests to the module. I have checked out and built the HEAD of the >>>>>>>>>> 2.22.x branch, and the delay does not occur there. Since the MapML >>>>>>>>>> module >>>>>>>>>> is a Spring REST controller, I was wondering if it might have been >>>>>>>>>> negatively impacted by a spring upgrade? >>>>>>>>>> >>>>>>>>>> Definitely willing to change the code to play nice with the new >>>>>>>>>> spring. >>>>>>>>>> >>>>>>>>>> Any advice on how to test / evaluate this is welcome. >>>>>>>>>> >>>>>>>>>> Thank you >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Geoserver-devel mailing list >>>>>>>>>> Geoserver-devel@lists.sourceforge.net >>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Geoserver-devel mailing list >>>>>>>>> Geoserver-devel@lists.sourceforge.net >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Andrea Aime >>>>>>>> >>>>>>>> == >>>>>>>> GeoServer Professional Services from the experts! >>>>>>>> >>>>>>>> Visit http://bit.ly/gs-services-us for more information. >>>>>>>> == >>>>>>>> >>>>>>>> Ing. Andrea Aime >>>>>>>> @geowolf >>>>>>>> Technical Lead >>>>>>>> >>>>>>>> GeoSolutions Group >>>>>>>> phone: +39 0584 962313 >>>>>>>> >>>>>>>> fax: +39 0584 1660272 >>>>>>>> >>>>>>>> mob: +39 339 8844549 >>>>>>>> >>>>>>>> https://www.geosolutionsgroup.com/ >>>>>>>> >>>>>>>> http://twitter.com/geosolutions_it >>>>>>>> >>>>>>>> ------------------------------------------------------- >>>>>>>> >>>>>>>> Con riferimento alla normativa sul trattamento dei dati personali >>>>>>>> (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati >>>>>>>> “GDPR”), >>>>>>>> si precisa che ogni circostanza inerente alla presente email (il suo >>>>>>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >>>>>>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >>>>>>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >>>>>>>> operazione è illecita. Le sarei comunque grato se potesse darmene >>>>>>>> notizia. >>>>>>>> >>>>>>>> This email is intended only for the person or entity to which it is >>>>>>>> addressed and may contain information that is privileged, confidential >>>>>>>> or >>>>>>>> otherwise protected from disclosure. We remind that - as provided by >>>>>>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of >>>>>>>> this >>>>>>>> e-mail or the information herein by anyone other than the intended >>>>>>>> recipient is prohibited. If you have received this email by mistake, >>>>>>>> please >>>>>>>> notify us immediately by telephone or e-mail >>>>>>>> >>>>>>> >>>> >>>> -- >>>> >>>> Regards, >>>> >>>> Andrea Aime >>>> >>>> == >>>> GeoServer Professional Services from the experts! >>>> >>>> Visit http://bit.ly/gs-services-us for more information. >>>> == >>>> >>>> Ing. Andrea Aime >>>> @geowolf >>>> Technical Lead >>>> >>>> GeoSolutions Group >>>> phone: +39 0584 962313 >>>> >>>> fax: +39 0584 1660272 >>>> >>>> mob: +39 339 8844549 >>>> >>>> https://www.geosolutionsgroup.com/ >>>> >>>> http://twitter.com/geosolutions_it >>>> >>>> ------------------------------------------------------- >>>> >>>> Con riferimento alla normativa sul trattamento dei dati personali (Reg. >>>> UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si >>>> precisa che ogni circostanza inerente alla presente email (il suo >>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >>>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia. >>>> >>>> This email is intended only for the person or entity to which it is >>>> addressed and may contain information that is privileged, confidential or >>>> otherwise protected from disclosure. We remind that - as provided by >>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this >>>> e-mail or the information herein by anyone other than the intended >>>> recipient is prohibited. If you have received this email by mistake, please >>>> notify us immediately by telephone or e-mail >>>> >>> >> >> -- >> >> Regards, >> >> Andrea Aime >> >> == >> GeoServer Professional Services from the experts! >> >> Visit http://bit.ly/gs-services-us for more information. >> == >> >> Ing. Andrea Aime >> @geowolf >> Technical Lead >> >> GeoSolutions Group >> phone: +39 0584 962313 >> >> fax: +39 0584 1660272 >> >> mob: +39 339 8844549 >> >> https://www.geosolutionsgroup.com/ >> >> http://twitter.com/geosolutions_it >> >> ------------------------------------------------------- >> >> Con riferimento alla normativa sul trattamento dei dati personali (Reg. >> UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si >> precisa che ogni circostanza inerente alla presente email (il suo >> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >> operazione è illecita. Le sarei comunque grato se potesse darmene notizia. >> >> This email is intended only for the person or entity to which it is >> addressed and may contain information that is privileged, confidential or >> otherwise protected from disclosure. We remind that - as provided by >> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this >> e-mail or the information herein by anyone other than the intended >> recipient is prohibited. If you have received this email by mistake, please >> notify us immediately by telephone or e-mail >> > -- Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
