Hi Thomas, it is indeed WICKET-6963. The reason from my understanding is the way we "treat" pages & components here. I tried to make a small wicket quickstart to show the problem but I wasn't able to reproduce the problem. I then tried to reproduce it with our cms base (https://github.com/brix-cms/brix-cms) and then it happens on rare occasions (even DEMO works... one just must hammer it with jmeter or sth like that - much concurrence needed).
See, we use brix cms and this has custom tags tiles etc. as well as a custom page that renders the content: https://github.com/brix-cms/brix-cms/blob/master/brix-core/src/main/java/org/brixcms/web/nodepage/BrixNodeWebPage.java https://github.com/brix-cms/brix-cms/blob/master/brix-core/src/main/java/org/brixcms/markup/web/BrixMarkupNodeWebPage.java What brix does is, basically spoken, it takes a node from a JCR and understands this as a kind of receipe to create a wicket page, decorate it with tiles (special kind of wicket-panels in brix terms), header items etc. Thus rendering and rerendering pages often... very often (contrary to regular wicket apps... these usually only render once as HTML-sources stays fixed - see https://github.com/brix-cms/brix-cms/wiki/Templating ). When now many concurrenct requests come in and at the same time it rerenders something then the final static with the "wrong" flag gets leaked into the wrong rendering and therefore creates this 503 errors. AFAIK a "static final" even allows java to reuse this item in any other threads without any limitations as only the reference is thread safe but not the contents - am I right here? Anyway, I can asure you that I made a nearly similar mistake by myself not long ago when I thought I optimize some fields to final static and only production errors let me know :X Do you know when a 9.9.1 will be out? Best, KB ----- Ursprüngliche Mail ----- > Von: "Thomas Heigl" > An: "dev" <dev@wicket.apache.org> > Gesendet: Donnerstag, 31. März 2022 11:35:45 > Betreff: Re: wicket 9.9.0 - MarkupExceptions after upgrade - any idea why I > get this? > It's just a single commit. I have reverted it for both 9.x and master. > > Best, > > Thomas > > On Thu, Mar 31, 2022 at 11:06 AM Andrea Del Bene > wrote: > >> On Thu, Mar 31, 2022 at 10:33 AM Thomas Heigl wrote: >> >> > Ok I had a closer look at AssociatedMarkupSourcingStrategy and the code >> > related to this flag is quite convoluted and full of TODOs and WHY >> > comments. I think I will have to do another take on this and replace the >> > field with a parameter, but this will take some time. >> > >> > I would suggest we revert WICKET-6963 for now and release 9.9.1. >> > >> > Sorry for that! I was confident that my change was good, because it >> worked >> > without problems in our production environment. >> > >> > >> Don't worry, these things happened :-) . Could you take care of reverting >> WICKET-6963 on git? Or if it's just a commit I can do it by myself. >> >> >> > Thomas >> > >> > On Thu, Mar 31, 2022 at 10:18 AM Thomas Heigl >> wrote: >> > >> > > Although the JavaDoc states that this should be possible: >> > > >> > > // reset for each render in case the strategy is re-used >> > > noMoreWicketHeadTagsAllowed = false; >> > > >> > > It would be great if we had a failing test-case for this. In my >> > production >> > > environment with about 5000 panels, there are none of these issues. >> > > >> > > Thomas >> > > >> > > On Thu, Mar 31, 2022 at 10:14 AM Thomas Heigl >> > wrote: >> > > >> > >> I think we will have to revert WICKET-6963. >> > >> >> > >> I somehow overlooked the non-final field noMoreWicketHeadTagsAllowed >> > >> in AssociatedMarkupSourcingStrategy. If this flag gets set, the >> > singleton >> > >> strategy doesn't work. >> > >> >> > >> Thomas >> > >> >> > >> On Thu, Mar 31, 2022 at 10:11 AM Thomas Heigl >> > >> wrote: >> > >> >> > >>> This is probably caused by my change in >> > >>> https://issues.apache.org/jira/browse/WICKET-6963 >> > >>> >> > >>> I have been using this code in production for a couple of weeks now >> > >>> without issues, but there seem to be cases where the singleton >> strategy >> > >>> doesn't work. >> > >>> >> > >>> Can you reproduce this issue in a test-case? >> > >>> >> > >>> Thomas >> > >>> >> > >>> On Thu, Mar 31, 2022 at 10:06 AM Maxim Solodovnik < >> > solomax...@gmail.com> >> > >>> wrote: >> > >>> >> > >>>> "<wicket:head> tags are only allowed before <body>, , >> > >>>> <wicket:panel> >> > >>>> etc. tag" >> > >>>> sounds reasonable :) >> > >>>> >> > >>>> On Thu, 31 Mar 2022 at 14:56, Korbinian Bachl < >> > >>>> korbinian.ba...@whiskyworld.de> wrote: >> > >>>> >> > >>>> > Hi, >> > >>>> > >> > >>>> > I deployed our app on 9.9.0 this morning and after initializing a >> > >>>> crawl of >> > >>>> > the page I ended up getting a low quote of 503s. >> > >>>> > >> > >>>> > The 503s are always the same: >> > >>>> > 2022-03-31 09:35:05,031 ERROR >> > >>>> [org.apache.wicket.DefaultExceptionMapper] >> > >>>> > Unexpected error occurred >> > >>>> > org.apache.wicket.markup.MarkupException: <wicket:head> tags are >> > only >> > >>>> > allowed before <body>, , <wicket:panel> etc. tag >> > >>>> > at >> > >>>> > >> > >>>> >> > >> org.apache.wicket.markup.html.panel.AssociatedMarkupSourcingStrategy.nextHeaderMarkup(AssociatedMarkupSourcingStrategy.java:341) >> > >>>> > ~[wicket-core-9.9.0.jar:9.9.0] >> > >>>> > at >> > >>>> > >> > >>>> >> > >> org.apache.wicket.markup.html.panel.AssociatedMarkupSourcingStrategy.renderHeadFromAssociatedMarkupFile(AssociatedMarkupSourcingStrategy.java:236) >> > >>>> > ~[wicket-core-9.9.0.jar:9.9.0] >> > >>>> > at >> > >>>> > >> > >>>> >> > >> org.apache.wicket.markup.html.panel.AssociatedMarkupSourcingStrategy.renderHead(AssociatedMarkupSourcingStrategy.java:204) >> > >>>> > ~[wicket-core-9.9.0.jar:9.9.0] >> > >>>> > at >> > >>>> > >> org.apache.wicket.Component.internalRenderHead(Component.java:2649) >> > >>>> > ~[wicket-core-9.9.0.jar:9.9.0] >> > >>>> > .... >> > >>>> > at >> > >>>> > >> > >>>> >> > >> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569) >> > >>>> > [nucleus-grizzly-all.jar:?] at >> > >>>> > >> > >>>> >> > >> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549) >> > >>>> > [nucleus-grizzly-all.jar:?] at >> java.lang.Thread.run(Thread.java:829) >> > >>>> [?:?] >> > >>>> > >> > >>>> > Any idea how to debug this or where it may come from (race >> condition >> > >>>> in >> > >>>> > our code as wicket became faster?)? >> > >>>> > It somehow seems to be a happen when requests coming in at the >> same >> > >>>> > time.... with 9.8.0 we got no such errors. >> > >>>> > >> > >>>> > Best, >> > >>>> > >> > >>>> > KB >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> >> > >>>> -- >> > >>>> Best regards, >> > >>>> Maxim >> > >>>> >> > >>> >> > >> >> >> -- >> Andrea Del Bene. >> Apache Wicket committer.