Hi Martin, Ok I've opened a ticket. I did some more debugging and I've described it to the best of my understanding. I could try and make a patch but its complex code and I don't want to mess anything up!! - not very confident with Wicket.
https://issues.apache.org/jira/browse/WICKET-6356 thanks On Mon, Apr 3, 2017 at 8:19 AM, Martin Grigorov <[email protected]> wrote: > Hi, > > On Thu, Mar 30, 2017 at 11:21 PM, Wayne W <[email protected]> > wrote: > > > Please ignore my last email I have done a lot more debugging and I > *think* > > I understand why replication does not work for us: > > > > > > Whenever a stateful pages is used (touched) the SessionEntry is updated, > > > i.e. the page is put into it. > > > The SessionEntry is stored as an attribute in the HTTP Session. > > > The web container has logic to detect if the HTTP session has been > > updated > > > in a request and if it was then it replicates the whole session to the > > > other nodes. > > > Once the session is replicated it is deserialized. Here > > > SessionEntry#readObject() comes to play. It reads the pages from the > > > SessionEntry and stores them into the disk (via IDiskStore). > > > So the stateful pages should be available in the disks of all nodes. > > > Stateless pages are always recreated from scratch, so they are not > > > replicated. > > > > > > I think this is a little wrong. Yes the SessionEntry in stored as an > > attribute in the http session and is replicated. However the pages in the > > SessionEntry cache are not replicated as they are transient. > > > > private transient List<IManageablePage> sessionCache; > > > > This looks like a bug! > It shouldn't be transient. > Please open a ticket! > Could you please provide your Tomcat configuration related to replication > too ? > A patch / Pull Request is also very welcome! > > Thank you! > > > > > > private transient List<Object> afterReadObject > > > > Therefore the pages are *not* deserialised as you say. The only time > > SessionEntry#readObject() > > comes to play (at least with Tomcat) is when a node is first brought up > and > > the daltamanager replicates ALL sessions across, at this point all the > > current SessionEntry's in the other nodes http sessions are written > across. > > After this is done SessionEntry#readObject() is not called anymore. Any > new > > pages/updates on the original instance are not replicated across (only > the > > sessionId in the SessionEntry). > > > > This means any new pages created on the old instance (after the new > > instance has started up) are not available in the http session or the > > second level page store on the new instance. > > Therefore when the old instance in shut down the user is load balanced to > > the new instance. At this point the link in the page Wicket is looking > for > > does not exist in the SessionEntry cache or the PageStore so it creates > the > > page and looks for the component/link. > > > > This causes a ComponentNotFoundException for us because the links are > > either in a DataView which is never rendered so does not exist, or the > > other links are actually added to the page in an Ajax request and again > > because the page is not rendered are not there, Wicket then throws the > > exception and it appears to the user the session is lost. > > > > This is at least what I am observing. I can provide a Quick start to > > demonstrate if needed. > > > > many thanks for you time thus far. > > > > > > On Thu, Mar 30, 2017 at 4:29 PM, Wayne W <[email protected]> > > wrote: > > > > > > > > > > > On Thu, Mar 30, 2017 at 4:09 PM, Martin Grigorov <[email protected] > > > > > wrote: > > > > > >> > > >> > > >> Whenever a stateful pages is used (touched) the SessionEntry is > updated, > > >> i.e. the page is put into it. > > >> > > > > > > Yes I can see that happening in PageStoreManager.storeTouchedPages() > > > > > > > > >> The SessionEntry is stored as an attribute in the HTTP Session. > > >> > > > > > > I'm not seeing this. In the wicket.Session class there is line : > > > > > > private transient ISessionStore sessionStore; > > > > > > Which is loaded from the Application each request. The SessionStore > > > contains the SessionEntry's no??? > > > > > > > > > > > >> The web container has logic to detect if the HTTP session has been > > updated > > >> in a request and if it was then it replicates the whole session to the > > >> other nodes. > > >> > > > > > > Yes I'm seeing the http session replicated no issues. > > > > > > > > >> Once the session is replicated it is deserialized. Here > > >> SessionEntry#readObject() comes to play. It reads the pages from the > > >> SessionEntry and stores them into the disk (via IDiskStore). > > >> So the stateful pages should be available in the disks of all nodes. > > >> Stateless pages are always recreated from scratch, so they are not > > >> replicated. > > >> > > > > > > > > > If this is the case why is wicket saying this page is stateless as the > > > SessionEntry is never replicated across?: > > > > > > > > > > > > public class HomePage extends WebPage { > > > > > > private static final long serialVersionUID = 1L; > > > > > > > > > public HomePage() { > > > > > > super(); > > > > > > > > > Session s= Session.get(); > > > > > > s.bind(); > > > > > > Integer i = (Integer)s.getAttribute("foo"); > > > > > > if (i == null) { > > > > > > i = new Integer(0); > > > > > > } > > > > > > i++; > > > > > > s.setAttribute("foo",i); > > > > > > add(new Label("version", (Integer)s.getAttribute("foo"))); > > > > > > > > > add(new Link<Void>("link1") { > > > > > > @Override > > > > > > public void onClick() > > > > > > { > > > > > > setResponsePage(new NextPage()); > > > > > > } > > > > > > }); > > > > > > add(new Link<Void>("link2") { > > > > > > @Override > > > > > > public void onClick() > > > > > > { > > > > > > setResponsePage(new HomePage()); > > > > > > } > > > > > > }); > > > > > > add(new AbstractDefaultAjaxBehavior() { > > > > > > @Override > > > > > > protected void respond(AjaxRequestTarget target) { > > > > > > getSession().setAttribute("someKey", UUID.randomUUID().toString()); > > > > > > System.out.println("SLEEEPING"); > > > > > > try { > > > > > > Thread.sleep(3000); > > > > > > }catch(Exception e) { > > > > > > } > > > > > > System.out.println("DONE"); > > > > > > } > > > > > > @Override > > > > > > public void renderHead(Component component, IHeaderResponse response) { > > > > > > super.renderHead(component, response); > > > > > > String js = "(" + getCallbackFunction().toString() +")();"; > > > > > > response.render(OnDomReadyHeaderItem.forScript(js)); > > > > > > } > > > > > > }); > > > > > > // TODO Add your page's components here > > > > > > > > > } > > > > > > } > > > > > > > > > > > > Sorry for all these questions but I must get to the bottom of this (5 > > days > > > work so far). Many thanks > > > > > > > > > > > >> > > >> > > >> > > > >> > > > >> > thanks > > >> > > > >> > On Tue, Mar 28, 2017 at 9:39 PM, Martin Grigorov < > > [email protected]> > > >> > wrote: > > >> > > > >> > > Hi, > > >> > > > > >> > > Did you mention already which version of Wicket you use ? > > >> > > There was an improvement related to reusing the page parameters > when > > >> > > constructing a new page instance in 7.0.0. > > >> > > I have the feeling you are using 6.x. Am I correct ? > > >> > > > > >> > > Martin Grigorov > > >> > > Wicket Training and Consulting > > >> > > https://twitter.com/mtgrigorov > > >> > > > > >> > > On Tue, Mar 28, 2017 at 7:22 PM, Wayne W < > > [email protected] > > >> > > > >> > > wrote: > > >> > > > > >> > > > Hello Martin, > > >> > > > > > >> > > > so I've been trying to still get to the bottom of this days > later. > > >> So > > >> > my > > >> > > > understanding is getting better and it appears that the session > > >> itself > > >> > is > > >> > > > replicated fine. However I've tracked my issue down to the > > >> following: > > >> > > > > > >> > > > - If I have a page that has 2 links at the top. Link A is a > > >> > Bookmarkable > > >> > > > link back to the same page - it is also mounted thus: > mountPage( > > >> > > > "/cube/documents/${0}", CubeDocumentPage.class); It has a Long > > page > > >> > param > > >> > > > needed to construct the page ( i.e. /cube/documents/12345 ). The > > >> second > > >> > > > link B is just a simple new Link<Void>("B") . > > >> > > > - If both instances are up and running , lets say I am on > instance > > >> A. I > > >> > > > visit link A 3 times. I then kill instance A and I am balanced > > over > > >> to > > >> > > > instance B. If I visit link B , the PageStoreManager cannot > find > > >> the > > >> > > page > > >> > > > that contains link B in the store and then tries to create a new > > >> > instance > > >> > > > of the page - the problem is then for some reason the page > > >> parameters > > >> > are > > >> > > > always null and the Long is never passed. Why is the page > > parameter > > >> > > always > > >> > > > null here? Trying to debug it, is seems the IPageManager is got > > from > > >> > the > > >> > > > wicket Application instance and not the Session, and this > > >> IPageManager > > >> > > > looks for a RequestAdaptor. Somewhere the page parameters are > > lost?. > > >> > > > > > >> > > > However I can get it to work this way: > > >> > > > - start instance A, visit the page and say click link A 3 times. > > >> > > > - Now start up instance B > > >> > > > - Kill instance A > > >> > > > - Click on the link B and this time the PageStoreManager finds > the > > >> page > > >> > > and > > >> > > > there the link B and everything works fine. > > >> > > > It only works if I don't visit another page just after instance > B > > >> > starts > > >> > > > up. > > >> > > > > > >> > > > > > >> > > > I will do some more debugging tomorrow to try and understand > this > > >> > > > PageStoreManager/request adaptor bit more. But if you have any > > ideas > > >> > I'd > > >> > > > appreciate it. > > >> > > > Thanks > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Fri, Mar 24, 2017 at 12:48 PM, Martin Grigorov < > > >> > [email protected]> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > On Fri, Mar 24, 2017 at 1:31 PM, Wayne W < > > >> > [email protected]> > > >> > > > > wrote: > > >> > > > > > > >> > > > > > Thanks Martin, > > >> > > > > > > > >> > > > > > I have a theory what this is, perhaps you could confirm? > > >> > > > > > > > >> > > > > > What I observe is the following with the replication: If I > > visit > > >> > > page A > > >> > > > > and > > >> > > > > > then visit page B, then kill the instance I am on the > session > > is > > >> > > > > > successfully failed over to the other node. Now I'm still > > >> looking > > >> > at > > >> > > > > page B > > >> > > > > > in the browser - if I hit back on the browser I get > > >> > > > PageExpiredException. > > >> > > > > > Is this the expected behaviour? > > >> > > > > > > > >> > > > > > > >> > > > > No. > > >> > > > > The expected behavior is to see the latest state of page A. > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > If it IS the expected behaviour, then the reason I think > that > > I > > >> > have > > >> > > an > > >> > > > > > issue with the page with the AbstractDefaultAjaxBehavior is > > >> because > > >> > > the > > >> > > > > > code in the AbstractDefaultAjaxBehaviour.respond is adding > > >> > > components > > >> > > > to > > >> > > > > > the page which in turn marks the Session dirty which in turn > > >> > > replicates > > >> > > > > the > > >> > > > > > session. If only the last page/session commit is only > > >> replicated it > > >> > > > could > > >> > > > > > explain why I get a ComponentNotFound exception from the > page > > as > > >> > > > > > it overwriten by the update from the > > AbstractDefaultAjaxBehavior > > >> > > > > > > > >> > > > > > > >> > > > > Any time the page is marked as dirty Wicket will store it in > 1) > > >> the > > >> > > HTTP > > >> > > > > session and 2) on the disk > > >> > > > > 1) will notify the web server (e.g. Tomcat) that it has to > > >> replicate > > >> > > the > > >> > > > > session because some attribute has been modified > > >> > > > > At node2 Wicket will detect the replicated session in > > >> #readObject() > > >> > and > > >> > > > > store it on the disk. > > >> > > > > From now on node2 will have the same history for that page as > > >> node1, > > >> > > i.e. > > >> > > > > PageA and PageB. > > >> > > > > > > >> > > > > In addition: > > >> > > > > Updates made in an Ajax requests do not add a new entry in the > > >> > history > > >> > > > > (storages) but overrides the previous entry for that page > > >> instance! > > >> > > > > I.e. when PageA is rendered Wicket will assign a pageId for > it, > > >> e.g. > > >> > 5, > > >> > > > and > > >> > > > > store it in the disk. > > >> > > > > Later when you make a change to PageA:5 in Ajax request then > > >> Wicket > > >> > > will > > >> > > > > override the entry on the disk. > > >> > > > > If you make a change in non-Ajax request then Wicket will > > assign a > > >> > new > > >> > > > > pageId, e.g. 6, and add a new entry in the history (disk), so > > you > > >> > will > > >> > > > have > > >> > > > > two entries for this session. > > >> > > > > Later when you open PageB then this will be pageId 7, etc. > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > What do you think? > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > On Fri, Mar 24, 2017 at 9:31 AM, Martin Grigorov < > > >> > > [email protected] > > >> > > > > > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > Hi, > > >> > > > > > > > > >> > > > > > > Once the http session is replicated this method should be > > >> > executed > > >> > > on > > >> > > > > the > > >> > > > > > > node that did not process the request > > >> > > > > > > org.apache.wicket.page.PageStoreManager.SessionEntry# > > >> > readObject(). > > >> > > > > > > Here Wicket will either store the page(s) on the disk or > > will > > >> > > > schedule > > >> > > > > > them > > >> > > > > > > for storing. > > >> > > > > > > There is no much usage of Loggers here so you will have to > > >> > attach a > > >> > > > > > > debugger and see what happens. > > >> > > > > > > > > >> > > > > > > Martin Grigorov > > >> > > > > > > Wicket Training and Consulting > > >> > > > > > > https://twitter.com/mtgrigorov > > >> > > > > > > > > >> > > > > > > On Thu, Mar 23, 2017 at 6:11 PM, Wayne W < > > >> > > > [email protected]> > > >> > > > > > > wrote: > > >> > > > > > > > > >> > > > > > > > If I put the AbstractDefaultAjaxBehavior back in, it > > always > > >> > > fails. > > >> > > > > If I > > >> > > > > > > > remove the AbstractDefaultAjaxBehavior most of the time > it > > >> > works, > > >> > > > but > > >> > > > > > > every > > >> > > > > > > > now and then it fails to replicate. Its really > > inconsistent. > > >> > > > > > > > > > >> > > > > > > > I have no idea how to get to the bottom of this. Any > > >> pointers > > >> > or > > >> > > > help > > >> > > > > > is > > >> > > > > > > > much appreciated. > > >> > > > > > > > > > >> > > > > > > > On Thu, Mar 23, 2017 at 4:53 PM, Wayne W < > > >> > > > > [email protected]> > > >> > > > > > > > wrote: > > >> > > > > > > > > > >> > > > > > > > > So it seems the homepage is not the only page. However > > >> > removing > > >> > > > the > > >> > > > > > > > > AbstractDefaultAjaxBehavior makes no difference I > still > > >> get > > >> > the > > >> > > > > same > > >> > > > > > > > > issues. Just don;t know where to start with this one. > Is > > >> > there > > >> > > a > > >> > > > > way > > >> > > > > > I > > >> > > > > > > > can > > >> > > > > > > > > turn on the logging around replication in wicket? > > >> > > > > > > > > > > >> > > > > > > > > On Thu, Mar 23, 2017 at 4:27 PM, Wayne W < > > >> > > > > > [email protected]> > > >> > > > > > > > > wrote: > > >> > > > > > > > > > > >> > > > > > > > >> Hi Martin, > > >> > > > > > > > >> > > >> > > > > > > > >> no - I still get the ComponentNotFoundException in > the > > >> new > > >> > > > > instance. > > >> > > > > > > :-/ > > >> > > > > > > > >> > > >> > > > > > > > >> On Thu, Mar 23, 2017 at 3:57 PM, Martin Grigorov < > > >> > > > > > > [email protected]> > > >> > > > > > > > >> wrote: > > >> > > > > > > > >> > > >> > > > > > > > >>> Hi, > > >> > > > > > > > >>> > > >> > > > > > > > >>> On Thu, Mar 23, 2017 at 4:20 PM, Wayne W < > > >> > > > > > > [email protected]> > > >> > > > > > > > >>> wrote: > > >> > > > > > > > >>> > > >> > > > > > > > >>> > Hi Martin, > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > that was a typo on my part. I've been doing a lot > > more > > >> > > > testing > > >> > > > > > > > before I > > >> > > > > > > > >>> > replied to make sure things are as I say. This is > > >> what I > > >> > am > > >> > > > > > > > observing: > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > - I have a 2 node tomcat cluster setup with > > apache > > >> > > > balancing > > >> > > > > > > > between > > >> > > > > > > > >>> > them locally on my machine. > > >> > > > > > > > >>> > - Testing with a very simple Wicket app I can > see > > >> the > > >> > > > > > > replication > > >> > > > > > > > is > > >> > > > > > > > >>> > working fine when shutting down either > instance. > > >> > > > > > > > >>> > - Testing with our very heavy weight wicket app > > it > > >> > > > sometimes > > >> > > > > > > work > > >> > > > > > > > >>> and > > >> > > > > > > > >>> > sometimes does not. > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > Now the last point above I have narrowed down to > our > > >> > > > homepage. > > >> > > > > > If I > > >> > > > > > > > am > > >> > > > > > > > >>> not > > >> > > > > > > > >>> > on the homepage the replication seems to work ok. > > >> However > > >> > > if > > >> > > > I > > >> > > > > am > > >> > > > > > > on > > >> > > > > > > > >>> the > > >> > > > > > > > >>> > homepage I always get a ComponentNotFoundException > > >> when > > >> > > > failing > > >> > > > > > > over > > >> > > > > > > > >>> to the > > >> > > > > > > > >>> > other instance. > > >> > > > > > > > >>> > The homepage has an AbstractDefaultAjaxBehavior > > which > > >> is > > >> > > > called > > >> > > > > > > once > > >> > > > > > > > >>> the > > >> > > > > > > > >>> > page is rendered, this in turn adds to the the > page > > a > > >> > > > DataView > > >> > > > > > > > >>> containing a > > >> > > > > > > > >>> > lot of content and links. The user needs to be > > logged > > >> in > > >> > > > before > > >> > > > > > > they > > >> > > > > > > > >>> see > > >> > > > > > > > >>> > the homepage, so the session is already setup. > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > I don't know what it is about this page that > breaks > > >> the > > >> > > > > > > replication. > > >> > > > > > > > Is > > >> > > > > > > > >>> > there anything around the ajax part that could do > > >> this? > > >> > Any > > >> > > > > > > pointers > > >> > > > > > > > >>> would > > >> > > > > > > > >>> > be most welcome as it will be a long long task to > > >> break > > >> > > that > > >> > > > > page > > >> > > > > > > > down > > >> > > > > > > > >>> and > > >> > > > > > > > >>> > do rounds and rounds of cluster testing > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > >> > > > > > > > >>> Let's try something! > > >> > > > > > > > >>> In you Ajax callback method do something like: > > >> > > > > > > > >>> getSession().setAttribute("someKey", > > >> > > > > UUID.randomUUID().toString()) > > >> > > > > > > > >>> Does this trigger replication ? > > >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > many thanks > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > On Wed, Mar 22, 2017 at 12:34 PM, Martin Grigorov > < > > >> > > > > > > > >>> [email protected]> > > >> > > > > > > > >>> > wrote: > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > Hi, > > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > "I can stop one of the instances and I'm not > > logged > > >> in" > > >> > > > > > > > >>> > > This statement says that you don't really have a > > >> > properly > > >> > > > > > > > configured > > >> > > > > > > > >>> > > failover. > > >> > > > > > > > >>> > > If those two Tomcat instances are in a cluster > > then > > >> the > > >> > > > http > > >> > > > > > > > sessions > > >> > > > > > > > >>> > > should be replicated and you should stay logged > in > > >> no > > >> > > > matter > > >> > > > > > > which > > >> > > > > > > > >>> one is > > >> > > > > > > > >>> > > serving the request. > > >> > > > > > > > >>> > > Wicket will store the used stateful page at the > > >> disks > > >> > for > > >> > > > all > > >> > > > > > > > >>> Tomcats in > > >> > > > > > > > >>> > > the cluster, if it the replication is actually > > >> working! > > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > On Wed, Mar 22, 2017 at 1:02 PM, Wayne W < > > >> > > > > > > > >>> [email protected]> > > >> > > > > > > > >>> > > wrote: > > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > Hi, > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > We have 2 instances of Tomcat running with > > Apache > > >> > > sitting > > >> > > > > in > > >> > > > > > > > front > > >> > > > > > > > >>> > > > balancing between the Tomcat instances. I have > > >> > session > > >> > > > > > > > replication > > >> > > > > > > > >>> > setup > > >> > > > > > > > >>> > > > which seems to work for basic bookmarkable > links > > >> on > > >> > the > > >> > > > > > pages. > > >> > > > > > > I > > >> > > > > > > > >>> can > > >> > > > > > > > >>> > stop > > >> > > > > > > > >>> > > > one of the instances and I'm not logged in as > it > > >> > > failover > > >> > > > > to > > >> > > > > > > the > > >> > > > > > > > >>> other > > >> > > > > > > > >>> > > > instance. > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > However for normal Link<?> and ajax links etc > I > > >> get > > >> > > > > > > > >>> > > > ComponentNotFoundException thrown as the new > > >> > instance > > >> > > > > cannot > > >> > > > > > > > find > > >> > > > > > > > >>> it > > >> > > > > > > > >>> > in > > >> > > > > > > > >>> > > > the session it seems. > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > Clearly I'm not understanding how wicket > manages > > >> the > > >> > > page > > >> > > > > > state > > >> > > > > > > > or > > >> > > > > > > > >>> I've > > >> > > > > > > > >>> > > > configured something wrong. > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > Does wicket support full session failover ? > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > Many thanks > > >> > > > > > > > >>> > > > > > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
