Please ignore my last email I have done a lot more debugging and I *think* I understand why replication does not work for us:
Whenever a stateful pages is used (touched) the SessionEntry is updated, > i.e. the page is put into it. > The SessionEntry is stored as an attribute in the HTTP Session. > The web container has logic to detect if the HTTP session has been updated > in a request and if it was then it replicates the whole session to the > other nodes. > Once the session is replicated it is deserialized. Here > SessionEntry#readObject() comes to play. It reads the pages from the > SessionEntry and stores them into the disk (via IDiskStore). > So the stateful pages should be available in the disks of all nodes. > Stateless pages are always recreated from scratch, so they are not > replicated. I think this is a little wrong. Yes the SessionEntry in stored as an attribute in the http session and is replicated. However the pages in the SessionEntry cache are not replicated as they are transient. private transient List<IManageablePage> sessionCache; private transient List<Object> afterReadObject Therefore the pages are *not* deserialised as you say. The only time SessionEntry#readObject() comes to play (at least with Tomcat) is when a node is first brought up and the daltamanager replicates ALL sessions across, at this point all the current SessionEntry's in the other nodes http sessions are written across. After this is done SessionEntry#readObject() is not called anymore. Any new pages/updates on the original instance are not replicated across (only the sessionId in the SessionEntry). This means any new pages created on the old instance (after the new instance has started up) are not available in the http session or the second level page store on the new instance. Therefore when the old instance in shut down the user is load balanced to the new instance. At this point the link in the page Wicket is looking for does not exist in the SessionEntry cache or the PageStore so it creates the page and looks for the component/link. This causes a ComponentNotFoundException for us because the links are either in a DataView which is never rendered so does not exist, or the other links are actually added to the page in an Ajax request and again because the page is not rendered are not there, Wicket then throws the exception and it appears to the user the session is lost. This is at least what I am observing. I can provide a Quick start to demonstrate if needed. many thanks for you time thus far. On Thu, Mar 30, 2017 at 4:29 PM, Wayne W <[email protected]> wrote: > > > On Thu, Mar 30, 2017 at 4:09 PM, Martin Grigorov <[email protected]> > wrote: > >> >> >> Whenever a stateful pages is used (touched) the SessionEntry is updated, >> i.e. the page is put into it. >> > > Yes I can see that happening in PageStoreManager.storeTouchedPages() > > >> The SessionEntry is stored as an attribute in the HTTP Session. >> > > I'm not seeing this. In the wicket.Session class there is line : > > private transient ISessionStore sessionStore; > > Which is loaded from the Application each request. The SessionStore > contains the SessionEntry's no??? > > > >> The web container has logic to detect if the HTTP session has been updated >> in a request and if it was then it replicates the whole session to the >> other nodes. >> > > Yes I'm seeing the http session replicated no issues. > > >> Once the session is replicated it is deserialized. Here >> SessionEntry#readObject() comes to play. It reads the pages from the >> SessionEntry and stores them into the disk (via IDiskStore). >> So the stateful pages should be available in the disks of all nodes. >> Stateless pages are always recreated from scratch, so they are not >> replicated. >> > > > If this is the case why is wicket saying this page is stateless as the > SessionEntry is never replicated across?: > > > > public class HomePage extends WebPage { > > private static final long serialVersionUID = 1L; > > > public HomePage() { > > super(); > > > Session s= Session.get(); > > s.bind(); > > Integer i = (Integer)s.getAttribute("foo"); > > if (i == null) { > > i = new Integer(0); > > } > > i++; > > s.setAttribute("foo",i); > > add(new Label("version", (Integer)s.getAttribute("foo"))); > > > add(new Link<Void>("link1") { > > @Override > > public void onClick() > > { > > setResponsePage(new NextPage()); > > } > > }); > > add(new Link<Void>("link2") { > > @Override > > public void onClick() > > { > > setResponsePage(new HomePage()); > > } > > }); > > add(new AbstractDefaultAjaxBehavior() { > > @Override > > protected void respond(AjaxRequestTarget target) { > > getSession().setAttribute("someKey", UUID.randomUUID().toString()); > > System.out.println("SLEEEPING"); > > try { > > Thread.sleep(3000); > > }catch(Exception e) { > > } > > System.out.println("DONE"); > > } > > @Override > > public void renderHead(Component component, IHeaderResponse response) { > > super.renderHead(component, response); > > String js = "(" + getCallbackFunction().toString() +")();"; > > response.render(OnDomReadyHeaderItem.forScript(js)); > > } > > }); > > // TODO Add your page's components here > > > } > > } > > > > Sorry for all these questions but I must get to the bottom of this (5 days > work so far). Many thanks > > > >> >> >> > >> > >> > thanks >> > >> > On Tue, Mar 28, 2017 at 9:39 PM, Martin Grigorov <[email protected]> >> > wrote: >> > >> > > Hi, >> > > >> > > Did you mention already which version of Wicket you use ? >> > > There was an improvement related to reusing the page parameters when >> > > constructing a new page instance in 7.0.0. >> > > I have the feeling you are using 6.x. Am I correct ? >> > > >> > > Martin Grigorov >> > > Wicket Training and Consulting >> > > https://twitter.com/mtgrigorov >> > > >> > > On Tue, Mar 28, 2017 at 7:22 PM, Wayne W <[email protected] >> > >> > > wrote: >> > > >> > > > Hello Martin, >> > > > >> > > > so I've been trying to still get to the bottom of this days later. >> So >> > my >> > > > understanding is getting better and it appears that the session >> itself >> > is >> > > > replicated fine. However I've tracked my issue down to the >> following: >> > > > >> > > > - If I have a page that has 2 links at the top. Link A is a >> > Bookmarkable >> > > > link back to the same page - it is also mounted thus: mountPage( >> > > > "/cube/documents/${0}", CubeDocumentPage.class); It has a Long page >> > param >> > > > needed to construct the page ( i.e. /cube/documents/12345 ). The >> second >> > > > link B is just a simple new Link<Void>("B") . >> > > > - If both instances are up and running , lets say I am on instance >> A. I >> > > > visit link A 3 times. I then kill instance A and I am balanced over >> to >> > > > instance B. If I visit link B , the PageStoreManager cannot find >> the >> > > page >> > > > that contains link B in the store and then tries to create a new >> > instance >> > > > of the page - the problem is then for some reason the page >> parameters >> > are >> > > > always null and the Long is never passed. Why is the page parameter >> > > always >> > > > null here? Trying to debug it, is seems the IPageManager is got from >> > the >> > > > wicket Application instance and not the Session, and this >> IPageManager >> > > > looks for a RequestAdaptor. Somewhere the page parameters are lost?. >> > > > >> > > > However I can get it to work this way: >> > > > - start instance A, visit the page and say click link A 3 times. >> > > > - Now start up instance B >> > > > - Kill instance A >> > > > - Click on the link B and this time the PageStoreManager finds the >> page >> > > and >> > > > there the link B and everything works fine. >> > > > It only works if I don't visit another page just after instance B >> > starts >> > > > up. >> > > > >> > > > >> > > > I will do some more debugging tomorrow to try and understand this >> > > > PageStoreManager/request adaptor bit more. But if you have any ideas >> > I'd >> > > > appreciate it. >> > > > Thanks >> > > > >> > > > >> > > > >> > > > On Fri, Mar 24, 2017 at 12:48 PM, Martin Grigorov < >> > [email protected]> >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > On Fri, Mar 24, 2017 at 1:31 PM, Wayne W < >> > [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Thanks Martin, >> > > > > > >> > > > > > I have a theory what this is, perhaps you could confirm? >> > > > > > >> > > > > > What I observe is the following with the replication: If I visit >> > > page A >> > > > > and >> > > > > > then visit page B, then kill the instance I am on the session is >> > > > > > successfully failed over to the other node. Now I'm still >> looking >> > at >> > > > > page B >> > > > > > in the browser - if I hit back on the browser I get >> > > > PageExpiredException. >> > > > > > Is this the expected behaviour? >> > > > > > >> > > > > >> > > > > No. >> > > > > The expected behavior is to see the latest state of page A. >> > > > > >> > > > > >> > > > > > >> > > > > > If it IS the expected behaviour, then the reason I think that I >> > have >> > > an >> > > > > > issue with the page with the AbstractDefaultAjaxBehavior is >> because >> > > the >> > > > > > code in the AbstractDefaultAjaxBehaviour.respond is adding >> > > components >> > > > to >> > > > > > the page which in turn marks the Session dirty which in turn >> > > replicates >> > > > > the >> > > > > > session. If only the last page/session commit is only >> replicated it >> > > > could >> > > > > > explain why I get a ComponentNotFound exception from the page as >> > > > > > it overwriten by the update from the AbstractDefaultAjaxBehavior >> > > > > > >> > > > > >> > > > > Any time the page is marked as dirty Wicket will store it in 1) >> the >> > > HTTP >> > > > > session and 2) on the disk >> > > > > 1) will notify the web server (e.g. Tomcat) that it has to >> replicate >> > > the >> > > > > session because some attribute has been modified >> > > > > At node2 Wicket will detect the replicated session in >> #readObject() >> > and >> > > > > store it on the disk. >> > > > > From now on node2 will have the same history for that page as >> node1, >> > > i.e. >> > > > > PageA and PageB. >> > > > > >> > > > > In addition: >> > > > > Updates made in an Ajax requests do not add a new entry in the >> > history >> > > > > (storages) but overrides the previous entry for that page >> instance! >> > > > > I.e. when PageA is rendered Wicket will assign a pageId for it, >> e.g. >> > 5, >> > > > and >> > > > > store it in the disk. >> > > > > Later when you make a change to PageA:5 in Ajax request then >> Wicket >> > > will >> > > > > override the entry on the disk. >> > > > > If you make a change in non-Ajax request then Wicket will assign a >> > new >> > > > > pageId, e.g. 6, and add a new entry in the history (disk), so you >> > will >> > > > have >> > > > > two entries for this session. >> > > > > Later when you open PageB then this will be pageId 7, etc. >> > > > > >> > > > > >> > > > > > >> > > > > > What do you think? >> > > > > > >> > > > > > >> > > > > > >> > > > > > On Fri, Mar 24, 2017 at 9:31 AM, Martin Grigorov < >> > > [email protected] >> > > > > >> > > > > > wrote: >> > > > > > >> > > > > > > Hi, >> > > > > > > >> > > > > > > Once the http session is replicated this method should be >> > executed >> > > on >> > > > > the >> > > > > > > node that did not process the request >> > > > > > > org.apache.wicket.page.PageStoreManager.SessionEntry# >> > readObject(). >> > > > > > > Here Wicket will either store the page(s) on the disk or will >> > > > schedule >> > > > > > them >> > > > > > > for storing. >> > > > > > > There is no much usage of Loggers here so you will have to >> > attach a >> > > > > > > debugger and see what happens. >> > > > > > > >> > > > > > > Martin Grigorov >> > > > > > > Wicket Training and Consulting >> > > > > > > https://twitter.com/mtgrigorov >> > > > > > > >> > > > > > > On Thu, Mar 23, 2017 at 6:11 PM, Wayne W < >> > > > [email protected]> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > If I put the AbstractDefaultAjaxBehavior back in, it always >> > > fails. >> > > > > If I >> > > > > > > > remove the AbstractDefaultAjaxBehavior most of the time it >> > works, >> > > > but >> > > > > > > every >> > > > > > > > now and then it fails to replicate. Its really inconsistent. >> > > > > > > > >> > > > > > > > I have no idea how to get to the bottom of this. Any >> pointers >> > or >> > > > help >> > > > > > is >> > > > > > > > much appreciated. >> > > > > > > > >> > > > > > > > On Thu, Mar 23, 2017 at 4:53 PM, Wayne W < >> > > > > [email protected]> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > So it seems the homepage is not the only page. However >> > removing >> > > > the >> > > > > > > > > AbstractDefaultAjaxBehavior makes no difference I still >> get >> > the >> > > > > same >> > > > > > > > > issues. Just don;t know where to start with this one. Is >> > there >> > > a >> > > > > way >> > > > > > I >> > > > > > > > can >> > > > > > > > > turn on the logging around replication in wicket? >> > > > > > > > > >> > > > > > > > > On Thu, Mar 23, 2017 at 4:27 PM, Wayne W < >> > > > > > [email protected]> >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > >> Hi Martin, >> > > > > > > > >> >> > > > > > > > >> no - I still get the ComponentNotFoundException in the >> new >> > > > > instance. >> > > > > > > :-/ >> > > > > > > > >> >> > > > > > > > >> On Thu, Mar 23, 2017 at 3:57 PM, Martin Grigorov < >> > > > > > > [email protected]> >> > > > > > > > >> wrote: >> > > > > > > > >> >> > > > > > > > >>> Hi, >> > > > > > > > >>> >> > > > > > > > >>> On Thu, Mar 23, 2017 at 4:20 PM, Wayne W < >> > > > > > > [email protected]> >> > > > > > > > >>> wrote: >> > > > > > > > >>> >> > > > > > > > >>> > Hi Martin, >> > > > > > > > >>> > >> > > > > > > > >>> > that was a typo on my part. I've been doing a lot more >> > > > testing >> > > > > > > > before I >> > > > > > > > >>> > replied to make sure things are as I say. This is >> what I >> > am >> > > > > > > > observing: >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > - I have a 2 node tomcat cluster setup with apache >> > > > balancing >> > > > > > > > between >> > > > > > > > >>> > them locally on my machine. >> > > > > > > > >>> > - Testing with a very simple Wicket app I can see >> the >> > > > > > > replication >> > > > > > > > is >> > > > > > > > >>> > working fine when shutting down either instance. >> > > > > > > > >>> > - Testing with our very heavy weight wicket app it >> > > > sometimes >> > > > > > > work >> > > > > > > > >>> and >> > > > > > > > >>> > sometimes does not. >> > > > > > > > >>> > >> > > > > > > > >>> > Now the last point above I have narrowed down to our >> > > > homepage. >> > > > > > If I >> > > > > > > > am >> > > > > > > > >>> not >> > > > > > > > >>> > on the homepage the replication seems to work ok. >> However >> > > if >> > > > I >> > > > > am >> > > > > > > on >> > > > > > > > >>> the >> > > > > > > > >>> > homepage I always get a ComponentNotFoundException >> when >> > > > failing >> > > > > > > over >> > > > > > > > >>> to the >> > > > > > > > >>> > other instance. >> > > > > > > > >>> > The homepage has an AbstractDefaultAjaxBehavior which >> is >> > > > called >> > > > > > > once >> > > > > > > > >>> the >> > > > > > > > >>> > page is rendered, this in turn adds to the the page a >> > > > DataView >> > > > > > > > >>> containing a >> > > > > > > > >>> > lot of content and links. The user needs to be logged >> in >> > > > before >> > > > > > > they >> > > > > > > > >>> see >> > > > > > > > >>> > the homepage, so the session is already setup. >> > > > > > > > >>> > >> > > > > > > > >>> > I don't know what it is about this page that breaks >> the >> > > > > > > replication. >> > > > > > > > Is >> > > > > > > > >>> > there anything around the ajax part that could do >> this? >> > Any >> > > > > > > pointers >> > > > > > > > >>> would >> > > > > > > > >>> > be most welcome as it will be a long long task to >> break >> > > that >> > > > > page >> > > > > > > > down >> > > > > > > > >>> and >> > > > > > > > >>> > do rounds and rounds of cluster testing >> > > > > > > > >>> > >> > > > > > > > >>> >> > > > > > > > >>> Let's try something! >> > > > > > > > >>> In you Ajax callback method do something like: >> > > > > > > > >>> getSession().setAttribute("someKey", >> > > > > UUID.randomUUID().toString()) >> > > > > > > > >>> Does this trigger replication ? >> > > > > > > > >>> >> > > > > > > > >>> >> > > > > > > > >>> > >> > > > > > > > >>> > many thanks >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > On Wed, Mar 22, 2017 at 12:34 PM, Martin Grigorov < >> > > > > > > > >>> [email protected]> >> > > > > > > > >>> > wrote: >> > > > > > > > >>> > >> > > > > > > > >>> > > Hi, >> > > > > > > > >>> > > >> > > > > > > > >>> > > "I can stop one of the instances and I'm not logged >> in" >> > > > > > > > >>> > > This statement says that you don't really have a >> > properly >> > > > > > > > configured >> > > > > > > > >>> > > failover. >> > > > > > > > >>> > > If those two Tomcat instances are in a cluster then >> the >> > > > http >> > > > > > > > sessions >> > > > > > > > >>> > > should be replicated and you should stay logged in >> no >> > > > matter >> > > > > > > which >> > > > > > > > >>> one is >> > > > > > > > >>> > > serving the request. >> > > > > > > > >>> > > Wicket will store the used stateful page at the >> disks >> > for >> > > > all >> > > > > > > > >>> Tomcats in >> > > > > > > > >>> > > the cluster, if it the replication is actually >> working! >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > > On Wed, Mar 22, 2017 at 1:02 PM, Wayne W < >> > > > > > > > >>> [email protected]> >> > > > > > > > >>> > > wrote: >> > > > > > > > >>> > > >> > > > > > > > >>> > > > Hi, >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > We have 2 instances of Tomcat running with Apache >> > > sitting >> > > > > in >> > > > > > > > front >> > > > > > > > >>> > > > balancing between the Tomcat instances. I have >> > session >> > > > > > > > replication >> > > > > > > > >>> > setup >> > > > > > > > >>> > > > which seems to work for basic bookmarkable links >> on >> > the >> > > > > > pages. >> > > > > > > I >> > > > > > > > >>> can >> > > > > > > > >>> > stop >> > > > > > > > >>> > > > one of the instances and I'm not logged in as it >> > > failover >> > > > > to >> > > > > > > the >> > > > > > > > >>> other >> > > > > > > > >>> > > > instance. >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > However for normal Link<?> and ajax links etc I >> get >> > > > > > > > >>> > > > ComponentNotFoundException thrown as the new >> > instance >> > > > > cannot >> > > > > > > > find >> > > > > > > > >>> it >> > > > > > > > >>> > in >> > > > > > > > >>> > > > the session it seems. >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > Clearly I'm not understanding how wicket manages >> the >> > > page >> > > > > > state >> > > > > > > > or >> > > > > > > > >>> I've >> > > > > > > > >>> > > > configured something wrong. >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > Does wicket support full session failover ? >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > Many thanks >> > > > > > > > >>> > > > >> > > > > > > > >>> > > >> > > > > > > > >>> > >> > > > > > > > >>> >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
