Hi, On Thu, Mar 30, 2017 at 11:21 PM, Wayne W <waynemailingli...@gmail.com> wrote:
> Please ignore my last email I have done a lot more debugging and I *think* > I understand why replication does not work for us: > > > Whenever a stateful pages is used (touched) the SessionEntry is updated, > > i.e. the page is put into it. > > The SessionEntry is stored as an attribute in the HTTP Session. > > The web container has logic to detect if the HTTP session has been > updated > > in a request and if it was then it replicates the whole session to the > > other nodes. > > Once the session is replicated it is deserialized. Here > > SessionEntry#readObject() comes to play. It reads the pages from the > > SessionEntry and stores them into the disk (via IDiskStore). > > So the stateful pages should be available in the disks of all nodes. > > Stateless pages are always recreated from scratch, so they are not > > replicated. > > > I think this is a little wrong. Yes the SessionEntry in stored as an > attribute in the http session and is replicated. However the pages in the > SessionEntry cache are not replicated as they are transient. > > private transient List<IManageablePage> sessionCache; > This looks like a bug! It shouldn't be transient. Please open a ticket! Could you please provide your Tomcat configuration related to replication too ? A patch / Pull Request is also very welcome! Thank you! > > private transient List<Object> afterReadObject > > Therefore the pages are *not* deserialised as you say. The only time > SessionEntry#readObject() > comes to play (at least with Tomcat) is when a node is first brought up and > the daltamanager replicates ALL sessions across, at this point all the > current SessionEntry's in the other nodes http sessions are written across. > After this is done SessionEntry#readObject() is not called anymore. Any new > pages/updates on the original instance are not replicated across (only the > sessionId in the SessionEntry). > > This means any new pages created on the old instance (after the new > instance has started up) are not available in the http session or the > second level page store on the new instance. > Therefore when the old instance in shut down the user is load balanced to > the new instance. At this point the link in the page Wicket is looking for > does not exist in the SessionEntry cache or the PageStore so it creates the > page and looks for the component/link. > > This causes a ComponentNotFoundException for us because the links are > either in a DataView which is never rendered so does not exist, or the > other links are actually added to the page in an Ajax request and again > because the page is not rendered are not there, Wicket then throws the > exception and it appears to the user the session is lost. > > This is at least what I am observing. I can provide a Quick start to > demonstrate if needed. > > many thanks for you time thus far. > > > On Thu, Mar 30, 2017 at 4:29 PM, Wayne W <waynemailingli...@gmail.com> > wrote: > > > > > > > On Thu, Mar 30, 2017 at 4:09 PM, Martin Grigorov <mgrigo...@apache.org> > > wrote: > > > >> > >> > >> Whenever a stateful pages is used (touched) the SessionEntry is updated, > >> i.e. the page is put into it. > >> > > > > Yes I can see that happening in PageStoreManager.storeTouchedPages() > > > > > >> The SessionEntry is stored as an attribute in the HTTP Session. > >> > > > > I'm not seeing this. In the wicket.Session class there is line : > > > > private transient ISessionStore sessionStore; > > > > Which is loaded from the Application each request. The SessionStore > > contains the SessionEntry's no??? > > > > > > > >> The web container has logic to detect if the HTTP session has been > updated > >> in a request and if it was then it replicates the whole session to the > >> other nodes. > >> > > > > Yes I'm seeing the http session replicated no issues. > > > > > >> Once the session is replicated it is deserialized. Here > >> SessionEntry#readObject() comes to play. It reads the pages from the > >> SessionEntry and stores them into the disk (via IDiskStore). > >> So the stateful pages should be available in the disks of all nodes. > >> Stateless pages are always recreated from scratch, so they are not > >> replicated. > >> > > > > > > If this is the case why is wicket saying this page is stateless as the > > SessionEntry is never replicated across?: > > > > > > > > public class HomePage extends WebPage { > > > > private static final long serialVersionUID = 1L; > > > > > > public HomePage() { > > > > super(); > > > > > > Session s= Session.get(); > > > > s.bind(); > > > > Integer i = (Integer)s.getAttribute("foo"); > > > > if (i == null) { > > > > i = new Integer(0); > > > > } > > > > i++; > > > > s.setAttribute("foo",i); > > > > add(new Label("version", (Integer)s.getAttribute("foo"))); > > > > > > add(new Link<Void>("link1") { > > > > @Override > > > > public void onClick() > > > > { > > > > setResponsePage(new NextPage()); > > > > } > > > > }); > > > > add(new Link<Void>("link2") { > > > > @Override > > > > public void onClick() > > > > { > > > > setResponsePage(new HomePage()); > > > > } > > > > }); > > > > add(new AbstractDefaultAjaxBehavior() { > > > > @Override > > > > protected void respond(AjaxRequestTarget target) { > > > > getSession().setAttribute("someKey", UUID.randomUUID().toString()); > > > > System.out.println("SLEEEPING"); > > > > try { > > > > Thread.sleep(3000); > > > > }catch(Exception e) { > > > > } > > > > System.out.println("DONE"); > > > > } > > > > @Override > > > > public void renderHead(Component component, IHeaderResponse response) { > > > > super.renderHead(component, response); > > > > String js = "(" + getCallbackFunction().toString() +")();"; > > > > response.render(OnDomReadyHeaderItem.forScript(js)); > > > > } > > > > }); > > > > // TODO Add your page's components here > > > > > > } > > > > } > > > > > > > > Sorry for all these questions but I must get to the bottom of this (5 > days > > work so far). Many thanks > > > > > > > >> > >> > >> > > >> > > >> > thanks > >> > > >> > On Tue, Mar 28, 2017 at 9:39 PM, Martin Grigorov < > mgrigo...@apache.org> > >> > wrote: > >> > > >> > > Hi, > >> > > > >> > > Did you mention already which version of Wicket you use ? > >> > > There was an improvement related to reusing the page parameters when > >> > > constructing a new page instance in 7.0.0. > >> > > I have the feeling you are using 6.x. Am I correct ? > >> > > > >> > > Martin Grigorov > >> > > Wicket Training and Consulting > >> > > https://twitter.com/mtgrigorov > >> > > > >> > > On Tue, Mar 28, 2017 at 7:22 PM, Wayne W < > waynemailingli...@gmail.com > >> > > >> > > wrote: > >> > > > >> > > > Hello Martin, > >> > > > > >> > > > so I've been trying to still get to the bottom of this days later. > >> So > >> > my > >> > > > understanding is getting better and it appears that the session > >> itself > >> > is > >> > > > replicated fine. However I've tracked my issue down to the > >> following: > >> > > > > >> > > > - If I have a page that has 2 links at the top. Link A is a > >> > Bookmarkable > >> > > > link back to the same page - it is also mounted thus: mountPage( > >> > > > "/cube/documents/${0}", CubeDocumentPage.class); It has a Long > page > >> > param > >> > > > needed to construct the page ( i.e. /cube/documents/12345 ). The > >> second > >> > > > link B is just a simple new Link<Void>("B") . > >> > > > - If both instances are up and running , lets say I am on instance > >> A. I > >> > > > visit link A 3 times. I then kill instance A and I am balanced > over > >> to > >> > > > instance B. If I visit link B , the PageStoreManager cannot find > >> the > >> > > page > >> > > > that contains link B in the store and then tries to create a new > >> > instance > >> > > > of the page - the problem is then for some reason the page > >> parameters > >> > are > >> > > > always null and the Long is never passed. Why is the page > parameter > >> > > always > >> > > > null here? Trying to debug it, is seems the IPageManager is got > from > >> > the > >> > > > wicket Application instance and not the Session, and this > >> IPageManager > >> > > > looks for a RequestAdaptor. Somewhere the page parameters are > lost?. > >> > > > > >> > > > However I can get it to work this way: > >> > > > - start instance A, visit the page and say click link A 3 times. > >> > > > - Now start up instance B > >> > > > - Kill instance A > >> > > > - Click on the link B and this time the PageStoreManager finds the > >> page > >> > > and > >> > > > there the link B and everything works fine. > >> > > > It only works if I don't visit another page just after instance B > >> > starts > >> > > > up. > >> > > > > >> > > > > >> > > > I will do some more debugging tomorrow to try and understand this > >> > > > PageStoreManager/request adaptor bit more. But if you have any > ideas > >> > I'd > >> > > > appreciate it. > >> > > > Thanks > >> > > > > >> > > > > >> > > > > >> > > > On Fri, Mar 24, 2017 at 12:48 PM, Martin Grigorov < > >> > mgrigo...@apache.org> > >> > > > wrote: > >> > > > > >> > > > > Hi, > >> > > > > > >> > > > > On Fri, Mar 24, 2017 at 1:31 PM, Wayne W < > >> > waynemailingli...@gmail.com> > >> > > > > wrote: > >> > > > > > >> > > > > > Thanks Martin, > >> > > > > > > >> > > > > > I have a theory what this is, perhaps you could confirm? > >> > > > > > > >> > > > > > What I observe is the following with the replication: If I > visit > >> > > page A > >> > > > > and > >> > > > > > then visit page B, then kill the instance I am on the session > is > >> > > > > > successfully failed over to the other node. Now I'm still > >> looking > >> > at > >> > > > > page B > >> > > > > > in the browser - if I hit back on the browser I get > >> > > > PageExpiredException. > >> > > > > > Is this the expected behaviour? > >> > > > > > > >> > > > > > >> > > > > No. > >> > > > > The expected behavior is to see the latest state of page A. > >> > > > > > >> > > > > > >> > > > > > > >> > > > > > If it IS the expected behaviour, then the reason I think that > I > >> > have > >> > > an > >> > > > > > issue with the page with the AbstractDefaultAjaxBehavior is > >> because > >> > > the > >> > > > > > code in the AbstractDefaultAjaxBehaviour.respond is adding > >> > > components > >> > > > to > >> > > > > > the page which in turn marks the Session dirty which in turn > >> > > replicates > >> > > > > the > >> > > > > > session. If only the last page/session commit is only > >> replicated it > >> > > > could > >> > > > > > explain why I get a ComponentNotFound exception from the page > as > >> > > > > > it overwriten by the update from the > AbstractDefaultAjaxBehavior > >> > > > > > > >> > > > > > >> > > > > Any time the page is marked as dirty Wicket will store it in 1) > >> the > >> > > HTTP > >> > > > > session and 2) on the disk > >> > > > > 1) will notify the web server (e.g. Tomcat) that it has to > >> replicate > >> > > the > >> > > > > session because some attribute has been modified > >> > > > > At node2 Wicket will detect the replicated session in > >> #readObject() > >> > and > >> > > > > store it on the disk. > >> > > > > From now on node2 will have the same history for that page as > >> node1, > >> > > i.e. > >> > > > > PageA and PageB. > >> > > > > > >> > > > > In addition: > >> > > > > Updates made in an Ajax requests do not add a new entry in the > >> > history > >> > > > > (storages) but overrides the previous entry for that page > >> instance! > >> > > > > I.e. when PageA is rendered Wicket will assign a pageId for it, > >> e.g. > >> > 5, > >> > > > and > >> > > > > store it in the disk. > >> > > > > Later when you make a change to PageA:5 in Ajax request then > >> Wicket > >> > > will > >> > > > > override the entry on the disk. > >> > > > > If you make a change in non-Ajax request then Wicket will > assign a > >> > new > >> > > > > pageId, e.g. 6, and add a new entry in the history (disk), so > you > >> > will > >> > > > have > >> > > > > two entries for this session. > >> > > > > Later when you open PageB then this will be pageId 7, etc. > >> > > > > > >> > > > > > >> > > > > > > >> > > > > > What do you think? > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > On Fri, Mar 24, 2017 at 9:31 AM, Martin Grigorov < > >> > > mgrigo...@apache.org > >> > > > > > >> > > > > > wrote: > >> > > > > > > >> > > > > > > Hi, > >> > > > > > > > >> > > > > > > Once the http session is replicated this method should be > >> > executed > >> > > on > >> > > > > the > >> > > > > > > node that did not process the request > >> > > > > > > org.apache.wicket.page.PageStoreManager.SessionEntry# > >> > readObject(). > >> > > > > > > Here Wicket will either store the page(s) on the disk or > will > >> > > > schedule > >> > > > > > them > >> > > > > > > for storing. > >> > > > > > > There is no much usage of Loggers here so you will have to > >> > attach a > >> > > > > > > debugger and see what happens. > >> > > > > > > > >> > > > > > > Martin Grigorov > >> > > > > > > Wicket Training and Consulting > >> > > > > > > https://twitter.com/mtgrigorov > >> > > > > > > > >> > > > > > > On Thu, Mar 23, 2017 at 6:11 PM, Wayne W < > >> > > > waynemailingli...@gmail.com> > >> > > > > > > wrote: > >> > > > > > > > >> > > > > > > > If I put the AbstractDefaultAjaxBehavior back in, it > always > >> > > fails. > >> > > > > If I > >> > > > > > > > remove the AbstractDefaultAjaxBehavior most of the time it > >> > works, > >> > > > but > >> > > > > > > every > >> > > > > > > > now and then it fails to replicate. Its really > inconsistent. > >> > > > > > > > > >> > > > > > > > I have no idea how to get to the bottom of this. Any > >> pointers > >> > or > >> > > > help > >> > > > > > is > >> > > > > > > > much appreciated. > >> > > > > > > > > >> > > > > > > > On Thu, Mar 23, 2017 at 4:53 PM, Wayne W < > >> > > > > waynemailingli...@gmail.com> > >> > > > > > > > wrote: > >> > > > > > > > > >> > > > > > > > > So it seems the homepage is not the only page. However > >> > removing > >> > > > the > >> > > > > > > > > AbstractDefaultAjaxBehavior makes no difference I still > >> get > >> > the > >> > > > > same > >> > > > > > > > > issues. Just don;t know where to start with this one. Is > >> > there > >> > > a > >> > > > > way > >> > > > > > I > >> > > > > > > > can > >> > > > > > > > > turn on the logging around replication in wicket? > >> > > > > > > > > > >> > > > > > > > > On Thu, Mar 23, 2017 at 4:27 PM, Wayne W < > >> > > > > > waynemailingli...@gmail.com> > >> > > > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > >> Hi Martin, > >> > > > > > > > >> > >> > > > > > > > >> no - I still get the ComponentNotFoundException in the > >> new > >> > > > > instance. > >> > > > > > > :-/ > >> > > > > > > > >> > >> > > > > > > > >> On Thu, Mar 23, 2017 at 3:57 PM, Martin Grigorov < > >> > > > > > > mgrigo...@apache.org> > >> > > > > > > > >> wrote: > >> > > > > > > > >> > >> > > > > > > > >>> Hi, > >> > > > > > > > >>> > >> > > > > > > > >>> On Thu, Mar 23, 2017 at 4:20 PM, Wayne W < > >> > > > > > > waynemailingli...@gmail.com> > >> > > > > > > > >>> wrote: > >> > > > > > > > >>> > >> > > > > > > > >>> > Hi Martin, > >> > > > > > > > >>> > > >> > > > > > > > >>> > that was a typo on my part. I've been doing a lot > more > >> > > > testing > >> > > > > > > > before I > >> > > > > > > > >>> > replied to make sure things are as I say. This is > >> what I > >> > am > >> > > > > > > > observing: > >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > - I have a 2 node tomcat cluster setup with > apache > >> > > > balancing > >> > > > > > > > between > >> > > > > > > > >>> > them locally on my machine. > >> > > > > > > > >>> > - Testing with a very simple Wicket app I can see > >> the > >> > > > > > > replication > >> > > > > > > > is > >> > > > > > > > >>> > working fine when shutting down either instance. > >> > > > > > > > >>> > - Testing with our very heavy weight wicket app > it > >> > > > sometimes > >> > > > > > > work > >> > > > > > > > >>> and > >> > > > > > > > >>> > sometimes does not. > >> > > > > > > > >>> > > >> > > > > > > > >>> > Now the last point above I have narrowed down to our > >> > > > homepage. > >> > > > > > If I > >> > > > > > > > am > >> > > > > > > > >>> not > >> > > > > > > > >>> > on the homepage the replication seems to work ok. > >> However > >> > > if > >> > > > I > >> > > > > am > >> > > > > > > on > >> > > > > > > > >>> the > >> > > > > > > > >>> > homepage I always get a ComponentNotFoundException > >> when > >> > > > failing > >> > > > > > > over > >> > > > > > > > >>> to the > >> > > > > > > > >>> > other instance. > >> > > > > > > > >>> > The homepage has an AbstractDefaultAjaxBehavior > which > >> is > >> > > > called > >> > > > > > > once > >> > > > > > > > >>> the > >> > > > > > > > >>> > page is rendered, this in turn adds to the the page > a > >> > > > DataView > >> > > > > > > > >>> containing a > >> > > > > > > > >>> > lot of content and links. The user needs to be > logged > >> in > >> > > > before > >> > > > > > > they > >> > > > > > > > >>> see > >> > > > > > > > >>> > the homepage, so the session is already setup. > >> > > > > > > > >>> > > >> > > > > > > > >>> > I don't know what it is about this page that breaks > >> the > >> > > > > > > replication. > >> > > > > > > > Is > >> > > > > > > > >>> > there anything around the ajax part that could do > >> this? > >> > Any > >> > > > > > > pointers > >> > > > > > > > >>> would > >> > > > > > > > >>> > be most welcome as it will be a long long task to > >> break > >> > > that > >> > > > > page > >> > > > > > > > down > >> > > > > > > > >>> and > >> > > > > > > > >>> > do rounds and rounds of cluster testing > >> > > > > > > > >>> > > >> > > > > > > > >>> > >> > > > > > > > >>> Let's try something! > >> > > > > > > > >>> In you Ajax callback method do something like: > >> > > > > > > > >>> getSession().setAttribute("someKey", > >> > > > > UUID.randomUUID().toString()) > >> > > > > > > > >>> Does this trigger replication ? > >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > > >> > > > > > > > >>> > many thanks > >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > > >> > > > > > > > >>> > On Wed, Mar 22, 2017 at 12:34 PM, Martin Grigorov < > >> > > > > > > > >>> mgrigo...@apache.org> > >> > > > > > > > >>> > wrote: > >> > > > > > > > >>> > > >> > > > > > > > >>> > > Hi, > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > "I can stop one of the instances and I'm not > logged > >> in" > >> > > > > > > > >>> > > This statement says that you don't really have a > >> > properly > >> > > > > > > > configured > >> > > > > > > > >>> > > failover. > >> > > > > > > > >>> > > If those two Tomcat instances are in a cluster > then > >> the > >> > > > http > >> > > > > > > > sessions > >> > > > > > > > >>> > > should be replicated and you should stay logged in > >> no > >> > > > matter > >> > > > > > > which > >> > > > > > > > >>> one is > >> > > > > > > > >>> > > serving the request. > >> > > > > > > > >>> > > Wicket will store the used stateful page at the > >> disks > >> > for > >> > > > all > >> > > > > > > > >>> Tomcats in > >> > > > > > > > >>> > > the cluster, if it the replication is actually > >> working! > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > On Wed, Mar 22, 2017 at 1:02 PM, Wayne W < > >> > > > > > > > >>> waynemailingli...@gmail.com> > >> > > > > > > > >>> > > wrote: > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > > Hi, > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > We have 2 instances of Tomcat running with > Apache > >> > > sitting > >> > > > > in > >> > > > > > > > front > >> > > > > > > > >>> > > > balancing between the Tomcat instances. I have > >> > session > >> > > > > > > > replication > >> > > > > > > > >>> > setup > >> > > > > > > > >>> > > > which seems to work for basic bookmarkable links > >> on > >> > the > >> > > > > > pages. > >> > > > > > > I > >> > > > > > > > >>> can > >> > > > > > > > >>> > stop > >> > > > > > > > >>> > > > one of the instances and I'm not logged in as it > >> > > failover > >> > > > > to > >> > > > > > > the > >> > > > > > > > >>> other > >> > > > > > > > >>> > > > instance. > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > However for normal Link<?> and ajax links etc I > >> get > >> > > > > > > > >>> > > > ComponentNotFoundException thrown as the new > >> > instance > >> > > > > cannot > >> > > > > > > > find > >> > > > > > > > >>> it > >> > > > > > > > >>> > in > >> > > > > > > > >>> > > > the session it seems. > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > Clearly I'm not understanding how wicket manages > >> the > >> > > page > >> > > > > > state > >> > > > > > > > or > >> > > > > > > > >>> I've > >> > > > > > > > >>> > > > configured something wrong. > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > Does wicket support full session failover ? > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > Many thanks > >> > > > > > > > >>> > > > > >> > > > > > > > >>> > > > >> > > > > > > > >>> > > >> > > > > > > > >>> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > > >