Hi,
On Thu, Mar 30, 2017 at 11:21 PM, Wayne W <waynemailingli...@gmail.com>
wrote:
> Please ignore my last email I have done a lot more debugging and I *think*
> I understand why replication does not work for us:
>
>
> Whenever a stateful pages is used (touched) the SessionEntry is updated,
> > i.e. the page is put into it.
> > The SessionEntry is stored as an attribute in the HTTP Session.
> > The web container has logic to detect if the HTTP session has been
> updated
> > in a request and if it was then it replicates the whole session to the
> > other nodes.
> > Once the session is replicated it is deserialized. Here
> > SessionEntry#readObject() comes to play. It reads the pages from the
> > SessionEntry and stores them into the disk (via IDiskStore).
> > So the stateful pages should be available in the disks of all nodes.
> > Stateless pages are always recreated from scratch, so they are not
> > replicated.
>
>
> I think this is a little wrong. Yes the SessionEntry in stored as an
> attribute in the http session and is replicated. However the pages in the
> SessionEntry cache are not replicated as they are transient.
>
> private transient List<IManageablePage> sessionCache;
>
This looks like a bug!
It shouldn't be transient.
Please open a ticket!
Could you please provide your Tomcat configuration related to replication
too ?
A patch / Pull Request is also very welcome!
Thank you!
>
> private transient List<Object> afterReadObject
>
> Therefore the pages are *not* deserialised as you say. The only time
> SessionEntry#readObject()
> comes to play (at least with Tomcat) is when a node is first brought up and
> the daltamanager replicates ALL sessions across, at this point all the
> current SessionEntry's in the other nodes http sessions are written across.
> After this is done SessionEntry#readObject() is not called anymore. Any new
> pages/updates on the original instance are not replicated across (only the
> sessionId in the SessionEntry).
>
> This means any new pages created on the old instance (after the new
> instance has started up) are not available in the http session or the
> second level page store on the new instance.
> Therefore when the old instance in shut down the user is load balanced to
> the new instance. At this point the link in the page Wicket is looking for
> does not exist in the SessionEntry cache or the PageStore so it creates the
> page and looks for the component/link.
>
> This causes a ComponentNotFoundException for us because the links are
> either in a DataView which is never rendered so does not exist, or the
> other links are actually added to the page in an Ajax request and again
> because the page is not rendered are not there, Wicket then throws the
> exception and it appears to the user the session is lost.
>
> This is at least what I am observing. I can provide a Quick start to
> demonstrate if needed.
>
> many thanks for you time thus far.
>
>
> On Thu, Mar 30, 2017 at 4:29 PM, Wayne W <waynemailingli...@gmail.com>
> wrote:
>
> >
> >
> > On Thu, Mar 30, 2017 at 4:09 PM, Martin Grigorov <mgrigo...@apache.org>
> > wrote:
> >
> >>
> >>
> >> Whenever a stateful pages is used (touched) the SessionEntry is updated,
> >> i.e. the page is put into it.
> >>
> >
> > Yes I can see that happening in PageStoreManager.storeTouchedPages()
> >
> >
> >> The SessionEntry is stored as an attribute in the HTTP Session.
> >>
> >
> > I'm not seeing this. In the wicket.Session class there is line :
> >
> > private transient ISessionStore sessionStore;
> >
> > Which is loaded from the Application each request. The SessionStore
> > contains the SessionEntry's no???
> >
> >
> >
> >> The web container has logic to detect if the HTTP session has been
> updated
> >> in a request and if it was then it replicates the whole session to the
> >> other nodes.
> >>
> >
> > Yes I'm seeing the http session replicated no issues.
> >
> >
> >> Once the session is replicated it is deserialized. Here
> >> SessionEntry#readObject() comes to play. It reads the pages from the
> >> SessionEntry and stores them into the disk (via IDiskStore).
> >> So the stateful pages should be available in the disks of all nodes.
> >> Stateless pages are always recreated from scratch, so they are not
> >> replicated.
> >>
> >
> >
> > If this is the case why is wicket saying this page is stateless as the
> > SessionEntry is never replicated across?:
> >
> >
> >
> > public class HomePage extends WebPage {
> >
> > private static final long serialVersionUID = 1L;
> >
> >
> > public HomePage() {
> >
> > super();
> >
> >
> > Session s= Session.get();
> >
> > s.bind();
> >
> > Integer i = (Integer)s.getAttribute("foo");
> >
> > if (i == null) {
> >
> > i = new Integer(0);
> >
> > }
> >
> > i++;
> >
> > s.setAttribute("foo",i);
> >
> > add(new Label("version", (Integer)s.getAttribute("foo")));
> >
> >
> > add(new Link<Void>("link1") {
> >
> > @Override
> >
> > public void onClick()
> >
> > {
> >
> > setResponsePage(new NextPage());
> >
> > }
> >
> > });
> >
> > add(new Link<Void>("link2") {
> >
> > @Override
> >
> > public void onClick()
> >
> > {
> >
> > setResponsePage(new HomePage());
> >
> > }
> >
> > });
> >
> > add(new AbstractDefaultAjaxBehavior() {
> >
> > @Override
> >
> > protected void respond(AjaxRequestTarget target) {
> >
> > getSession().setAttribute("someKey", UUID.randomUUID().toString());
> >
> > System.out.println("SLEEEPING");
> >
> > try {
> >
> > Thread.sleep(3000);
> >
> > }catch(Exception e) {
> >
> > }
> >
> > System.out.println("DONE");
> >
> > }
> >
> > @Override
> >
> > public void renderHead(Component component, IHeaderResponse response) {
> >
> > super.renderHead(component, response);
> >
> > String js = "(" + getCallbackFunction().toString() +")();";
> >
> > response.render(OnDomReadyHeaderItem.forScript(js));
> >
> > }
> >
> > });
> >
> > // TODO Add your page's components here
> >
> >
> > }
> >
> > }
> >
> >
> >
> > Sorry for all these questions but I must get to the bottom of this (5
> days
> > work so far). Many thanks
> >
> >
> >
> >>
> >>
> >> >
> >> >
> >> > thanks
> >> >
> >> > On Tue, Mar 28, 2017 at 9:39 PM, Martin Grigorov <
> mgrigo...@apache.org>
> >> > wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > Did you mention already which version of Wicket you use ?
> >> > > There was an improvement related to reusing the page parameters when
> >> > > constructing a new page instance in 7.0.0.
> >> > > I have the feeling you are using 6.x. Am I correct ?
> >> > >
> >> > > Martin Grigorov
> >> > > Wicket Training and Consulting
> >> > > https://twitter.com/mtgrigorov
> >> > >
> >> > > On Tue, Mar 28, 2017 at 7:22 PM, Wayne W <
> waynemailingli...@gmail.com
> >> >
> >> > > wrote:
> >> > >
> >> > > > Hello Martin,
> >> > > >
> >> > > > so I've been trying to still get to the bottom of this days later.
> >> So
> >> > my
> >> > > > understanding is getting better and it appears that the session
> >> itself
> >> > is
> >> > > > replicated fine. However I've tracked my issue down to the
> >> following:
> >> > > >
> >> > > > - If I have a page that has 2 links at the top. Link A is a
> >> > Bookmarkable
> >> > > > link back to the same page - it is also mounted thus: mountPage(
> >> > > > "/cube/documents/${0}", CubeDocumentPage.class); It has a Long
> page
> >> > param
> >> > > > needed to construct the page ( i.e. /cube/documents/12345 ). The
> >> second
> >> > > > link B is just a simple new Link<Void>("B") .
> >> > > > - If both instances are up and running , lets say I am on instance
> >> A. I
> >> > > > visit link A 3 times. I then kill instance A and I am balanced
> over
> >> to
> >> > > > instance B. If I visit link B , the PageStoreManager cannot find
> >> the
> >> > > page
> >> > > > that contains link B in the store and then tries to create a new
> >> > instance
> >> > > > of the page - the problem is then for some reason the page
> >> parameters
> >> > are
> >> > > > always null and the Long is never passed. Why is the page
> parameter
> >> > > always
> >> > > > null here? Trying to debug it, is seems the IPageManager is got
> from
> >> > the
> >> > > > wicket Application instance and not the Session, and this
> >> IPageManager
> >> > > > looks for a RequestAdaptor. Somewhere the page parameters are
> lost?.
> >> > > >
> >> > > > However I can get it to work this way:
> >> > > > - start instance A, visit the page and say click link A 3 times.
> >> > > > - Now start up instance B
> >> > > > - Kill instance A
> >> > > > - Click on the link B and this time the PageStoreManager finds the
> >> page
> >> > > and
> >> > > > there the link B and everything works fine.
> >> > > > It only works if I don't visit another page just after instance B
> >> > starts
> >> > > > up.
> >> > > >
> >> > > >
> >> > > > I will do some more debugging tomorrow to try and understand this
> >> > > > PageStoreManager/request adaptor bit more. But if you have any
> ideas
> >> > I'd
> >> > > > appreciate it.
> >> > > > Thanks
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Fri, Mar 24, 2017 at 12:48 PM, Martin Grigorov <
> >> > mgrigo...@apache.org>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > On Fri, Mar 24, 2017 at 1:31 PM, Wayne W <
> >> > waynemailingli...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Thanks Martin,
> >> > > > > >
> >> > > > > > I have a theory what this is, perhaps you could confirm?
> >> > > > > >
> >> > > > > > What I observe is the following with the replication: If I
> visit
> >> > > page A
> >> > > > > and
> >> > > > > > then visit page B, then kill the instance I am on the session
> is
> >> > > > > > successfully failed over to the other node. Now I'm still
> >> looking
> >> > at
> >> > > > > page B
> >> > > > > > in the browser - if I hit back on the browser I get
> >> > > > PageExpiredException.
> >> > > > > > Is this the expected behaviour?
> >> > > > > >
> >> > > > >
> >> > > > > No.
> >> > > > > The expected behavior is to see the latest state of page A.
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > If it IS the expected behaviour, then the reason I think that
> I
> >> > have
> >> > > an
> >> > > > > > issue with the page with the AbstractDefaultAjaxBehavior is
> >> because
> >> > > the
> >> > > > > > code in the AbstractDefaultAjaxBehaviour.respond is adding
> >> > > components
> >> > > > to
> >> > > > > > the page which in turn marks the Session dirty which in turn
> >> > > replicates
> >> > > > > the
> >> > > > > > session. If only the last page/session commit is only
> >> replicated it
> >> > > > could
> >> > > > > > explain why I get a ComponentNotFound exception from the page
> as
> >> > > > > > it overwriten by the update from the
> AbstractDefaultAjaxBehavior
> >> > > > > >
> >> > > > >
> >> > > > > Any time the page is marked as dirty Wicket will store it in 1)
> >> the
> >> > > HTTP
> >> > > > > session and 2) on the disk
> >> > > > > 1) will notify the web server (e.g. Tomcat) that it has to
> >> replicate
> >> > > the
> >> > > > > session because some attribute has been modified
> >> > > > > At node2 Wicket will detect the replicated session in
> >> #readObject()
> >> > and
> >> > > > > store it on the disk.
> >> > > > > From now on node2 will have the same history for that page as
> >> node1,
> >> > > i.e.
> >> > > > > PageA and PageB.
> >> > > > >
> >> > > > > In addition:
> >> > > > > Updates made in an Ajax requests do not add a new entry in the
> >> > history
> >> > > > > (storages) but overrides the previous entry for that page
> >> instance!
> >> > > > > I.e. when PageA is rendered Wicket will assign a pageId for it,
> >> e.g.
> >> > 5,
> >> > > > and
> >> > > > > store it in the disk.
> >> > > > > Later when you make a change to PageA:5 in Ajax request then
> >> Wicket
> >> > > will
> >> > > > > override the entry on the disk.
> >> > > > > If you make a change in non-Ajax request then Wicket will
> assign a
> >> > new
> >> > > > > pageId, e.g. 6, and add a new entry in the history (disk), so
> you
> >> > will
> >> > > > have
> >> > > > > two entries for this session.
> >> > > > > Later when you open PageB then this will be pageId 7, etc.
> >> > > > >
> >> > > > >
> >> > > > > >
> >> > > > > > What do you think?
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Fri, Mar 24, 2017 at 9:31 AM, Martin Grigorov <
> >> > > mgrigo...@apache.org
> >> > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi,
> >> > > > > > >
> >> > > > > > > Once the http session is replicated this method should be
> >> > executed
> >> > > on
> >> > > > > the
> >> > > > > > > node that did not process the request
> >> > > > > > > org.apache.wicket.page.PageStoreManager.SessionEntry#
> >> > readObject().
> >> > > > > > > Here Wicket will either store the page(s) on the disk or
> will
> >> > > > schedule
> >> > > > > > them
> >> > > > > > > for storing.
> >> > > > > > > There is no much usage of Loggers here so you will have to
> >> > attach a
> >> > > > > > > debugger and see what happens.
> >> > > > > > >
> >> > > > > > > Martin Grigorov
> >> > > > > > > Wicket Training and Consulting
> >> > > > > > > https://twitter.com/mtgrigorov
> >> > > > > > >
> >> > > > > > > On Thu, Mar 23, 2017 at 6:11 PM, Wayne W <
> >> > > > waynemailingli...@gmail.com>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > If I put the AbstractDefaultAjaxBehavior back in, it
> always
> >> > > fails.
> >> > > > > If I
> >> > > > > > > > remove the AbstractDefaultAjaxBehavior most of the time it
> >> > works,
> >> > > > but
> >> > > > > > > every
> >> > > > > > > > now and then it fails to replicate. Its really
> inconsistent.
> >> > > > > > > >
> >> > > > > > > > I have no idea how to get to the bottom of this. Any
> >> pointers
> >> > or
> >> > > > help
> >> > > > > > is
> >> > > > > > > > much appreciated.
> >> > > > > > > >
> >> > > > > > > > On Thu, Mar 23, 2017 at 4:53 PM, Wayne W <
> >> > > > > waynemailingli...@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > So it seems the homepage is not the only page. However
> >> > removing
> >> > > > the
> >> > > > > > > > > AbstractDefaultAjaxBehavior makes no difference I still
> >> get
> >> > the
> >> > > > > same
> >> > > > > > > > > issues. Just don;t know where to start with this one. Is
> >> > there
> >> > > a
> >> > > > > way
> >> > > > > > I
> >> > > > > > > > can
> >> > > > > > > > > turn on the logging around replication in wicket?
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Mar 23, 2017 at 4:27 PM, Wayne W <
> >> > > > > > waynemailingli...@gmail.com>
> >> > > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > >> Hi Martin,
> >> > > > > > > > >>
> >> > > > > > > > >> no - I still get the ComponentNotFoundException in the
> >> new
> >> > > > > instance.
> >> > > > > > > :-/
> >> > > > > > > > >>
> >> > > > > > > > >> On Thu, Mar 23, 2017 at 3:57 PM, Martin Grigorov <
> >> > > > > > > mgrigo...@apache.org>
> >> > > > > > > > >> wrote:
> >> > > > > > > > >>
> >> > > > > > > > >>> Hi,
> >> > > > > > > > >>>
> >> > > > > > > > >>> On Thu, Mar 23, 2017 at 4:20 PM, Wayne W <
> >> > > > > > > waynemailingli...@gmail.com>
> >> > > > > > > > >>> wrote:
> >> > > > > > > > >>>
> >> > > > > > > > >>> > Hi Martin,
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > that was a typo on my part. I've been doing a lot
> more
> >> > > > testing
> >> > > > > > > > before I
> >> > > > > > > > >>> > replied to make sure things are as I say. This is
> >> what I
> >> > am
> >> > > > > > > > observing:
> >> > > > > > > > >>> >
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > - I have a 2 node tomcat cluster setup with
> apache
> >> > > > balancing
> >> > > > > > > > between
> >> > > > > > > > >>> > them locally on my machine.
> >> > > > > > > > >>> > - Testing with a very simple Wicket app I can see
> >> the
> >> > > > > > > replication
> >> > > > > > > > is
> >> > > > > > > > >>> > working fine when shutting down either instance.
> >> > > > > > > > >>> > - Testing with our very heavy weight wicket app
> it
> >> > > > sometimes
> >> > > > > > > work
> >> > > > > > > > >>> and
> >> > > > > > > > >>> > sometimes does not.
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > Now the last point above I have narrowed down to our
> >> > > > homepage.
> >> > > > > > If I
> >> > > > > > > > am
> >> > > > > > > > >>> not
> >> > > > > > > > >>> > on the homepage the replication seems to work ok.
> >> However
> >> > > if
> >> > > > I
> >> > > > > am
> >> > > > > > > on
> >> > > > > > > > >>> the
> >> > > > > > > > >>> > homepage I always get a ComponentNotFoundException
> >> when
> >> > > > failing
> >> > > > > > > over
> >> > > > > > > > >>> to the
> >> > > > > > > > >>> > other instance.
> >> > > > > > > > >>> > The homepage has an AbstractDefaultAjaxBehavior
> which
> >> is
> >> > > > called
> >> > > > > > > once
> >> > > > > > > > >>> the
> >> > > > > > > > >>> > page is rendered, this in turn adds to the the page
> a
> >> > > > DataView
> >> > > > > > > > >>> containing a
> >> > > > > > > > >>> > lot of content and links. The user needs to be
> logged
> >> in
> >> > > > before
> >> > > > > > > they
> >> > > > > > > > >>> see
> >> > > > > > > > >>> > the homepage, so the session is already setup.
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > I don't know what it is about this page that breaks
> >> the
> >> > > > > > > replication.
> >> > > > > > > > Is
> >> > > > > > > > >>> > there anything around the ajax part that could do
> >> this?
> >> > Any
> >> > > > > > > pointers
> >> > > > > > > > >>> would
> >> > > > > > > > >>> > be most welcome as it will be a long long task to
> >> break
> >> > > that
> >> > > > > page
> >> > > > > > > > down
> >> > > > > > > > >>> and
> >> > > > > > > > >>> > do rounds and rounds of cluster testing
> >> > > > > > > > >>> >
> >> > > > > > > > >>>
> >> > > > > > > > >>> Let's try something!
> >> > > > > > > > >>> In you Ajax callback method do something like:
> >> > > > > > > > >>> getSession().setAttribute("someKey",
> >> > > > > UUID.randomUUID().toString())
> >> > > > > > > > >>> Does this trigger replication ?
> >> > > > > > > > >>>
> >> > > > > > > > >>>
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > many thanks
> >> > > > > > > > >>> >
> >> > > > > > > > >>> >
> >> > > > > > > > >>> >
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > On Wed, Mar 22, 2017 at 12:34 PM, Martin Grigorov <
> >> > > > > > > > >>> mgrigo...@apache.org>
> >> > > > > > > > >>> > wrote:
> >> > > > > > > > >>> >
> >> > > > > > > > >>> > > Hi,
> >> > > > > > > > >>> > >
> >> > > > > > > > >>> > > "I can stop one of the instances and I'm not
> logged
> >> in"
> >> > > > > > > > >>> > > This statement says that you don't really have a
> >> > properly
> >> > > > > > > > configured
> >> > > > > > > > >>> > > failover.
> >> > > > > > > > >>> > > If those two Tomcat instances are in a cluster
> then
> >> the
> >> > > > http
> >> > > > > > > > sessions
> >> > > > > > > > >>> > > should be replicated and you should stay logged in
> >> no
> >> > > > matter
> >> > > > > > > which
> >> > > > > > > > >>> one is
> >> > > > > > > > >>> > > serving the request.
> >> > > > > > > > >>> > > Wicket will store the used stateful page at the
> >> disks
> >> > for
> >> > > > all
> >> > > > > > > > >>> Tomcats in
> >> > > > > > > > >>> > > the cluster, if it the replication is actually
> >> working!
> >> > > > > > > > >>> > >
> >> > > > > > > > >>> > >
> >> > > > > > > > >>> > > On Wed, Mar 22, 2017 at 1:02 PM, Wayne W <
> >> > > > > > > > >>> waynemailingli...@gmail.com>
> >> > > > > > > > >>> > > wrote:
> >> > > > > > > > >>> > >
> >> > > > > > > > >>> > > > Hi,
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > > > We have 2 instances of Tomcat running with
> Apache
> >> > > sitting
> >> > > > > in
> >> > > > > > > > front
> >> > > > > > > > >>> > > > balancing between the Tomcat instances. I have
> >> > session
> >> > > > > > > > replication
> >> > > > > > > > >>> > setup
> >> > > > > > > > >>> > > > which seems to work for basic bookmarkable links
> >> on
> >> > the
> >> > > > > > pages.
> >> > > > > > > I
> >> > > > > > > > >>> can
> >> > > > > > > > >>> > stop
> >> > > > > > > > >>> > > > one of the instances and I'm not logged in as it
> >> > > failover
> >> > > > > to
> >> > > > > > > the
> >> > > > > > > > >>> other
> >> > > > > > > > >>> > > > instance.
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > > > However for normal Link<?> and ajax links etc I
> >> get
> >> > > > > > > > >>> > > > ComponentNotFoundException thrown as the new
> >> > instance
> >> > > > > cannot
> >> > > > > > > > find
> >> > > > > > > > >>> it
> >> > > > > > > > >>> > in
> >> > > > > > > > >>> > > > the session it seems.
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > > > Clearly I'm not understanding how wicket manages
> >> the
> >> > > page
> >> > > > > > state
> >> > > > > > > > or
> >> > > > > > > > >>> I've
> >> > > > > > > > >>> > > > configured something wrong.
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > > > Does wicket support full session failover ?
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > > > Many thanks
> >> > > > > > > > >>> > > >
> >> > > > > > > > >>> > >
> >> > > > > > > > >>> >
> >> > > > > > > > >>>
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
