Re: Equivalent for PerSessionPageStore in Wicket 9

2020-04-11 Thread Sven Meier

Hi Thomas,

actually not bad news at all (for Wicket 9 at least).

The old page manager implementation had so many special concepts and 
solutions, it's easy to miss one.


A soft reference feature can easily be added/restored. I'm already 
checking where it fits best.


Thanks for your thorough testing.

Best regards
Sven


On 11.04.20 10:58, Thomas Heigl wrote:

Hi all,

Bad news. My application was caught in a GC loop after running for 8 hours.
The old generation was exhausted.

I couldn't get a heap dump at that time but restarted the application, took
a heap dump after about an hour, and reverted back to Wicket 8.

The problem is this: The heap was full of objects referencing
InMemoryPageStore, i.e. the in-memory 2nd-level cache for pages. My first
thought was that there is something wrong with the implementation of that
store and pages do not get limited or removed correctly.  So I debugged
it locally but everything is working fine.

Then I noticed that there are a lot of instances of Hibernate entities on
the heap. So there definitely is an issue with models somewhere in my
application. To be sure that this is not a new issue, I took another heap
dump from production - now running Wicket 8 again - and compared it.
There are undetached entity models on the heap as well.

So why does it not OOM with Wicket 8? Well, the PerSessionPageStore
(roughly the equivalent of InMemoryPageStore in Wicket 9) uses
SoftReferences for storing pages. InMemoryPageStore does not and GC cannot
reclaim memory from it.

So while this surfaced some issues in my application that I just fixed, I
believe that InMemoryPageStore should use SoftReferences or another
implementation based on SoftReferences should be added to Wicket 9. A cache
should not consume all the memory if it can easily re-fetch
the value from persistent storage.

I guess the reason for not using SoftReferences in InMemoryPageStore is
that it can theoretically be used as a "persistent" store for pages. If that
behavior is really required, I suggest adding another implementation using
SoftReferences.

Best regards,

Thomas

On Fri, Apr 10, 2020 at 7:19 PM Martin Grigorov 
wrote:


On Fri, Apr 10, 2020 at 4:01 PM Thomas Heigl  wrote:


FYI: I deployed Wicket 9.0.0-M5 to production an hour ago. 100k requests
served and no issues so far.


Awesome!
Thank you for testing it!



Great work!

Thomas

On Wed, Apr 8, 2020 at 3:13 PM Sven Meier  wrote:


Many thanks Maxim!

Sven

On 08.04.20 14:29, Maxim Solodovnik wrote:

Released :)

On Wed, 8 Apr 2020 at 15:41, Maxim Solodovnik 

wrote:

OK

Will start new release process in couple of hours
Please stop me if you will find any blocker :)

On Wed, 8 Apr 2020 at 14:36, Thomas Heigl 

wrote:

Hi Maxim,

It works for me now!

Thomas

On Wed, Apr 8, 2020 at 9:17 AM Maxim Solodovnik <

solomax...@gmail.com>

wrote:


Thanks a million!

On Wed, 8 Apr 2020 at 14:10, Thomas Heigl 

wrote:

Hi Maxim,

I'm testing against the snapshot now. Will get back to you

shortly.

Thomas

On Wed, Apr 8, 2020 at 2:53 AM Maxim Solodovnik <

solomax...@gmail.com>

wrote:


Hello All,

M5 seems to be broken (deploy has failed more than 10 times

during

my

build attempts)
I have to start another release
Could you please tell when can I start?

On Wed, 8 Apr 2020 at 07:01, Maxim Solodovnik <

solomax...@gmail.com

wrote:

Hello Thomas,

Please test M6-SNAPSHOT (so I don't have to release M5.2 :

On Wed, 8 Apr 2020 at 02:39, Thomas Heigl 
wrote:

Hi Maxim,

That would be great. I want to do some more extensive testing

and

then

deploy M5 into production. ;)

Thomas

On Tue, Apr 7, 2020 at 7:50 PM Maxim Solodovnik <

solomax...@gmail.com>

wrote:


I can pack another release
later this week ...

On Wed, 8 Apr 2020 at 00:48, Thomas Heigl <

tho...@umschalt.com

wrote:

Thanks Sven!

Did your changes make it into the release? Or did they just

miss

it?

Thomas

On Tue, Apr 7, 2020 at 7:43 PM Sven Meier 

wrote:

Hi Thomas,

yes, you're right:

wicketstuff data stores missed some adjustments to the

latest

updates

in

wicket-core.

And SessionQuotaManagingDataStore$DelegatedPage must be

serializable of

course.

I've pushed changes to wicketstuff master.

Thanks
Sven


On 07.04.20 14:14, Thomas Heigl wrote:

And one more thing. There is now a warning logged just

before

serialization:

WARN o.a.w.pageStore.AsynchronousPageStore: Delegated

page

store

'org.apache.wicket.pageStore.SerializingPageStore' can

not be

asynchronous

On Tue, Apr 7, 2020 at 2:09 PM Thomas Heigl <

tho...@umschalt.com>

wrote:

The cause is the following MetaData entry in the session:

class

org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$1=org.wicketstuff.datastores.common.SessionQuotaManagingDataStore$SizeLimitedData@4090594a

On Tue, Apr 7, 2020 at 1:59 PM Thomas Heigl <

tho...@umschalt.com>

wrote:

Hi Sven,

I just found time to give this a try with Wicket

9.0.0-M5.

There

seem

to

be issues with serialization now.

My new 

Re: Equivalent for PerSessionPageStore in Wicket 9

2020-04-11 Thread Thomas Heigl
Hi all,

Bad news. My application was caught in a GC loop after running for 8 hours.
The old generation was exhausted.

I couldn't get a heap dump at that time but restarted the application, took
a heap dump after about an hour, and reverted back to Wicket 8.

The problem is this: The heap was full of objects referencing
InMemoryPageStore, i.e. the in-memory 2nd-level cache for pages. My first
thought was that there is something wrong with the implementation of that
store and pages do not get limited or removed correctly.  So I debugged
it locally but everything is working fine.

Then I noticed that there are a lot of instances of Hibernate entities on
the heap. So there definitely is an issue with models somewhere in my
application. To be sure that this is not a new issue, I took another heap
dump from production - now running Wicket 8 again - and compared it.
There are undetached entity models on the heap as well.

So why does it not OOM with Wicket 8? Well, the PerSessionPageStore
(roughly the equivalent of InMemoryPageStore in Wicket 9) uses
SoftReferences for storing pages. InMemoryPageStore does not and GC cannot
reclaim memory from it.

So while this surfaced some issues in my application that I just fixed, I
believe that InMemoryPageStore should use SoftReferences or another
implementation based on SoftReferences should be added to Wicket 9. A cache
should not consume all the memory if it can easily re-fetch
the value from persistent storage.

I guess the reason for not using SoftReferences in InMemoryPageStore is
that it can theoretically be used as a "persistent" store for pages. If that
behavior is really required, I suggest adding another implementation using
SoftReferences.

Best regards,

Thomas

On Fri, Apr 10, 2020 at 7:19 PM Martin Grigorov 
wrote:

> On Fri, Apr 10, 2020 at 4:01 PM Thomas Heigl  wrote:
>
> > FYI: I deployed Wicket 9.0.0-M5 to production an hour ago. 100k requests
> > served and no issues so far.
> >
>
> Awesome!
> Thank you for testing it!
>
>
> >
> > Great work!
> >
> > Thomas
> >
> > On Wed, Apr 8, 2020 at 3:13 PM Sven Meier  wrote:
> >
> > > Many thanks Maxim!
> > >
> > > Sven
> > >
> > > On 08.04.20 14:29, Maxim Solodovnik wrote:
> > > > Released :)
> > > >
> > > > On Wed, 8 Apr 2020 at 15:41, Maxim Solodovnik 
> > > wrote:
> > > >> OK
> > > >>
> > > >> Will start new release process in couple of hours
> > > >> Please stop me if you will find any blocker :)
> > > >>
> > > >> On Wed, 8 Apr 2020 at 14:36, Thomas Heigl 
> > wrote:
> > > >>> Hi Maxim,
> > > >>>
> > > >>> It works for me now!
> > > >>>
> > > >>> Thomas
> > > >>>
> > > >>> On Wed, Apr 8, 2020 at 9:17 AM Maxim Solodovnik <
> > solomax...@gmail.com>
> > > >>> wrote:
> > > >>>
> > >  Thanks a million!
> > > 
> > >  On Wed, 8 Apr 2020 at 14:10, Thomas Heigl 
> > > wrote:
> > > > Hi Maxim,
> > > >
> > > > I'm testing against the snapshot now. Will get back to you
> shortly.
> > > >
> > > > Thomas
> > > >
> > > > On Wed, Apr 8, 2020 at 2:53 AM Maxim Solodovnik <
> > > solomax...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hello All,
> > > >>
> > > >> M5 seems to be broken (deploy has failed more than 10 times
> during
> > > my
> > > >> build attempts)
> > > >> I have to start another release
> > > >> Could you please tell when can I start?
> > > >>
> > > >> On Wed, 8 Apr 2020 at 07:01, Maxim Solodovnik <
> > solomax...@gmail.com
> > > >
> > > >> wrote:
> > > >>> Hello Thomas,
> > > >>>
> > > >>> Please test M6-SNAPSHOT (so I don't have to release M5.2 :
> > > >>>
> > > >>> On Wed, 8 Apr 2020 at 02:39, Thomas Heigl  >
> > >  wrote:
> > >  Hi Maxim,
> > > 
> > >  That would be great. I want to do some more extensive testing
> > and
> > >  then
> > >  deploy M5 into production. ;)
> > > 
> > >  Thomas
> > > 
> > >  On Tue, Apr 7, 2020 at 7:50 PM Maxim Solodovnik <
> > >  solomax...@gmail.com>
> > >  wrote:
> > > 
> > > > I can pack another release
> > > > later this week ...
> > > >
> > > > On Wed, 8 Apr 2020 at 00:48, Thomas Heigl <
> tho...@umschalt.com
> > >
> > > >> wrote:
> > > >> Thanks Sven!
> > > >>
> > > >> Did your changes make it into the release? Or did they just
> > >  miss
> > > >> it?
> > > >> Thomas
> > > >>
> > > >> On Tue, Apr 7, 2020 at 7:43 PM Sven Meier 
> > >  wrote:
> > > >>> Hi Thomas,
> > > >>>
> > > >>> yes, you're right:
> > > >>>
> > > >>> wicketstuff data stores missed some adjustments to the
> latest
> > > >> updates
> > > > in
> > > >>> wicket-core.
> > > >>>
> > > >>> And SessionQuotaManagingDataStore$DelegatedPage must be
> > > >> serializable of
> > > >>> course.
> > > >>>
> > > >>> I've