Re: [h2] MvStore cache leak

2020-11-01 Thread pwagland
Regarding b), would using a `long` be an option instead of `int`? long has 
a lot less overhead than `BigInteger`, and should handle even the most 
crazy cache sizes that people will want to throw at the code!
On Monday, October 26, 2020 at 10:17:04 AM UTC+1 Noel Grandin wrote:

> We should either
>
> (a) throw an exception if the cache size is so big we can't handle it
>
> (b) use BigInteger or store the cache sizes in kilobytes or megabytes to 
> avoid overflow.
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/4f4d9e1e-4f01-45a4-a2f1-dd7293cd475en%40googlegroups.com.


Re: [h2] MvStore cache leak

2020-10-26 Thread Noel Grandin

We should either

(a) throw an exception if the cache size is so big we can't handle it

(b) use BigInteger or store the cache sizes in kilobytes or megabytes to avoid 
overflow.

--
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/00da5c81-1d63-8359-1a3b-598b0b2a4db8%40gmail.com.


Re: [h2] MvStore cache leak

2020-10-26 Thread 'Mark McKeown' via H2 Database
Thanks Noel, Andrei,
The cache is split into 16 Segments each segment tracks memory used
and Max cache size using longs.

Looking at the code the overflow would happen if the size of a page or  an
Value is over 2GB, the memory used is stored as an in  - or in the case of
a page
if two Values of 1GB are added to a page, you cannot split a page which has
1GB and the second value is added before checking the size which could
cause the
overflow.

I suspect someone has added very large values which has triggered this.

cheers
Mark

case PUT: {
value = decisionMaker.selectValue(result, value);
p = p.copy();
if (index < 0) {
p.insertLeaf(-index - 1, key, value);
int keyCount;
while ((keyCount = p.getKeyCount()) > store.getKeysPerPage()
|| p.getMemory() > store.getMaxPageSize()
&& keyCount > (p.isLeaf() ? 1 : 2)) {






On Sat, Oct 24, 2020 at 3:18 AM Andrei Tokar  wrote:

> I guess, you next step could be to reduce configured cache size to some
> reasonable number.
> What makes you believe that you actually need such huge cache?
> If you have configured MVStore cache size to be 4Gb, how can you expect
> usedMemory field (which is java int) to always stay positive?
>
> On Friday, October 23, 2020 at 2:15:44 PM UTC-4 Noel Grandin wrote:
>
>> That looks like an integer overflow somewhere.
>>
>> I would tweak the source code in the places that modify that field, to
>> dump stack traces, and work backwards.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "H2 Database" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to h2-database+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/h2-database/eaf4273c-c2d7-41dc-b77b-5e76ff9c2aafn%40googlegroups.com
> 
> .
>


-- 
*MARK MC KEOWN ARCHITECT*

*E* mark.mcke...@wandisco.com

-- 


 

THIS MESSAGE AND ANY ATTACHMENTS 
ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED

If this message was 
misdirected, WANdisco, Inc. and its subsidiaries, ("WANdisco") does not 
waive any confidentiality or privilege. If you are not the intended 
recipient, please notify us immediately and destroy the message without 
disclosing its contents to anyone. Any distribution, use or copying of this 
email or the information it contains by other than an intended recipient is 
unauthorized. The views and opinions expressed in this email message are 
the author's own and may not reflect the views and opinions of WANdisco, 
unless the author is authorized by WANdisco to express such views or 
opinions on its behalf. All email sent to or from this address is subject 
to electronic storage and review by WANdisco. Although WANdisco operates 
anti-virus programs, it does not accept responsibility for any damage 
whatsoever caused by viruses being passed.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CALK6y0VA%2BdZhh2uktY9Mv7onhFxZTKAuzoHeAfHg3e0bRBn6bg%40mail.gmail.com.


Re: [h2] MvStore cache leak

2020-10-23 Thread Andrei Tokar
I guess, you next step could be to reduce configured cache size to some 
reasonable number.
What makes you believe that you actually need such huge cache?
If you have configured MVStore cache size to be 4Gb, how can you expect 
usedMemory field (which is java int) to always stay positive?

On Friday, October 23, 2020 at 2:15:44 PM UTC-4 Noel Grandin wrote:

> That looks like an integer overflow somewhere. 
>
> I would tweak the source code in the places that modify that field, to 
> dump stack traces, and work backwards.
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/eaf4273c-c2d7-41dc-b77b-5e76ff9c2aafn%40googlegroups.com.


Re: [h2] MvStore cache leak

2020-10-23 Thread Noel Grandin
That looks like an integer overflow somewhere.

I would tweak the source code in the places that modify that field, to dump
stack traces, and work backwards.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CAFYHVnUx8kDfN99kBjEJdH_aLDicMKX4wHmR5C8D6eQEwtSWUg%40mail.gmail.com.