Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Arjun Sahoo]

The problem still exists in 2.1.214 and with LOB_TIMEOUT=1000. The issue
could not be reproduced with the  current git build, but we can't use any
build other than official release. May I know when the next official
release is?.


On Tue, May 23, 2023 at 5:41 AM Andrei Tokar  wrote:

> And to confirm that problem is still there, please run it against at least
> latest released version (2.1.214), or even better against current git build.
> I have that feeling that you just run queries with LOB in the result which
> causes extensive LOB cloning. Try to add ;LOB_TIMEOUT=1000 (1 sec instead
> of default 5 min)
> .
> On Monday, May 22, 2023 at 2:45:23 AM UTC-4 Andreas Reichel wrote:
>
>> Greetings!
>>
>> On Sun, 2023-05-21 at 23:42 -0700, Arjun Sahoo wrote:
>>
>> Reproducing the test case is easy, keep on ingesting and deleting
>> records, you will see the memory usage increasing over time. If if query
>> the table simultaneously , you will reach the trend faster.
>>
>>
>>
>> You will need to write a self containing unit test simulating this, Only
>> this way people will help ypu.
>>
>> Best regards
>> Andreas
>>
> --
> You received this message because you are subscribed to the Google Groups
> "H2 Database" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to h2-database+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/h2-database/84e85db2-fc01-4e92-a4d2-6e162ec3a30bn%40googlegroups.com
> 
> .
>


-- 
With Regards,
 Arjun Sahoo

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CANjTRCT2%2BV7QbohC9CKGkxce3zGJaSr1k%3Dyvk7YtSWchTv5QEg%40mail.gmail.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Andrei Tokar]

And to confirm that problem is still there, please run it against at least 
latest released version (2.1.214), or even better against current git build.
I have that feeling that you just run queries with LOB in the result which 
causes extensive LOB cloning. Try to add ;LOB_TIMEOUT=1000 (1 sec instead 
of default 5 min)
.
On Monday, May 22, 2023 at 2:45:23 AM UTC-4 Andreas Reichel wrote:

> Greetings!
>
> On Sun, 2023-05-21 at 23:42 -0700, Arjun Sahoo wrote:
>
> Reproducing the test case is easy, keep on ingesting and deleting records, 
> you will see the memory usage increasing over time. If if query the table 
> simultaneously , you will reach the trend faster.
>
>
>
> You will need to write a self containing unit test simulating this, Only 
> this way people will help ypu.
>
> Best regards
> Andreas
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/84e85db2-fc01-4e92-a4d2-6e162ec3a30bn%40googlegroups.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Andreas Reichel]

Greetings!

On Sun, 2023-05-21 at 23:42 -0700, Arjun Sahoo wrote:
> Reproducing the test case is easy, keep on ingesting and deleting
> records, you will see the memory usage increasing over time. If if
> query the table simultaneously , you will reach the trend faster.


You will need to write a self containing unit test simulating this,
Only this way people will help ypu.

Best regards
Andreas

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/cb335290abd9d8e0c5d10d9c0180c6c74d62a89d.camel%40manticore-projects.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Noel Grandin]

(1) That is not expected behaviour, my H2 databases grow to a point and
then stabilise (and I run my H2 databases pretty hard)

(2) in-memory is slightly different, we have a couple of places where we do
things different because we know it's a memory database

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CAFYHVnWxhGSnbE36c5ySxua2M%2BbXT-6LzUS8HHyq1O4Z%2BQsTLw%40mail.gmail.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Andreas Reichel]

Noel,

question for better understanding please: Would a MEM database not grow
exactly as a File backed database keeps growing forever?
Or in other words, is there any difference between a MEMORY database
and File backed at TEMP FS?

I ask because all my file backed H2 databases keep growing and never
release all filespace until I export to file, create new and compress.
So I would expect the same (mis-) behavior for any memory DB.

Setting up a reliable test should be easy: Create a DB, populate with
an indexed table and data and run queries continuously for a longer
time. Check if size peaks or grows forever.

Cheers
Andreas


-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/725d35848838abedb93e986fe512b580d4415a7b.camel%40manticore-projects.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Noel Grandin]

I don't have any obvious answers.

You can try 2 things.

First, try building H2 from git master and seeing if the changes since the
last official release help.

If that doesn't help, we're going to need a standalone test case (no
Spring, etc) to reproduce the problem and then we might be able to fix it.

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CAFYHVnWR-m8vzvi9LE7XBf59CJRLGz_x_rm2CrauUL%3DchW3O1g%40mail.gmail.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Arjun Sahoo]

I am using spring data jpa to delete the records.
@Modifying
@Transactional
@Query(value = "delete from table where start_time_ms <= :startTimeMs", 
nativeQuery = true)
Integer deleteInBulkByStartTime(@Param("startTimeMs") Long startTimeMs) throws 
SQLException;

 Spring takes care of  managing the transaction.


On Sunday, May 21, 2023 at 1:09:55 AM UTC+5:30 Noel Grandin wrote:

> check that all your transactions are being closed. Holding a transaction 
> open means that the storage cannot be freed.
>
>  
>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/7c3dc13a-1ccc-4333-932e-4ac29a4d15e9n%40googlegroups.com.

Re: [h2] In-Memory H2 deleted rows not garbage collected.

[Noel Grandin]

check that all your transactions are being closed. Holding a transaction
open means that the storage cannot be freed.



>
>

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/CAFYHVnXhZosk3b1LEK-b_6%2BTFFePhQ1%3DbkWRnvb5Auhx4OoEZA%40mail.gmail.com.

[h2] In-Memory H2 deleted rows not garbage collected.

[Arjun Sahoo]

Hi,

 I am using Spring boot In-Memory H2DB. I am storing latest 100k records in 
the database, so keep on deleting old records and inserting new records. 
Application runs fine for 4-5 hours, after that it throws OOM Exception. 
The memory consumption keeps on increasing even after deleting old records. 
I took the thread dump and analysed in MAT, it shows one memory leak.

One instance of org.h2.mvstore.tx.TransactionStore$TxMapBuilder$TMVMap 
loaded by org.springframework.boot.loader.LaunchedURLClassLoader @ 
0x6c0737918 occupies 2,08,58,61,856 (77.95%) bytes. The memory is 
accumulated in one instance of org.h2.mvstore.Page$PageReference[], loaded 
by org.springframework.boot.loader.LaunchedURLClassLoader @ 0x6c0737918, 
which occupies 2,08,58,61,384 (77.95%) bytes.

When I restart the application with same number of records in H2db, it 
takes only 30% of memory.

*H2db Version*: 2.1.212

*Spring boot*: 2.7.1

*Spring data JPA*:2.7.1

*spring.datasource.url*
=jdbc:h2:mem:db;DB_CLOSE_ON_EXIT=FALSE;DATABASE_TO_LOWER=TRUE;CASE_INSENSITIVE_IDENTIFIERS=TRUE;

-- 
You received this message because you are subscribed to the Google Groups "H2 
Database" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to h2-database+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/h2-database/ff956fd3-e598-4895-9060-b6c925351cbfn%40googlegroups.com.