[
https://issues.apache.org/jira/browse/ARTEMIS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558246#comment-17558246
]
Fabio Nascimento Brandão commented on ARTEMIS-3868:
---------------------------------------------------
Hi [~clebertsuconic] , the test is exactly with only live records! This is why
the records weren't removed. The problem here is that in the first compact,
2.1MB grows to 10GB! Then in the second compact it shrink to 20MB (that's ok
for me, I don't expect to go to 2.1MB again, the problem is this jump to 10GB).
The main problem (that was showed in the POC that I've sent) is exactly that
the split create a lot of files.
I'm using paging.
I'll try to use the 2.23 and simulate the problem.
I'm not asking questions, I debugged the code and reproduced the problem with
the attached code. I'm facing this issue every week in a production server
(like other problem that I opened a issue here and was merged).
So, thanks for your time. I'll try the latest version and if I confirm the
problem in it, I'll report here.
Thanks again
> Compact process filling all the disk
> ------------------------------------
>
> Key: ARTEMIS-3868
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3868
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: ActiveMQ-Artemis-Native
> Affects Versions: 2.19.0, 2.20.0, 2.21.0, 2.22.0, 2.23.0, 2.23.1
> Reporter: Fabio Nascimento Brandão
> Assignee: Clebert Suconic
> Priority: Major
> Attachments: CompactDiskUsage.java, image-2022-06-21-19-33-59-276.png
>
>
> We are having some problems with disk space usage. Basically we have a lot of
> queues and one of them fail to consume the messages (problem in a
> microservice for about 10 minutes).
> Then we use the message redelivery feature to process these messages again
> after some delay.
> The problems ocurrs with de journal compact. This process creates a lot of
> files until fill the disk.
> I took some created files at production server and tried to compact them. The
> disk usage grow a lot and only after the second compact process the disk
> usage stayed at an acceptable level.
> I attached a "test" to reproduce the problem. The test create one journal
> file with 2048 records and using only 2,1MB. The records have an information
> that is the "compact count". I adjusted this field to 1 and 2.
> What I found out is when the compactor process a record with this field less
> than 2 (after some record being greater or equals to 2), the JournalCompactor
> creates a new file.
> In this test, when I start the journal it creates a new file of 10MB (default
> configuration) and keeps the 2,1MB file.
> When I ran the compact process, It creates 1024 files and keeps the old 2.
> 1024 * 10MB = 10GB!!!
> If I ran the compact process again, It shrink to only 2 files of 10MB!!!
> The problem seems to be in the JournalCompactor at method checkCompact. I
> think we can remove this method and assume the flow like it returned false:
> [https://github.com/apache/activemq-artemis/blob/main/artemis-journal/src/main/java/org/apache/activemq/artemis/core/journal/impl/JournalCompactor.java#L190-L203]
> This was an old code that was introduced at HornetQ:
> [https://github.com/hornetq/hornetq/commit/93af1cb92f4050e54e83c8daa9c67ce43dbcfead]
> I also attached an image with the disk usage of my production server with the
> problem.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
