lukaszzborek opened a new issue, #2924:
URL: https://github.com/apache/iggy/issues/2924
### Bug description
I've observed the iggy server's memory usage constantly growing.
It's not a huge increase, only 10-15 MB for every 2-3 million messages.
I checked it on multiple last edge builds
When the Iggy server hits its memory limit, it uses almost 100% of all CPUs.
Usually, I can't log in to the server to restart the service, so I have to do
it via the Proxmox UI.
After restarting, memory usage returns to normal. However, two things
started happening:
1. cleaners did not respect the message expiration. Segments were deleted
just after the segment rotation (the first run of the next cleaner).
2. consumers (pull next offset) are beginning to lose their offsets.
Randomly skip random numbers of messages
I have 10 message expiry, with 128mb segment size
Offset lose is only on topic which was on during cpu drain. On new topics
created after restart, everythink looks good.
Also i forgot to check memory usaged in system. I only see some chart in
Proxmox UI and logs from iggy stats.
I can send local_data, which is saved after a server restart or CPU drain. I
cannot attach it here because it's over 100 MB. You can write messages to the
topic, and then try to pull the next one. It should skip messages sometimes.
Feel free to ask if you have any questions, or want additional data or
something 😅
### Deployment
Pre-built binary / release artifact
### Versions
0.7.2-edge.1, hash: c15100698874c38f88f36a4bb62ba205c3e53db2, built at:
2026-03-10T21:13:24.620646481Z
### Hardware / environment
proxmox, lxc, linux
### Sample code
nuget verions: 0.7.0
```csharp
var iggy = IggyClientFactory.CreateClient(new IggyClientConfigurator()
{
BaseAddress = "localhost:8090",
Protocol = Protocol.Tcp,
AutoLoginSettings = new AutoLoginSettings()
{
Enabled = true,
Username = "otherapp",
Password = "xxxxxxx"
}
});
await iggy.ConnectAsync();
var stream = Identifier.String("main");
var topic = Identifier.String("orderbook-topic");
var cg = "cg-orderbook";
await iggy.JoinConsumerGroupAsync(stream, topic, Identifier.String(cg));
var offset = await iggy.GetOffsetAsync(Consumer.Group(cg), stream, topic, 0);
ulong lastOffset = 0L;
var count = 10u;
while (true)
{
var messages = await iggy.PollMessagesAsync(stream, topic, 0,
Consumer.Group(cg), PollingStrategy.Next(), count, true);
if (messages.Messages.Count == 0)
{
await Task.Delay(100);
continue;
}
foreach (var message in messages.Messages)
{
if (message.Header.Offset - lastOffset != 1)
{
Console.WriteLine(
$"Last offset: {lastOffset}, Current offset:
{message.Header.Offset}, Difference: {message.Header.Offset - lastOffset}");
}
lastOffset = message.Header.Offset;
//await Task.Delay(11);
}
}
```
### Logs
_No response_
### Iggy server config
[current_config.txt](https://github.com/user-attachments/files/25946410/current_config.txt)
### Reproduction
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
