andreinnanu opened a new pull request, #4488: URL: https://github.com/apache/iggy/pull/4488
## Which issue does this PR address? Relates to #4140 This PR alone doesn't solve the issue. A following one will. ## Rationale Enables accounting disk usage, preparing the ground for the configurable disk usage limit discussed within #4140. As agreed there, this is the first of two PRs: accounting here, the limit and the refusal of new messages in the follow-up. ## What changed? Iggy had no view of how full its data volume is, so the disk usage limit planned in #4140 had nothing to check against. A periodic `statvfs` sample alone is not enough: a burst can fill the remaining space between two samples. A process-wide estimate (`DISK_USAGE`) of the bytes used on the data volume was added. The write sites charge what they allocate, and the existing `iggy-system-stats` sampler seeds the estimate at boot and re-bases it from `statvfs`, so writes on the data volume which are not under Iggy's control are seen with every sample. Preallocation charges only newly allocated space: re-reserving an existing segment at recovery does not change the estimate. The estimate is exported as the `disk_used_bytes` gauge on `/metrics`. Accounting only: nothing is refused yet. ### Open question based on #4140 discussion How do we want to approach freezing the sampler? The goal was to add a burst test with the sampler frozen, but currently there is no mechanism for that. Reusing `sysinfo_print_interval` doesn't work because its minimum value is 1s. ## Local Execution - Passed: /metrics includes the `disk_used_bytes` gauge and its value and follows `df` while a producer writes messages - Pre-commit hooks pass <!-- You must run your code locally before submitting. "Relying on CI" is not acceptable - PRs from authors who haven't run the code will be closed. Did you have `prek` installed? It runs automatically on commit and covers all project languages. See [CONTRIBUTING.md](https://github.com/apache/iggy/blob/master/CONTRIBUTING.md). --> ## AI Usage - used Claude Code for generating tests & some of the functions - verified by manual tests & code review <!-- If AI tools were used, please answer: 1. Which tools? (e.g., GitHub Copilot, Claude, ChatGPT) 2. Scope of usage? (e.g., autocomplete, generated functions, entire implementation) 3. How did you verify the generated code works correctly? 4. Can you explain every line of the code if asked? If no AI tools were used, write "None" or delete this section. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
