GitHub user PPisz added a comment to the discussion: database consitency
@DaanHoogland Ok, based on these queries: https://github.com/apache/cloudstack/blob/15c2e50338a514cb51e6ef3443261a7d3971f69c/usage/src/main/java/com/cloud/usage/UsageSanityChecker.java#L41 We managed to track down one VM that was deleted in January and events are logged for it every day at 11:59 PM: select * from usage_event where resource_id = 1790 and type like 'VM.%'; +-------+------------+------------+---------------------+---------+-------------+-----------------------+-------------+-------------+------+---------------+-----------+--------------+ | id | type | account_id | created | zone_id | resource_id | resource_name | offering_id | template_id | size | resource_type | processed | virtual_size | +-------+------------+------------+---------------------+---------+-------------+-----------------------+-------------+-------------+------+---------------+-----------+--------------+ | 62870 | VM.CREATE | 83 | 2025-01-21 20:56:09 | 1 | 1790 | meritus-dev-control-2 | 71 | 316 | NULL | KVM | 0 | NULL | | 62897 | VM.DESTROY | 83 | 2025-01-21 20:56:10 | 1 | 1790 | meritus-dev-control-2 | 71 | 316 | NULL | KVM | 0 | NULL | +-------+------------+------------+---------------------+---------+-------------+-----------------------+-------------+-------------+------+---------------+-----------+--------------+ 2 rows in set (0.574 sec) Events: select vm.id, vm.name, vm.removed, cu.* from -> cloud_usage.cloud_usage cu -> inner join cloud.vm_instance vm -> where vm.type = 'User' -> and cu.usage_type in (1 , 2) -> and cu.usage_id = vm.id and cu.start_date > vm.removed; +------+-----------------------+---------------------+---------+---------+------------+-----------+-----------------------------------------------------------------------+---------------+------------+--------------------+----------------+-----------------------+-------------+-------------+----------+------+------+------------+---------------------+---------------------+--------------+-----------+-----------+--------+------------------+-----------+-------+ | id | name | removed | id | zone_id | account_id | domain_id | description | usage_display | usage_type | raw_usage | vm_instance_id | vm_name | offering_id | template_id | usage_id | type | size | network_id | start_date | end_date | virtual_size | cpu_speed | cpu_cores | memory | quota_calculated | is_hidden | state | +------+-----------------------+---------------------+---------+---------+------------+-----------+-----------------------------------------------------------------------+---------------+------------+--------------------+----------------+-----------------------+-------------+-------------+----------+------+------+------------+---------------------+---------------------+--------------+-----------+-----------+--------+------------------+-----------+-------+ | 1790 | meritus-dev-control-2 | 2025-01-22 14:17:24 | 7578010 | 1 | 83 | 20 | meritus-dev-control-2 allocated (ServiceOffering: 71) (Template: 316) | 24 Hrs | 2 | 24 | 1790 | meritus-dev-control-2 | 71 | 316 | 1790 | KVM | NULL | NULL | 2025-10-27 00:00:00 | 2025-10-27 23:59:59 | NULL | NULL | 4 | 8192 | 0 | 0 | NULL | | 1790 | meritus-dev-control-2 | 2025-01-22 14:17:24 | 7836640 | 1 | 83 | 20 | meritus-dev-control-2 allocated (ServiceOffering: 71) (Template: 316) | 24 Hrs | 2 | 24 | 1790 | meritus-dev-control-2 | 71 | 316 | 1790 | KVM | NULL | NULL | 2025-10-27 00:00:00 | 2025-10-27 23:59:59 | NULL | NULL | 4 | 8192 | 0 | 0 | NULL | | 1790 | meritus-dev-control-2 | 2025-01-22 14:17:24 | 7842127 | 1 | 83 | 20 | meritus-dev-control-2 allocated (ServiceOffering: 71) (Template: 316) | 24 Hrs | 2 | 24 | 1790 | meritus-dev-control-2 | 71 | 316 | 1790 | KVM | NULL | NULL | 2025-10-28 00:00:00 | 2025-10-28 23:59:59 | NULL | NULL | 4 | 8192 | 0 | 0 | NULL | | 1790 | meritus-dev-control-2 | 2025-01-22 14:17:24 | 8100760 | 1 | 83 | 20 | meritus-dev-control-2 allocated (ServiceOffering: 71) (Template: 316) | 24 Hrs | 2 | 24 | 1790 | meritus-dev-control-2 | 71 | 316 | 1790 | KVM | NULL | NULL | 2025-10-27 00:00:00 | 2025-10-27 23:59:59 | NULL | NULL | 4 | 8192 | 0 | 0 | NULL | | 1790 | meritus-dev-control-2 | 2025-01-22 14:17:24 | 8106247 | 1 | 83 | 20 | meritus-dev-control-2 allocated (ServiceOffering: 71) (Template: 316) | 24 Hrs | 2 | 24 | 1790 | meritus-dev-control-2 | 71 | 316 | 1790 | KVM | NULL | NULL | 2025-10-28 00:00:00 | 2025-10-28 23:59:59 | NULL | NULL | 4 | 8192 | 0 | 0 | NULL | | We've checked several resources and they appear to have valid deletion dates. We're continuing our investigation, but any suggestions on what might be generating these events? GitHub link: https://github.com/apache/cloudstack/discussions/11958#discussioncomment-14881695 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
