winterhazel commented on issue #10687: URL: https://github.com/apache/cloudstack/issues/10687#issuecomment-2810671645
> Older CREATE events shouldn’ t be removed but the newer ones should. The network was actually “ created” on the first create. It may have to be “implemented” multiple times after that , which also lead to CREATE events, but those are not the creation. removing older events instead of the newer ones will lead to missed usage data @DaanHoogland my changes are not removing CREATE events. Modifying existing records in `usage_event` is not an option because it will lead to more problems. I'll try to explain things from the beginning, please tell me if you still see an issue in this: The way Usage works is around three things: events (from the `usage_event` table), helper entries (different tables for different resources `usage_networks`), and the parsers. - **Events** are used to create helper entries - **Helper entries** indicate a resource's state at a period in time, and are used to generate usage records - **Parsers** create usage records (at `cloud_usage`) based on the helper entries Let's use the volume event model as an example. a. Suppose a period between dates A (last time Usage executed) and D (next execution): 1. The user creates a volume at B (between A and D). A CREATE event is saved to `usage_event` 2. The user, then, resizes the volume at a date C (between B and D). A RESIZE event is saved to `usage_event` b. Then, the Usage job runs at D: 1. First, Usage will look through all events created since the last time the job executed and process them 1.1. It will see that a CREATE event for a volume was issued, so it will create a helper entry in `usage_volume` to tell the parser that it should generate usage records for the period between the entry's `created` field (B) until the `deleted` field (currently null, that is, until the current moment D) 1.2. It will see that a RESIZE event for a volume was issued. To identify that there was a change in the volume, it creates another entry in `usage_volume` with `created` set to C, and marks the previous one as removed at date C 2. After this, the "parsing" process will execute: 2.1. The volume parser will check `usage_volume` for entries that apply to the period between A and D 2.1.1. It will see that there is one entry that starts at B and ends at C, so it will generate an usage record that contains how much time there is between B and C, and information about the volume (before the resize) that are in the helper entry 2.1.2. It will see that there is one entry that starts at C and does not have an end date, so it will generate an usage record that contains how much time there is between C and D, and information about the volume (after the resize) that are in the helper entry My changes implement b.1.2 for the network event processing. That is, Usage will mark previous helper entries (**not events!**) as removed when it sees a CREATE event for network, but there are already existing helper entries. This does not lead to missing usage data. The usage records will be generated as intended, as exemplified in this comment, and events remain unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
