Hi, we are/were facing an issue where we were running into an OOM error in Sentry. This happened right after startup and it is because of SENTRY-2461[1].
We're trying to understand how this is supposed to work. We see HmsFollower waking up every 500ms. It looks at the current notification Id we have stored in SENTRY_HMS_NOTIFICATION_ID (e.g. 100). It then subtracts 1 (e.g. 99) and retrieves any notifications newer than that (e.g. "100"). HmsFollower then calls NotificationProcessor#processNotificationEvent to process the event. If that method returns false we explicitly persist the notification Id There's a few things that we find weird about this: * We see hundreds of duplicates being processed because we never check if we already have processed the same notification id. Now I do see comments in the code that Hive reuses those ids so this might be intentional? * But we also store these duplicate notification ids over and over. In the normal scenario where the id hasn't changed in the last 500ms we just store it again and because the table does not have a constraint on duplicates it stores tens of thousands of duplicates for us at least until the purging process kicks in 12h later I guess our question is: Is this normal behavior? Should we see thousands of duplicates and should Sentry reprocess the last notification over and over? Thank you very much! Cheers, Lars [1] <https://issues.apache.org/jira/browse/SENTRY-2461>
