HansMarcus01 opened a new pull request, #39728:
URL: https://github.com/apache/beam/pull/39728
## Summary of Changes
This pull request switches the execution mode of the Pub/Sub subscription
cleaner for NYC taxi rides from simulation to active deletion by changing
`dry_run=True` to `dry_run=False` inside:
`stale_cleaner.py` ➔ `clean_pubsub_subscriptions()`
## Why This Change is Safe (7-Day Empirical Validation)
Prior to enabling active deletion, we ran the subscription cleaner in
`dry_run=True` (Simulation Mode) for 7 days (from August 4, 2026, to August 11,
2026).
We monitored the state of the tracked resources by periodically downloading
and analyzing the JSON audit state stored in GCS:
`gs://apache-beam-testing-pabloem/stale_cleaner/pubsub_subscription.json`
### Key Empirical Findings:
- **100% Resource Abandonment**: Multiple subscriptions under the pattern
`taxirides-realtime_beam_-...` created on August 4 remained active, completely
unconsumed, and were continuously updated every single day until August 11
without being acknowledged.
- **Daily Accumulation Pattern**: Between 4 and 5 new orphaned subscriptions
were added daily and remained in the backlog.
### JSON State Evidence (August 11 Audit):
```json
{
"projects/apache-beam-testing/subscriptions/taxirides-realtime_beam_-6413074580631200911":
{
"resource_name":
"projects/apache-beam-testing/subscriptions/taxirides-realtime_beam_-6413074580631200911",
"creation_date": "2026-08-04T00:53:24.441535",
"last_update_date": "2026-08-11T12:24:57.889076"
},
"projects/apache-beam-testing/subscriptions/taxirides-realtime_beam_3858484106805878348":
{
"resource_name":
"projects/apache-beam-testing/subscriptions/taxirides-realtime_beam_3858484106805878348",
"creation_date": "2026-08-04T00:53:24.441562",
"last_update_date": "2026-08-11T12:24:57.889077"
}
}
```
## Verification & Testing
- Unit test have been run successfully
- No other prefixes are impacted; the active deletion only targets the
explicit NYC taxi prefix `taxirides-realtime_beam_.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
