[
https://issues.apache.org/jira/browse/BEAM-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kiley Sok updated BEAM-12903:
-----------------------------
Description:
FR to create a solution to clean up lingering GCE, VMs, Dataproc clusters, etc.
Some sort of perodic job which runs and deletes leaked GCE VMs would prevent
the resource concerns caused by failed tests as in:
https://issues.apache.org/jira/browse/BEAM-12898
Possible solutions:
* Name all the resources with a convention that represents a TTL. Then the
resources can be deleted by a cron/periodic job if the TTL has expired
* Dataproc cluster may have a TTL configuration option itself which the flink
on dataproc test can use.
* Similar to how we delete old Dataflow jobs in
https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
was:
FR to create a solution to clean up lingering GCE, VMs, Dataproc clusters, etc.
Some sort of perodic job which runs and deletes leaked GCE VMs would prevent
the resource concerns caused by failed tests as in:
https://issues.apache.org/jira/browse/BEAM-12898?filter=-2
Possible solutions:
* Name all the resources with a convention that represents a TTL. Then the
resources can be deleted by a cron/periodic job if the TTL has expired
* Dataproc cluster may have a TTL configuration option itself which the flink
on dataproc test can use.
> Implement a solution to cleanup GCP resources in beam testing projects
> ----------------------------------------------------------------------
>
> Key: BEAM-12903
> URL: https://issues.apache.org/jira/browse/BEAM-12903
> Project: Beam
> Issue Type: Test
> Components: testing
> Reporter: Alex Amato
> Priority: P3
>
> FR to create a solution to clean up lingering GCE, VMs, Dataproc clusters,
> etc.
> Some sort of perodic job which runs and deletes leaked GCE VMs would prevent
> the resource concerns caused by failed tests as in:
> https://issues.apache.org/jira/browse/BEAM-12898
> Possible solutions:
> * Name all the resources with a convention that represents a TTL. Then the
> resources can be deleted by a cron/periodic job if the TTL has expired
> * Dataproc cluster may have a TTL configuration option itself which the
> flink on dataproc test can use.
> * Similar to how we delete old Dataflow jobs in
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_CancelStaleDataflowJobs.groovy
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
