[ 
https://issues.apache.org/jira/browse/BEAM-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434292#comment-16434292
 ] 

yifan zou commented on BEAM-3908:
---------------------------------

[~alanmyrvold] It was fixed. All stale jobs were canceled. I'll close this bug.

> Leaderboard / gamestats leaking Dataflow Jobs
> ---------------------------------------------
>
>                 Key: BEAM-3908
>                 URL: https://issues.apache.org/jira/browse/BEAM-3908
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow, testing
>    Affects Versions: Not applicable
>            Reporter: Luke Cwik
>            Assignee: yifan zou
>            Priority: Critical
>             Fix For: 2.5.0
>
>
> I found that the leaderboard/gamestats Dataflow streaming jobs weren't being 
> cleaned up by the test infrastructure which lead to quota issues because all 
> the VMs/disks/memory being consumed causing other Jenkins jobs to fail.
> I manually stopped all the jobs that had been running for more then 12 hrs. 
> There were about 20 jobs like this.
> Example links:
> https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-06_22_19_25-7861256924404398606?project=apache-beam-testing&organizationId=433637338589
> https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-06_18_49_54-7185486205606862436?project=apache-beam-testing&organizationId=433637338589
> https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-06_23_14_26-6599347078760080693?project=apache-beam-testing&organizationId=433637338589
> https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-03-06_18_32_33-7276493109541122240?project=apache-beam-testing&organizationId=433637338589



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to