Thanks Kasia. Below are the most frequent flakes I have encountered in postcommit failures, but there may be more. Let's track resolution in respective Jira's and avoid re-creating duplicates.
https://issues.apache.org/jira/browse/BEAM-7527 - ModuleNotFoundError (No module named 'beam_runner_api_pb2') in Python 3 test suites that run in parallel. https://issues.apache.org/jira/browse/BEAM-5874 - BigQueryQueryToTableIT is flaky on direct runner, incomplete assertion: expected checksum is e1fbcb5ca479a5ca5f9ecf444d6998beee4d44c6) but: [some console output] https://issues.apache.org/jira/browse/BEAM-7749 - Occasional flakes due to insufficient quota: enough quota in apache-beam-testing. https://issues.apache.org/jira/browse/BEAM-7463 - BigQueryQueryToTableIT is flaky on direct runner (incorrect checksum) https://issues.apache.org/jira/browse/BEAM-6202 - Dataflow runner occasionally fails with "Job did not reach to a terminal state after waiting indefinitely", while job succeeds. https://issues.apache.org/jira/browse/BEAM-7546 - Portable WordCount-on-Flink Precommit is flaky - temporary folder not found. Fix submitted, awaiting verification. I'll try to find owners for issues that currently don't have an owner. Thanks, Valentyn On Mon, Jul 15, 2019 at 9:17 AM Katarzyna Kucharczyk < ka.kucharc...@gmail.com> wrote: > Hi, > > this is something we also observed in Python LoadTests. I created another > issue on Friday issue [BEAM-7733] > <https://issues.apache.org/jira/browse/BEAM-7733> because I didn't see > the one you shared, Valentyn. Whole problem seems to be a bit similar to this > ([BEAM-5529]) <https://issues.apache.org/jira/browse/BEAM-5529> but I am > not sure if a pipeline in the resolved issue had same HTTP error. > > Thanks, > Kasia > > On Mon, Jul 15, 2019 at 7:11 AM Valentyn Tymofieiev <valen...@google.com> > wrote: > >> This is actually a quite frequent error right now. >> https://issues.apache.org/jira/browse/BEAM-6202 is opened to make >> Dataflow runner more robust when polling status of a running Dataflow job. >> >> On Wed, Mar 6, 2019 at 8:18 PM Valentyn Tymofieiev <valen...@google.com> >> wrote: >> >>> Hi Juta, >>> >>> I also looked into the precommit errors and seems like the errors are >>> associated with a disruption around that time unrelated to Beam or Beam >>> test infrastructure. Please try again. >>> >>> Thanks, >>> Valentyn >>> >>> On Wed, Mar 6, 2019 at 4:18 PM Mark Liu <mark...@google.com> wrote: >>> >>>> I saw HttpError from the console log >>>> <https://scans.gradle.com/s/b3s5klq6ohydi/console-log#L6817> in the >>>> job you provided above >>>> <https://builds.apache.org/job/beam_PostCommit_Python3_Verify_PR/77/>: >>>> >>>> "error": { >>>> "code": 404, >>>> "message": "(b8b815bd6683759f): Information about job >>>> 2019-03-06_04_32_19-1780414729477627626 could not be found in our system. >>>> Please double check the id is correct. If it is please contact customer >>>> support.", >>>> "status": "NOT_FOUND" >>>> } >>>> >>>> Looks like it's caused by Dataflow service flaky, not your test. Since >>>> the current postcommit >>>> <https://builds.apache.org/job/beam_PostCommit_Python_Verify/> is >>>> green now, you can run postcommit again. >>>> >>>> Mark >>>> >>>> >>>> On Wed, Mar 6, 2019 at 10:04 AM Juta Staes <juta.st...@ml6.eu> wrote: >>>> >>>>> Hi all, >>>>> >>>>> I am working on adding Python 3 integration tests to the post commit >>>>> tests. >>>>> >>>>> Since yesterday none of the post commit tests seem to pass. There are >>>>> errors in the build jobs as can be seen in the Post-Commit Tests Status >>>>> table at https://github.com/apache/beam/pull/7946 (and in other prs >>>>> as well) and at >>>>> https://builds.apache.org/job/beam_PostCommit_Python3_Verify_PR/77/console >>>>> >>>>> AssertionError: Job did not reach to a terminal state after waiting >>>>> indefinitely. >>>>> >>>>> >>>>> Yesterday my tests did pass: >>>>> https://builds.apache.org/job/beam_PostCommit_Python3_Verify_PR/61/console >>>>> but >>>>> failed on a build timed out. >>>>> >>>>> Does someone has any idea what is happening with the post commit tests? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> >>>>> [image: https://ml6.eu] <https://ml6.eu/> >>>>> >>>>> * Juta Staes* >>>>> ML6 Gent >>>>> <https://www.google.be/maps/place/ML6/@51.037408,3.7044893,17z/data=!3m1!4b1!4m5!3m4!1s0x47c37161feeca14b:0xb8f72585fdd21c90!8m2!3d51.037408!4d3.706678?hl=nl> >>>>> >>>>> **** DISCLAIMER **** >>>>> This email and any files transmitted with it are confidential and >>>>> intended solely for the use of the individual or entity to whom they are >>>>> addressed. If you have received this email in error please notify the >>>>> system manager. This message contains confidential information and is >>>>> intended only for the individual named. If you are not the named addressee >>>>> you should not disseminate, distribute or copy this e-mail. Please notify >>>>> the sender immediately by e-mail if you have received this e-mail by >>>>> mistake and delete this e-mail from your system. If you are not the >>>>> intended recipient you are notified that disclosing, copying, distributing >>>>> or taking any action in reliance on the contents of this information is >>>>> strictly prohibited. >>>>> >>>>
