ryanthompson591 commented on issue #21121: URL: https://github.com/apache/beam/issues/21121#issuecomment-1285911807
Just wanted to update with what I did in case I don't come back to this. Getting this test to run locally entailed two things. 1. Change the timeout from 6 minutes to 12 minutes. This is because if you run it locally it takes 3 minutes to even start up dataflow and the test will time out. 2. Here's a command to run it -- ``` python -m pytest -o log_cli=True -o log_level=Info apache_beam/examples/streaming_wordcount_it_test.py --test-pipeline-options=' --runner=TestDataflowRunner --project=apache-beam-testing --region=us-central1 --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=dist/apache-beam-2.43.0.dev0.tar.gz --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test' ``` or ``` scripts/run_integration_test.sh --test_opts "apache_beam/examples/streaming_wordcount_it_test.py" --sdk_location "dist/apache-beam-2.43.0.dev0.tar.gz" ``` I checked the dataflow job that was linked in [Abacn](https://github.com/Abacn) comment. The job showed "about 500" messages in each transform (which was the prediction) though it didn't promise exactness (the failure showed 501 messages). I am convinced this is a real breakage and may indicate a more serious issue. I just couldn't reproduce the flake on my local runs as they took 8 minutes each. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
