[
https://issues.apache.org/jira/browse/BEAM-9150?focusedWorklogId=733368&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-733368
]
ASF GitHub Bot logged work on BEAM-9150:
----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Feb/22 21:46
Start Date: 25/Feb/22 21:46
Worklog Time Spent: 10m
Work Description: emilymye opened a new pull request #16955:
URL: https://github.com/apache/beam/pull/16955
This PR:
* Adds clean-up step prior to running hourly runner
* Fixes variable/naming of tables to make table name (per-runner) more clear
* `s/verify_steaming_result/verify_streaming_result`
Tested by cherry-picking into a 2.37.0 test branch
(https://github.com/apache/beam/pull/16944), test run:
beam.apache.org/job/beam_PostRelease_Python_Candidate/228/)
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit
[https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more
information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 733368)
Remaining Estimate: 0h
Time Spent: 10m
> beam_PostRelease_Python_Candidate fails
> ---------------------------------------
>
> Key: BEAM-9150
> URL: https://issues.apache.org/jira/browse/BEAM-9150
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Reporter: Udi Meiri
> Priority: P2
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://builds.apache.org/job/beam_PostRelease_Python_Candidate/171/timestamps/?time=HH:mm:ss&timeZone=GMT-8&appendLog&locale=en_US
> Some interesting lines:
> {code}
> 17:48:36
> release/src/main/python-release/run_release_candidate_python_quickstart.sh:
> line 144: 20935 Killed python -m
> apache_beam.examples.streaming_wordcount --input_topic
> projects/$PROJECT_ID/topics/$PUBSUB_TOPIC1 --output_topic
> projects/$PROJECT_ID/topics/$PUBSUB_TOPIC2 --streaming
> {code}
> {code}
> 18:05:25
> ############################################################################
> 18:05:25 Running HourlyTeamScore example with DirectRunner
> 18:05:25
> ############################################################################
> 18:05:25
> /tmp/tmp.gnpMNIFhcL/temp_virtualenv_python2.7/local/lib/python2.7/site-packages/apache_beam/__init__.py:84:
> UserWarning: You are using Apache Beam with Python 2. New releases of Apache
> Beam will soon support Python 3 only.
> 18:05:25 'You are using Apache Beam with Python 2. '
> 18:05:33 INFO:root:Missing pipeline option (runner). Executing pipeline
> using the default runner: DirectRunner.
> 18:05:33 INFO:root:Setting socket default timeout to 60 seconds.
> 18:05:33 INFO:root:socket default timeout is 60.0 seconds.
> 18:05:33 INFO:oauth2client.transport:Attempting refresh to obtain initial
> access_token
> 18:05:34
> /tmp/tmp.gnpMNIFhcL/temp_virtualenv_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1220:
> BeamDeprecationWarning: options is deprecated since First stable release.
> References to <pipeline>.options will not be supported
> 18:05:34 experiments = p.options.view_as(DebugOptions).experiments or []
> 18:05:37 INFO:root:==================== <function
> annotate_downstream_side_inputs at 0x7f131dc46f50> ====================
> 18:05:37 INFO:root:==================== <function
> fix_side_input_pcoll_coders at 0x7f131dc3e0c8> ====================
> 18:05:37 INFO:root:==================== <function lift_combiners at
> 0x7f131dc3e140> ====================
> 18:05:37 INFO:root:==================== <function expand_sdf at
> 0x7f131dc3e1b8> ====================
> 18:05:37 INFO:root:==================== <function expand_gbk at
> 0x7f131dc3e230> ====================
> 18:05:37 INFO:root:==================== <function sink_flattens at
> 0x7f131dc3e320> ====================
> 18:05:37 INFO:root:==================== <function greedily_fuse at
> 0x7f131dc3e398> ====================
> 18:05:37 INFO:root:==================== <function read_to_impulse at
> 0x7f131dc3e410> ====================
> 18:05:37 INFO:root:==================== <function impulse_to_input at
> 0x7f131dc3e488> ====================
> 18:05:37 INFO:root:==================== <function inject_timer_pcollections
> at 0x7f131dc3e5f0> ====================
> 18:05:37 INFO:root:==================== <function sort_stages at
> 0x7f131dc3e668> ====================
> 18:05:37 INFO:root:==================== <function window_pcollection_coders
> at 0x7f131dc3e6e0> ====================
> 18:05:37 INFO:root:Creating state cache with size 100
> 18:05:37 INFO:root:Created Worker handler
> <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object
> at 0x7f12fc89c290> for environment urn: "beam:env:embedded_python:v1"
> 18:05:37
> 18:05:37 INFO:root:Running
> ((ref_AppliedPTransform_ReadInputText/Read/_SDFBoundedSourceWrapper/Impulse_5)+(ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+((ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction)+(ref_PCollection_PCollection_1_split/Write))
> 18:05:38 INFO:root:Running
> (ref_PCollection_PCollection_1_split/Read)+((((ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process)+((ref_AppliedPTransform_HourlyTeamScore/ParseGameEventFn_8)+((ref_AppliedPTransform_HourlyTeamScore/FilterStartTime_9)+((ref_AppliedPTransform_HourlyTeamScore/FilterEndTime_10)+((ref_AppliedPTransform_HourlyTeamScore/AddEventTimestamps_11)+((ref_AppliedPTransform_HourlyTeamScore/FixedWindowsTeam_12)+(ref_AppliedPTransform_HourlyTeamScore/ExtractAndSumScore/Map(<lambda
> at
> hourly_team_score.py:142>)_14)))))))+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Precombine))+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Group/Write))
> 18:05:39 INFO:root:Running
> ((HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Group/Read)+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Merge))+((HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/ExtractOutputs)+((ref_AppliedPTransform_TeamScoresDict_22)+((ref_AppliedPTransform_WriteTeamScoreSums/ConvertToRow_24)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination_27)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds_28)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_30)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_32)+(WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write))))))))
> 18:05:39 INFO:root:Running
> (WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_37)+(ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_38))+(ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_40))
> 18:05:40 bq head -n 500 $DATASET.hourly_team_score_python_$1
> 18:05:43 Did not find team scores, retry left: 2
> 18:05:58 bq head -n 500 $DATASET.hourly_team_score_python_$1
> 18:06:00 Did not find team scores, retry left: 1
> 18:06:15 bq head -n 500 $DATASET.hourly_team_score_python_$1
> 18:06:17 Did not find team scores, retry left: 0
> 18:06:32 FAILED: HourlyTeamScore example failed running on direct-runner.
> Did not found scores of team AntiqueBrassPlatypus in
> beam_postrelease_mobile_gaming.leader_board
> 18:06:32
> ############################################################################
> 18:06:32 Validation FAILED
> 18:06:32
> ############################################################################
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
