[
https://issues.apache.org/jira/browse/BEAM-12572?focusedWorklogId=695434&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-695434
]
ASF GitHub Bot logged work on BEAM-12572:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Dec/21 00:17
Start Date: 14/Dec/21 00:17
Worklog Time Spent: 10m
Work Description: benWize edited a comment on pull request #16154:
URL: https://github.com/apache/beam/pull/16154#issuecomment-988323116
Hi @tvalentyn, I've been working in this PR to execute the examples in
Direct, Dataflow, Spark, and Flink runners.
I added some tests to the examples that weren't running the same pipeline in
the existing tests.
Some of the tests fail for specific runners, would you help me to review if
I have something missing in the config task for each runner, or maybe If there
are examples that are expected to fail for a specific runner, and I need to
ignore those.
-
[Dataflow](https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Dataflow/3/testReport/):
Almost all examples succeed, only
[this](https://github.com/apache/beam/blob/014b3805dc94762372c6871c1885307ea759397a/sdks/python/apache_beam/examples/cookbook/bigquery_side_input_test.py#L68)
example is failing, not sure If is because of how the output file is created,
maybe writing to GCS would work.
-
[Spark](https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Spark/2/#showFailuresLink)
here multiple tests are failing, most are integration tests that require setUp
or tearDown of external resources.
-
[Flink](https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Flink/2/#showFailuresLink)
same as Spark, most of the integration tests failing.
-
[Direct](https://ci-beam.apache.org/job/beam_PostCommit_Python_Examples_Direct/3/#showFailuresLink)
some integration tests failing too.
I think now most of the examples have a corresponding test that executes it,
<del>the only one missing is
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/cookbook/filters.py#L75
because I'm not sure what BQ dataset can I use to test it. </del> (Test added
in
https://github.com/apache/beam/pull/16154/commits/06d43c2aebb1ebb0ed5910bbba81bfa769285045)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 695434)
Time Spent: 12h (was: 11h 50m)
> All beam examples should get continuously exercised on at least 2 runners
> -------------------------------------------------------------------------
>
> Key: BEAM-12572
> URL: https://issues.apache.org/jira/browse/BEAM-12572
> Project: Beam
> Issue Type: New Feature
> Components: testing
> Reporter: Valentyn Tymofieiev
> Assignee: Benjamin Gonzalez
> Priority: P3
> Time Spent: 12h
> Remaining Estimate: 0h
>
> Sometimes our examples become broken without us noticing. For example, see:
> https://lists.apache.org/thread.html/r45340bbee91a6caf798fe62d24388f645f8792cc7506351fd66adec3%40%3Cdev.beam.apache.org%3E
> We should test our examples better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
