[
https://issues.apache.org/jira/browse/BEAM-12572?focusedWorklogId=717158&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-717158
]
ASF GitHub Bot logged work on BEAM-12572:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Jan/22 17:08
Start Date: 28/Jan/22 17:08
Worklog Time Spent: 10m
Work Description: ryanthompson591 commented on a change in pull request
#16154:
URL: https://github.com/apache/beam/pull/16154#discussion_r794666629
##########
File path: sdks/python/apache_beam/examples/complete/estimate_pi.py
##########
@@ -101,21 +101,27 @@ def expand(self, pcoll):
| 'Sum' >> beam.CombineGlobally(combine_results).without_defaults())
-def run(argv=None):
+def run(argv=None, save_main_session=True):
parser = argparse.ArgumentParser()
parser.add_argument(
'--output', required=True, help='Output file to write results to.')
+ parser.add_argument(
Review comment:
Prefer just using a constant to an extra argument unless we anticipate
anyone changing this for any reason.
##########
File path:
sdks/python/apache_beam/examples/complete/game/hourly_team_score_it_test.py
##########
@@ -90,6 +90,35 @@ def test_hourly_team_score_it(self):
self.test_pipeline.get_full_options_as_args(**extra_opts),
save_main_session=False)
+ @pytest.mark.examples_postcommit
+ def test_hourly_team_score_output_checksum_on_small_input(self):
+ # Small dataset to prevent OOM when running in local runners
Review comment:
I'm not sure what OOM is
##########
File path: sdks/python/apache_beam/examples/complete/game/user_score_test.py
##########
@@ -42,7 +40,6 @@ class UserScoreTest(unittest.TestCase):
'user1_team1,team1,14,1447697463000,2015-11-16 18:11:03.955',
]
- @pytest.mark.examples_postcommit
Review comment:
just curious why are we removing these?
##########
File path: sdks/python/apache_beam/examples/complete/autocomplete_test.py
##########
@@ -37,7 +41,11 @@ class AutocompleteTest(unittest.TestCase):
KINGLEAR_HASH_SUM = 268011785062540
KINGLEAR_INPUT = 'gs://dataflow-samples/shakespeare/kinglear.txt'
- @pytest.mark.examples_postcommit
+ def create_file(self, path, contents):
Review comment:
can we rename this to something more meaningful than create_file. Like
create_content_output_file or whatever this actually represents.
##########
File path: sdks/python/apache_beam/examples/complete/estimate_pi_test.py
##########
@@ -51,6 +54,20 @@ def test_basics(self):
# trials.
assert_that(result, in_between(3.125, 3.155))
+ @pytest.mark.examples_postcommit
+ def test_estimate_pi_output_file(self):
+ temp_folder = tempfile.mkdtemp()
+ estimate_pi.run(
+ ['--output', os.path.join(temp_folder, 'result'), '--tries', '5000'],
+ save_main_session=False)
+ # Load result file and compare.
+ with open_shards(os.path.join(temp_folder, 'result-*-of-*')) as
result_file:
+ result = json.loads(result_file.read().strip())[2]
Review comment:
Try to make this line more readable, maybe a constant to what [2]
means...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 717158)
Time Spent: 32h (was: 31h 50m)
> All beam examples should get continuously exercised on at least 2 runners
> -------------------------------------------------------------------------
>
> Key: BEAM-12572
> URL: https://issues.apache.org/jira/browse/BEAM-12572
> Project: Beam
> Issue Type: New Feature
> Components: testing
> Reporter: Valentyn Tymofieiev
> Assignee: Benjamin Gonzalez
> Priority: P3
> Time Spent: 32h
> Remaining Estimate: 0h
>
> Sometimes our examples become broken without us noticing. For example, see:
> https://lists.apache.org/thread.html/r45340bbee91a6caf798fe62d24388f645f8792cc7506351fd66adec3%40%3Cdev.beam.apache.org%3E
> We should test our examples better.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
