[
https://issues.apache.org/jira/browse/BEAM-13377?focusedWorklogId=691132&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-691132
]
ASF GitHub Bot logged work on BEAM-13377:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Dec/21 15:45
Start Date: 06/Dec/21 15:45
Worklog Time Spent: 10m
Work Description: ilya-kozyrev commented on a change in pull request
#16140:
URL: https://github.com/apache/beam/pull/16140#discussion_r763128242
##########
File path: playground/infrastructure/ci_cd.py
##########
@@ -16,37 +16,76 @@
"""
Module implements CI/CD steps for Beam Playground examples
"""
-
+import argparse
import asyncio
import os
+from typing import List, Literal
+import config
+from api.v1.api_pb2 import Sdk
from cd_helper import CDHelper
from ci_helper import CIHelper
-from helper import find_examples, get_supported_categories
+from helper import find_examples, get_supported_categories, Example
from logger import setup_logger
+parser = argparse.ArgumentParser(
+ description="CI/CD Steps for Playground objects")
+parser.add_argument(
+ "--step",
+ dest="step",
+ required=True,
+ help="CI step to verify all beam examples/tests/katas. CD step to save all
"
+ "beam examples/tests/katas and their outputs on the GCS",
+ choices=["CI", "CD"])
+parser.add_argument(
+ "--sdk",
+ dest="sdk",
+ required=True,
+ help="Supported SDKs",
+ choices=config.Config.SUPPORTED_SDK)
+
+root_dir = os.getenv("BEAM_ROOT_DIR")
+categories_file = os.getenv("BEAM_EXAMPLE_CATEGORIES")
-def ci_step():
+
+def ci_step(examples: List[Example]):
"""
CI step to verify all beam examples/tests/katas
"""
- setup_logger()
- root_dir = os.getenv("BEAM_ROOT_DIR")
- categories_file = os.getenv("BEAM_EXAMPLE_CATEGORIES")
- supported_categories = get_supported_categories(categories_file)
+
ci_helper = CIHelper()
- examples = find_examples(root_dir, supported_categories)
asyncio.run(ci_helper.verify_examples(examples))
-def cd_step():
+def cd_step(examples: List[Example]):
"""
CD step to save all beam examples/tests/katas and their outputs on the GCS
"""
- setup_logger()
- root_dir = os.getenv("BEAM_ROOT_DIR")
- categories_file = os.getenv("BEAM_EXAMPLE_CATEGORIES")
- supported_categories = get_supported_categories(categories_file)
cd_helper = CDHelper()
- examples = find_examples(root_dir, supported_categories)
cd_helper.store_examples(examples)
+
+
+def _check_envs():
+ if root_dir is None:
+ raise KeyError(
+ "BEAM_ROOT_DIR environment variable should be specified in os")
+ if categories_file is None:
+ raise KeyError(
+ "BEAM_EXAMPLE_CATEGORIES environment variable should be specified in
os"
+ )
+
+
+def run_ci_cd(step: Literal["CI", "CD"], sdk: Sdk):
+ supported_categories = get_supported_categories(categories_file)
+ examples = find_examples(root_dir, supported_categories, sdk)
+ if step == "CI":
+ ci_step(examples=examples)
+ if step == "CD":
+ cd_step(examples=examples)
Review comment:
that's not our business, I've passed choices to argparser
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 691132)
Time Spent: 1.5h (was: 1h 20m)
> [Playground] Update CI/CD python scripts to find, verify and store examples
> ---------------------------------------------------------------------------
>
> Key: BEAM-13377
> URL: https://issues.apache.org/jira/browse/BEAM-13377
> Project: Beam
> Issue Type: Improvement
> Components: beam-playground
> Reporter: Aydar Zaynutdinov
> Assignee: Ilya Kozyrev
> Priority: P3
> Labels: beam-playground-backend, beam-playground-devops,
> beam-playground-sprint-5
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Now CI/CD scripts find, verify and store all examples (for each supported
> SDK) at the same time.
> Need to work with examples only for selected SDK (selected SDK from the
> environment).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
