TobKed commented on a change in pull request #11877: URL: https://github.com/apache/beam/pull/11877#discussion_r436067638
########## File path: .github/workflows/build_wheels.yml ########## @@ -0,0 +1,141 @@ +name: Build python wheels + +on: + push: + branches: + - master + - release-* + tags: + - v* + +jobs: + + build_source: + runs-on: ubuntu-18.04 + steps: + - name: Checkout code + uses: actions/checkout@v2 + - name: Install python + uses: actions/setup-python@v2 + with: + python-version: 3.7 + - name: Get build dependencies + working-directory: ./sdks/python + run: python3 -m pip install cython && python3 -m pip install -r build-requirements.txt + - name: Install wheels + run: python3 -m pip install wheel + - name: Buld source + working-directory: ./sdks/python + run: python3 setup.py sdist --formats=gztar,zip + - name: Unzip source + working-directory: ./sdks/python + run: unzip dist/$(ls dist | grep .zip | head -n 1) + - name: Rename source directory + working-directory: ./sdks/python + run: mv $(ls | grep apache-beam) apache-beam-source + - name: Upload source + uses: actions/upload-artifact@v2 + with: + name: source + path: sdks/python/apache-beam-source + - name: Upload compressed sources + uses: actions/upload-artifact@v2 + with: + name: source_gztar_zip Review comment: 1. Currently two steps: `List sources on GCS bucket` and `Copy wheels to GCS bucket` are listing files of specific types. Instead of this two separate steps I could create job which will list all files in specific gcs folder. I think it would be much cleaner and explicit. Did I understand correctly your idea? About cleaning up these GCS locations I consider two options: - setting lifecycle management on the bucket which will delete files older than some arbitrary age, e.g. 365 days. I think advantage of this is that will be maintenance free. - creating another scheduled workflow on github actions which will delete gcs folders if corresponding branch does not exist anymore. Could be scheduled to run e.g. once pre week. Which option has more sense for you? 2. "Upload" steps perform file upload as artifacts so they could be passed between jobs and being available for download for 90 days (if not deleted earlier). These artifacts are picked up later by "Upload to GCS" jobs. What do you think about renaming these steps e.g.: "Upload wheels" -> "Upload wheels as artifacts" ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org