Valentyn Tymofieiev created BEAM-7474:
-----------------------------------------
Summary: Add SDK harness containers for Py 3.6, Py 3.7
Key: BEAM-7474
URL: https://issues.apache.org/jira/browse/BEAM-7474
Project: Beam
Issue Type: Sub-task
Components: sdk-py-harness
Reporter: Valentyn Tymofieiev
Currently we can build a Py3-compatible container image with gradle by running:
./gradlew :sdks:python:container:py3:docker
This builds a docker container image like:
valentyn-docker-apache.bintray.io/beam/python3
The code for this is defined in:
https://github.com/apache/beam/blob/ae60a72b03f3a2b6b2a06667ec1868a7acc8e38f/sdks/python/container/py3/build.gradle#L48
To support portable runners that use a container (e.g. Flink) on multiple
versions of Python 3, we should make it possible to build Python 3-compatible
SDK harness containers bundled with any desired python version.
We could have several gradle projects:
:sdks:python:container:py35:docker
:sdks:python:container:py36:docker
:sdks:python:container:py37:docker
and several Dockerfiles to support this:
sdks/python/container/py35/Dockerfile
sdks/python/container/py36/Dockerfile
sdks/python/container/py37/Dockerfile
The only difference right now would be the base image used in FROM field in
Dockerfile.
Alternatively, we could have one parameterized Dockerfile that starts with :
{code}
ARG BASE_IMAGE
FROM $BASE_IMAGE
...
{code}
I think the latter approach, may result in complications later if these
containers will need to diverge down the road.
cc'ing a few folks who may have some feedback on this: [~angoenka] [~mxm]
[~robertwb] [~Juta] [~fredo838].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
