[
https://issues.apache.org/jira/browse/BEAM-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentyn Tymofieiev updated BEAM-7474:
--------------------------------------
Description:
Currently we can build a Py3-compatible container image with gradle by running:
./gradlew :sdks:python:container:py3:docker
This builds a docker container image like:
valentyn-docker-apache.bintray.io/beam/python3
The code for this is defined in:
https://github.com/apache/beam/blob/ae60a72b03f3a2b6b2a06667ec1868a7acc8e38f/sdks/python/container/py3/build.gradle#L48
To support portable runners that use a container (e.g. Flink) on multiple
versions of Python 3, we should make it possible to build Python 3-compatible
SDK harness containers bundled with any desired python version.
We could have several gradle projects:
:sdks:python:container:py35:docker
:sdks:python:container:py36:docker
:sdks:python:container:py37:docker
and several Dockerfiles to support this:
sdks/python/container/py35/Dockerfile
sdks/python/container/py36/Dockerfile
sdks/python/container/py37/Dockerfile
The only difference right now would be the base image used in FROM field in
Dockerfile.
Alternatively, we could have one parameterized Dockerfile that starts with :
{code}
ARG BASE_IMAGE
FROM $BASE_IMAGE
...
{code}
I think the latter approach, may result in complications later if these
containers will need to diverge down the road.
cc'ing a few folks who may have some feedback on this: [~angoenka] [~mxm]
[~robertwb] [~Juta] [~frederik].
was:
Currently we can build a Py3-compatible container image with gradle by running:
./gradlew :sdks:python:container:py3:docker
This builds a docker container image like:
valentyn-docker-apache.bintray.io/beam/python3
The code for this is defined in:
https://github.com/apache/beam/blob/ae60a72b03f3a2b6b2a06667ec1868a7acc8e38f/sdks/python/container/py3/build.gradle#L48
To support portable runners that use a container (e.g. Flink) on multiple
versions of Python 3, we should make it possible to build Python 3-compatible
SDK harness containers bundled with any desired python version.
We could have several gradle projects:
:sdks:python:container:py35:docker
:sdks:python:container:py36:docker
:sdks:python:container:py37:docker
and several Dockerfiles to support this:
sdks/python/container/py35/Dockerfile
sdks/python/container/py36/Dockerfile
sdks/python/container/py37/Dockerfile
The only difference right now would be the base image used in FROM field in
Dockerfile.
Alternatively, we could have one parameterized Dockerfile that starts with :
{code}
ARG BASE_IMAGE
FROM $BASE_IMAGE
...
{code}
I think the latter approach, may result in complications later if these
containers will need to diverge down the road.
cc'ing a few folks who may have some feedback on this: [~angoenka] [~mxm]
[~robertwb] [~Juta] [~fredo838].
> Add SDK harness containers for Py 3.6, Py 3.7
> ---------------------------------------------
>
> Key: BEAM-7474
> URL: https://issues.apache.org/jira/browse/BEAM-7474
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-harness
> Reporter: Valentyn Tymofieiev
> Priority: Major
>
> Currently we can build a Py3-compatible container image with gradle by
> running:
> ./gradlew :sdks:python:container:py3:docker
> This builds a docker container image like:
> valentyn-docker-apache.bintray.io/beam/python3
> The code for this is defined in:
> https://github.com/apache/beam/blob/ae60a72b03f3a2b6b2a06667ec1868a7acc8e38f/sdks/python/container/py3/build.gradle#L48
> To support portable runners that use a container (e.g. Flink) on multiple
> versions of Python 3, we should make it possible to build Python
> 3-compatible SDK harness containers bundled with any desired python version.
> We could have several gradle projects:
> :sdks:python:container:py35:docker
> :sdks:python:container:py36:docker
> :sdks:python:container:py37:docker
> and several Dockerfiles to support this:
>
> sdks/python/container/py35/Dockerfile
> sdks/python/container/py36/Dockerfile
> sdks/python/container/py37/Dockerfile
> The only difference right now would be the base image used in FROM field in
> Dockerfile.
> Alternatively, we could have one parameterized Dockerfile that starts with :
> {code}
> ARG BASE_IMAGE
> FROM $BASE_IMAGE
> ...
> {code}
> I think the latter approach, may result in complications later if these
> containers will need to diverge down the road.
> cc'ing a few folks who may have some feedback on this: [~angoenka] [~mxm]
> [~robertwb] [~Juta] [~frederik].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
