This is an automated email from the ASF dual-hosted git repository.
1996fanrui pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new ac1b1357 [FLINK-39646][examples] Update python example to Flink 2.2
(#1111)
ac1b1357 is described below
commit ac1b1357d60bc99d49c6057223c39f845e948df1
Author: Aleksandr Savonin <[email protected]>
AuthorDate: Mon May 18 12:28:02 2026 +0200
[FLINK-39646][examples] Update python example to Flink 2.2 (#1111)
---
examples/flink-python-example/Dockerfile | 29 ++++++++++-------------
examples/flink-python-example/README.md | 15 +++++++-----
examples/flink-python-example/python-example.yaml | 6 ++---
3 files changed, 24 insertions(+), 26 deletions(-)
diff --git a/examples/flink-python-example/Dockerfile
b/examples/flink-python-example/Dockerfile
index 3046bdf8..6fb1e61a 100644
--- a/examples/flink-python-example/Dockerfile
+++ b/examples/flink-python-example/Dockerfile
@@ -16,27 +16,22 @@
# limitations under the License.
################################################################################
# Check
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/#using-flink-python-on-docker
for more details
-FROM flink:1.16
+FROM flink:2.2
-# install python3: it has updated Python to 3.9 in Debian 11 and so install
Python 3.7 from source, \
-# it currently only supports Python 3.6, 3.7 and 3.8 in PyFlink officially.
+# The flink:2.x base image runs as the non-root `flink` user; switch to root
+# to install system packages, then drop privileges again before adding the job.
+USER root
+# install python3 and pip3
RUN apt-get update -y && \
-apt-get install -y build-essential libssl-dev zlib1g-dev libbz2-dev libffi-dev
&& \
-wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz && \
-tar -xvf Python-3.7.9.tgz && \
-cd Python-3.7.9 && \
-./configure --without-tests --enable-shared && \
-make -j6 && \
-make install && \
-ldconfig /usr/local/lib && \
-cd .. && rm -f Python-3.7.9.tgz && rm -rf Python-3.7.9 && \
-ln -s /usr/local/bin/python3 /usr/local/bin/python && \
-apt-get clean && \
-rm -rf /var/lib/apt/lists/*
+ apt-get install -y python3 python3-pip python3-dev && \
+ rm -rf /var/lib/apt/lists/* && \
+ ln -s /usr/bin/python3 /usr/bin/python
-# install PyFlink
-RUN pip3 install "apache-flink>=1.16.0,<1.17.1"
+# install PyFlink (PyFlink 2.2 supports Python 3.9, 3.10, 3.11, 3.12).
+# --break-system-packages is required because the base image is Ubuntu 24.04
+# (Noble), which enforces PEP 668.
+RUN pip3 install --break-system-packages apache-flink==2.2.0
# add python script
USER flink
diff --git a/examples/flink-python-example/README.md
b/examples/flink-python-example/README.md
index b3dfa21d..53c547b3 100644
--- a/examples/flink-python-example/README.md
+++ b/examples/flink-python-example/README.md
@@ -35,7 +35,7 @@ This is an end-to-end example of running Flink Python jobs
using the Flink Kuber
Flink supports Python jobs in application mode by utilizing
`org.apache.flink.client.python.PythonDriver` class as the
entry class. With the Flink Kubernetes Operator, we can reuse this class to
run Python jobs as well.
-The class is packaged in flink-python_${scala_version}-${flink_version}.jar
which is in the default Flink image.
+The class is packaged in flink-python-${flink_version}.jar which is in the
default Flink image.
So we do not need to create a new job jar. Instead, we just set `entryClass`
of the job crd to
`org.apache.flink.client.python.PythonDriver`. After applying the job yaml,
the launched job manager pod will run the `main()`
method of PythonDriver and parse arguments declared in the `args` field of the
job crd.
@@ -46,11 +46,11 @@ user-defined arguments should be placed in the end. Check
the [doc](https://nigh
A working example would be:
```yaml
-args: ["-pyfs", "/opt/flink/usrlib/pythonjob/python_demo.py", "-pyclientexec",
"/usr/local/bin/python3", "-py", "/opt/flink/usrlib/pythonjob/python_demo.py",
"-myarg", "123"]
+args: ["-pyfs", "/opt/flink/usrlib/python_demo.py", "-pyclientexec",
"/usr/bin/python3", "-py", "/opt/flink/usrlib/python_demo.py", "-myarg", "123"]
```
But the following will throw exception:
```yaml
-args: ["-myarg", "123", "-pyfs", "/opt/flink/usrlib/pythonjob/python_demo.py",
"-pyclientexec", "/usr/local/bin/python3", "-py",
"/opt/flink/usrlib/pythonjob/python_demo.py"]
+args: ["-myarg", "123", "-pyfs", "/opt/flink/usrlib/python_demo.py",
"-pyclientexec", "/usr/bin/python3", "-py", "/opt/flink/usrlib/python_demo.py"]
```
## Usage
@@ -63,16 +63,19 @@ Dockerfile
**Step 2**: Build docker image
-Check this
[doc](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/#using-flink-python-on-docker)
for more details about building Pyflink image. Note, Pyflink 1.15.3 is only
supported on x86 arch.
+Check this
[doc](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/#using-flink-python-on-docker)
for more details about building a PyFlink image. The included Dockerfile is
based on `flink:2.2` and installs PyFlink 2.2.0 (which supports Python 3.9,
3.10, 3.11 and 3.12).
+
+Note: PyFlink's `pemja` dependency only ships pre-built Linux wheels for
`x86_64`, and the `flink:2.2` base image is JRE-only, so building from source
on other architectures fails. Build for `linux/amd64` explicitly - the
resulting image runs natively on x86 and on Apple Silicon via Docker Desktop's
emulation.
+
```bash
# Uncomment when building for local minikube env:
# eval $(minikube docker-env)
-DOCKER_BUILDKIT=1 docker build . -t flink-python-example:latest
+DOCKER_BUILDKIT=1 docker build --platform linux/amd64 . -t
flink-python-example:latest
```
This step will create an image based on an official Flink base image including
the Python scripts.
-**Step 4**: Create FlinkDeployment Yaml and Submit
+**Step 3**: Create FlinkDeployment Yaml and Submit
Edit the included `python-example.yaml` so that the `job.args` section points
to the Python script that you wish to execute, then submit it.
diff --git a/examples/flink-python-example/python-example.yaml
b/examples/flink-python-example/python-example.yaml
index 87fbacaf..8ede374a 100644
--- a/examples/flink-python-example/python-example.yaml
+++ b/examples/flink-python-example/python-example.yaml
@@ -22,7 +22,7 @@ metadata:
name: python-example
spec:
image: flink-python-example:latest
- flinkVersion: v1_20
+ flinkVersion: v2_2
flinkConfiguration:
taskmanager.numberOfTaskSlots: 1
serviceAccount: flink
@@ -35,8 +35,8 @@ spec:
memory: "2048m"
cpu: 1
job:
- jarURI: local:///opt/flink/opt/flink-python_2.12-1.16.1.jar # Note, this
jarURI is actually a placeholder
+ jarURI: local:///opt/flink/opt/flink-python-2.2.0.jar # Note, this jarURI
is actually a placeholder
entryClass: "org.apache.flink.client.python.PythonDriver"
- args: ["-pyclientexec", "/usr/local/bin/python3", "-py",
"/opt/flink/usrlib/python_demo.py"]
+ args: ["-pyclientexec", "/usr/bin/python3", "-py",
"/opt/flink/usrlib/python_demo.py"]
parallelism: 1
upgradeMode: stateless
