geyanggang opened a new issue, #13265:
URL: https://github.com/apache/gravitino/issues/13265
### What would you like to be improved?
The three engine connectors (Trino, Flink, Spark) are published to Maven
Central and as tarballs on the downloads page, but there is no first-party
container image in the Apache Gravitino repository for installing them into
engines running on Kubernetes.
A common deployment pattern is to inject the connector plugin/runtime jars
into Trino, Flink or Spark pods via an init container that shares a volume with
the engine. Today operators have to assemble this themselves. Providing
official, Red Hat UBI-based connector images in-tree would:
Give Kubernetes users a ready-to-use init-container image per engine,
containing the complete plugin/runtime layout (for Trino, the connector jar
plus its dependency jars; no database JDBC drivers, which come from the
engine's own plugins).
Keep the image contents purely Apache-2.0 with the correct LICENSE/NOTICE,
aligned with the connector artifacts.
Track whatever connector versions a branch supports automatically, so the
same Dockerfiles work across main, branch-1.3, etc. without edits.
The images are init containers, not servers: the entrypoint copies the jars
for the requested version into a mounted /target volume and exits.
### How should we improve?
Add dev/docker/{trino,flink,spark}-connectors/ build directories, each with:
A Dockerfile based on registry.access.redhat.com/ubi10/ubi-minimal, with Red
Hat certification labels, a /licenses directory (LICENSE + NOTICE), non-root
user (UID 1000, GID 0) and OpenShift arbitrary-UID compatibility.
A *-connectors-dependency.sh script that discovers the connector version
modules from the Gradle project (so the version matrix is not hard-coded) and
builds them:
Trino: every trino-connector-<band> module (assembleTrinoConnector).
Flink: every flink-runtime-<ver> module (Scala 2.12).
Spark: every spark-runtime-<major> module, deriving Scala variants (2.12
always; 2.13 for Spark 3.4+; Spark 4.0 is Scala 2.13 only and detected from its
build script).
A copy-connector.sh entrypoint that resolves the requested version from the
directories baked into the image at runtime.
A README.md documenting install and configuration.
Image names would follow apache/gravitino-{trino,flink,spark}-connector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
