[
https://issues.apache.org/jira/browse/FLINK-41413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purushottam Sinha updated FLINK-41413:
--------------------------------------
Description:
*Problem*
* *CI is red.* On ubuntu-26.04 runners (kernel {{7.0.0-1019-azure}}),
{{mongo:8.0.32}} won't start: "Linux kernel versions 6.19 and newer has a known
incompatibility". Fail-fast then cancels the whole matrix (seen on #72, #79).
* *MongoDB 9.0 isn't tested.* It went GA on 2026-09-28, and our driver (5.11.1)
predates 9.0 support.
*Changes*
||What||Now||Target||Why||
|{{mongodb8.version}} ({{pom.xml}})|8.0.32|8.0.37+|Kernel fix (SERVER-131779)|
|{{mongodb.driver.version}} ({{pom.xml}})|5.11.1|5.14.0|Adds MongoDB 9.0
support (JAVA-6164)|
|{{mongodb9}} profile + {{push_pr.yml}} matrix|-|9.0.4+|9.0.0 to 9.0.3 have the
same kernel bug|
|SQL jar {{META-INF/NOTICE}}|5.1.1|5.14.0|Out of date since FLINK-40713|
*Blocker: official Docker images lag behind*
||Line||Fixed in||Released by MongoDB||Official {{mongo}} image||
|8.0|8.0.37|(/) 2026-10-30|(x) newest is 8.0.34|
|8.3|8.3.16|(/) 2026-10-30|(x) newest is 8.3.13|
|9.0|9.0.4|(/) 2026-09-30|(/) 9.0.4|
* docker-library/mongo hasn't updated since 2026-10-14. Last time the delay was
about 3 weeks (docker-library/mongo#772).
* Fallback: MongoDB's own image
{{mongodb/mongodb-community-server:8.0.37-ubuntu2204}} already has the fix.
Using it needs {{MongoTestUtil}} changes
({{asCompatibleSubstituteFor("mongo")}}) and a re-check of the sharded setup.
*Acceptance*
* Every {{mongodb4}} to {{mongodb9}} job passes on ubuntu-26.04 runners.
* The NOTICE matches the bundled driver versions.
*References*
* PRs: [#72|https://github.com/apache/flink-connector-mongodb/pull/72],
[#79|https://github.com/apache/flink-connector-mongodb/pull/79]
* MongoDB: [SERVER-121912|https://jira.mongodb.org/browse/SERVER-121912],
[SERVER-131779|https://jira.mongodb.org/browse/SERVER-131779],
[SERVER-136848|https://jira.mongodb.org/browse/SERVER-136848], [downloads
index|https://downloads.mongodb.org/full.json]
* Driver: [5.14.0 release
notes|https://github.com/mongodb/mongo-java-driver/releases/tag/r5.14.0]
* Docker: [docker-library/mongo
versions.json|https://github.com/docker-library/mongo/blob/master/versions.json],
[#772|https://github.com/docker-library/mongo/issues/772]
was:
*Problem*
* On the GitHub Actions ubuntu-26.04 runners (kernel {{7.0.0-1019-azure}}),
{{mongo:8.0.32}} exits at startup. The {{mongodb8}} jobs fail, and fail-fast
then cancels the rest of the matrix, so every PR shows red CI.
* MongoDB 9.0 went GA on 2026-09-28, but the connector isn't tested against it,
and its driver (5.11.1) predates 9.0 support.
*Evidence*
* {{pom.xml}} pins {{<mongodb8.version>8.0.32</mongodb8.version>}} (line 58)
and {{<mongodb.driver.version>5.11.1</mongodb.driver.version>}} (line 53). The
{{.github/workflows/push_pr.yml}} matrix (line 31) only goes up to {{mongodb8}}.
* The container log says "MongoDB cannot start: Linux kernel versions 6.19 and
newer has a known incompatibility"
([SERVER-121912|https://jira.mongodb.org/browse/SERVER-121912],
[SERVER-136848|https://jira.mongodb.org/browse/SERVER-136848]). The same
failure shows up on
[#72|https://github.com/apache/flink-connector-mongodb/pull/72] and
[#79|https://github.com/apache/flink-connector-mongodb/pull/79].
* MongoDB fixed the kernel check for Ubuntu in 8.0.37, 8.3.16 and 9.0.4
([SERVER-131779|https://jira.mongodb.org/browse/SERVER-131779]). MongoDB 9.0.0
to 9.0.3 hit the same failure, and 9.0.2 is confirmed failing in SERVER-136848.
* MongoDB released 8.0.37 on 2026-10-30 and 8.0.38 on 2026-11-05 ([downloads
index|https://downloads.mongodb.org/full.json]). The official {{mongo}} image
on Docker Hub still tops out at 8.0.34, because
[docker-library/mongo|https://github.com/docker-library/mongo/blob/master/versions.json]
hasn't updated since 2026-10-14. Last time the delay was about 3 weeks: 8.0.34
was released on 2026-09-24, and its image was published on 2026-10-13
([docker-library/mongo#772|https://github.com/docker-library/mongo/issues/772]).
The newest official 8.3 image, {{mongo:8.3.13}}, still has the old check, so
moving to 8.3 doesn't help yet.
* Driver 5.14.0 adds "JAVA-6164 Bump maxWireVersion to 29 for MongoDB 9.0"
([release
notes|https://github.com/mongodb/mongo-java-driver/releases/tag/r5.14.0]).
* {{flink-sql-connector-mongodb/src/main/resources/META-INF/NOTICE}} still
lists the driver as 5.1.1, but the jar has bundled 5.11.1 since FLINK-40713.
*Proposed fix*
* Once {{mongo:8.0.37}} (or a later 8.0.x) is on Docker Hub, raise
{{mongodb8.version}} to that version.
* Bump {{mongodb-driver-sync}} from 5.11.1 to 5.14.0, and update the SQL
connector NOTICE to match.
* Add a {{mongodb9}} profile pinned to 9.0.4 or later (never 9.0.0 to 9.0.3),
and add it to the {{push_pr.yml}} matrix.
* Fallback, if the official image takes too long: MongoDB's own image already
has the fixed version ({{mongodb/mongodb-community-server:8.0.37-ubuntu2204}}).
Switching to it means changing {{MongoTestUtil}} to use that image with
{{.asCompatibleSubstituteFor("mongo")}}, and re-checking {{MongoDBContainer}}
and the sharded-cluster setup.
*Acceptance*
* Every job from {{mongodb4}} (4.4.30) to {{mongodb9}} in {{push_pr.yml}}
passes on ubuntu-26.04 runners.
* The SQL jar NOTICE matches the bundled {{bson}}, {{mongodb-driver-core}} and
{{mongodb-driver-sync}} versions.
> Connectors / MongoDB: Bump MongoDB 8.0 to 8.0.37, mongo-driver to 5.14.0, and
> add MongoDB 9.0
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-41413
> URL: https://issues.apache.org/jira/browse/FLINK-41413
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / MongoDB
> Reporter: Purushottam Sinha
> Priority: Major
>
> *Problem*
> * *CI is red.* On ubuntu-26.04 runners (kernel {{7.0.0-1019-azure}}),
> {{mongo:8.0.32}} won't start: "Linux kernel versions 6.19 and newer has a
> known incompatibility". Fail-fast then cancels the whole matrix (seen on #72,
> #79).
> * *MongoDB 9.0 isn't tested.* It went GA on 2026-09-28, and our driver
> (5.11.1) predates 9.0 support.
> *Changes*
> ||What||Now||Target||Why||
> |{{mongodb8.version}} ({{pom.xml}})|8.0.32|8.0.37+|Kernel fix (SERVER-131779)|
> |{{mongodb.driver.version}} ({{pom.xml}})|5.11.1|5.14.0|Adds MongoDB 9.0
> support (JAVA-6164)|
> |{{mongodb9}} profile + {{push_pr.yml}} matrix|-|9.0.4+|9.0.0 to 9.0.3 have
> the same kernel bug|
> |SQL jar {{META-INF/NOTICE}}|5.1.1|5.14.0|Out of date since FLINK-40713|
> *Blocker: official Docker images lag behind*
> ||Line||Fixed in||Released by MongoDB||Official {{mongo}} image||
> |8.0|8.0.37|(/) 2026-10-30|(x) newest is 8.0.34|
> |8.3|8.3.16|(/) 2026-10-30|(x) newest is 8.3.13|
> |9.0|9.0.4|(/) 2026-09-30|(/) 9.0.4|
> * docker-library/mongo hasn't updated since 2026-10-14. Last time the delay
> was about 3 weeks (docker-library/mongo#772).
> * Fallback: MongoDB's own image
> {{mongodb/mongodb-community-server:8.0.37-ubuntu2204}} already has the fix.
> Using it needs {{MongoTestUtil}} changes
> ({{asCompatibleSubstituteFor("mongo")}}) and a re-check of the sharded setup.
> *Acceptance*
> * Every {{mongodb4}} to {{mongodb9}} job passes on ubuntu-26.04 runners.
> * The NOTICE matches the bundled driver versions.
> *References*
> * PRs: [#72|https://github.com/apache/flink-connector-mongodb/pull/72],
> [#79|https://github.com/apache/flink-connector-mongodb/pull/79]
> * MongoDB: [SERVER-121912|https://jira.mongodb.org/browse/SERVER-121912],
> [SERVER-131779|https://jira.mongodb.org/browse/SERVER-131779],
> [SERVER-136848|https://jira.mongodb.org/browse/SERVER-136848], [downloads
> index|https://downloads.mongodb.org/full.json]
> * Driver: [5.14.0 release
> notes|https://github.com/mongodb/mongo-java-driver/releases/tag/r5.14.0]
> * Docker: [docker-library/mongo
> versions.json|https://github.com/docker-library/mongo/blob/master/versions.json],
> [#772|https://github.com/docker-library/mongo/issues/772]
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
