Savonitar commented on code in PR #72: URL: https://github.com/apache/flink-connector-mongodb/pull/72#discussion_r4342999060
########## AGENTS.md: ########## @@ -0,0 +1,216 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Flink MongoDB Connector AI Agent Instructions + +This file provides guidance for AI coding agents working with the Apache Flink MongoDB connector codebase. + +## Prerequisites + +- Java 11, 17, or 21 (the `main` branch build matrix; release branch `v2.0` also supports Java 8) +- Maven 3.8.6 (there is no Maven wrapper in this repository; use `mvn` directly) +- Git +- Docker (every `*ITCase` starts MongoDB through Testcontainers, including sharded-cluster tests via `MongoShardedContainers`) +- Unix-like environment (Linux, macOS) +- The connector builds against `flink.version` in the root `pom.xml`. A change must also work against every Flink version in `.github/workflows/push_pr.yml` and the `weekly.yml` matrix. +- MongoDB server compatibility is tested via Maven profiles (`mongodb4`, `mongodb5`, `mongodb6`, `mongodb7`, each pinning `mongodb.version` to a specific server release); CI runs the full test suite once per profile. + +## Commands + +### Build + +- Build without tests: `mvn clean package -DskipTests` +- Full build with tests: `mvn clean verify` +- Build against a specific MongoDB server version (what CI does): `mvn clean verify -Pmongodb7` +- Build against another Flink version: `mvn clean verify -Dflink.version=<version>` +- Single module: `mvn clean install -DskipTests -pl flink-connector-mongodb` +- Skip non-essential checks while iterating: `-Dspotless.check.skip=true -Dcheckstyle.skip=true -Drat.skip=true` +- Dependency convergence check (not run by default — `dependency-convergence` is bound to `<phase>none</phase>` in `flink-connector-parent` "as it interacts badly with shade-plugin", and this repository's CI does not invoke it either): `mvn -pl <module> org.apache.maven.plugins:maven-enforcer-plugin:enforce -Denforcer.rules=dependencyConvergence` Review Comment: Should we distinguish between Weekly CI and PR CI? ########## AGENTS.md: ########## @@ -0,0 +1,216 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Flink MongoDB Connector AI Agent Instructions + +This file provides guidance for AI coding agents working with the Apache Flink MongoDB connector codebase. + +## Prerequisites + +- Java 11, 17, or 21 (the `main` branch build matrix; release branch `v2.0` also supports Java 8) +- Maven 3.8.6 (there is no Maven wrapper in this repository; use `mvn` directly) +- Git +- Docker (every `*ITCase` starts MongoDB through Testcontainers, including sharded-cluster tests via `MongoShardedContainers`) +- Unix-like environment (Linux, macOS) +- The connector builds against `flink.version` in the root `pom.xml`. A change must also work against every Flink version in `.github/workflows/push_pr.yml` and the `weekly.yml` matrix. +- MongoDB server compatibility is tested via Maven profiles (`mongodb4`, `mongodb5`, `mongodb6`, `mongodb7`, each pinning `mongodb.version` to a specific server release); CI runs the full test suite once per profile. + +## Commands + +### Build + +- Build without tests: `mvn clean package -DskipTests` +- Full build with tests: `mvn clean verify` +- Build against a specific MongoDB server version (what CI does): `mvn clean verify -Pmongodb7` +- Build against another Flink version: `mvn clean verify -Dflink.version=<version>` +- Single module: `mvn clean install -DskipTests -pl flink-connector-mongodb` +- Skip non-essential checks while iterating: `-Dspotless.check.skip=true -Dcheckstyle.skip=true -Drat.skip=true` +- Dependency convergence check (not run by default — `dependency-convergence` is bound to `<phase>none</phase>` in `flink-connector-parent` "as it interacts badly with shade-plugin", and this repository's CI does not invoke it either): `mvn -pl <module> org.apache.maven.plugins:maven-enforcer-plugin:enforce -Denforcer.rules=dependencyConvergence` + +### Testing + +- `*Test` classes run in the `test` phase, `*ITCase` classes run in the `integration-test` phase. +- Single unit test class: `mvn test -pl flink-connector-mongodb -Dtest=MongoWriterTest` Review Comment: > MongoWriterTest Could we use an existing unit test in this example? There is no MongoWriterTest in this repo, so the current command selects no test. ########## AGENTS.md: ########## @@ -0,0 +1,216 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Flink MongoDB Connector AI Agent Instructions + +This file provides guidance for AI coding agents working with the Apache Flink MongoDB connector codebase. + +## Prerequisites + +- Java 11, 17, or 21 (the `main` branch build matrix; release branch `v2.0` also supports Java 8) +- Maven 3.8.6 (there is no Maven wrapper in this repository; use `mvn` directly) +- Git +- Docker (every `*ITCase` starts MongoDB through Testcontainers, including sharded-cluster tests via `MongoShardedContainers`) +- Unix-like environment (Linux, macOS) +- The connector builds against `flink.version` in the root `pom.xml`. A change must also work against every Flink version in `.github/workflows/push_pr.yml` and the `weekly.yml` matrix. +- MongoDB server compatibility is tested via Maven profiles (`mongodb4`, `mongodb5`, `mongodb6`, `mongodb7`, each pinning `mongodb.version` to a specific server release); CI runs the full test suite once per profile. + +## Commands + +### Build + +- Build without tests: `mvn clean package -DskipTests` +- Full build with tests: `mvn clean verify` +- Build against a specific MongoDB server version (what CI does): `mvn clean verify -Pmongodb7` +- Build against another Flink version: `mvn clean verify -Dflink.version=<version>` +- Single module: `mvn clean install -DskipTests -pl flink-connector-mongodb` +- Skip non-essential checks while iterating: `-Dspotless.check.skip=true -Dcheckstyle.skip=true -Drat.skip=true` +- Dependency convergence check (not run by default — `dependency-convergence` is bound to `<phase>none</phase>` in `flink-connector-parent` "as it interacts badly with shade-plugin", and this repository's CI does not invoke it either): `mvn -pl <module> org.apache.maven.plugins:maven-enforcer-plugin:enforce -Denforcer.rules=dependencyConvergence` + +### Testing + +- `*Test` classes run in the `test` phase, `*ITCase` classes run in the `integration-test` phase. +- Single unit test class: `mvn test -pl flink-connector-mongodb -Dtest=MongoWriterTest` +- Single ITCase: `mvn verify -pl flink-connector-mongodb -Dtest=MongoSinkITCase` +- ArchUnit rules: `mvn test -pl flink-connector-mongodb -Dtest='*ArchitectureTest'`, then check `git status flink-connector-mongodb/archunit-violations/` (see Testing Standards) +- MongoDB in tests comes from `MongoTestUtil` (`createMongoDBContainer()`, `createMongoDBShardedContainers()`), which reads the target server version from the `mongodb.version` system property — set by whichever `mongodbN` profile is active, not hardcoded in test code. +- CI is defined by `apache/flink-connector-shared-utils` (`.github/workflows/ci.yml@ci_utils`); its Maven command line is the reference when a local run differs from CI. + +### Code Quality + +- Format code: `mvn spotless:apply` (skipped automatically on JDK 21 — google-java-format does not run there; format on 11 or 17) +- Check formatting: `mvn spotless:check` +- Checkstyle: `mvn checkstyle:check` (config: `tools/maven/checkstyle.xml`, suppressions: `tools/maven/suppressions.xml`) +- License headers: `mvn apache-rat:check` +- `japicmp` compares against `japicmp.referenceVersion` from the root `pom.xml` (`<connector version>-<flink version>` format) and only checks `@Public`/`@PublicEvolving` API. + +### Documentation + +- `docs/content/docs/connectors/` and `docs/content.zh/docs/connectors/` hold the DataStream and Table docs (English and Chinese, same file set). Option tables are hand-written HTML rows — nothing is generated from `ConfigOption` definitions. +- `docs/data/mongodb.yml` only declares the connector's release `version`/`variants` metadata consumed by the Flink docs site build; it is not a source of table option content. + +## Repository Structure + +### Modules + +- `flink-connector-mongodb` — The connector: `MongoSource` (FLIP-27, `Boundedness.BOUNDED` only — this is a batch/bounded source, there is no CDC or unbounded streaming mode), `MongoSink` (Sink V2), and the Table/SQL factory (`MongoDynamicTableFactory`). +- `flink-sql-connector-mongodb` — Shaded SQL jar; relocates `com.mongodb`/`org.bson`. +- `flink-connector-mongodb-e2e-tests` — End-to-end tests against a real Flink distribution. +- `flink-connector-mongodb-table-tests` — Table/SQL-layer integration tests kept in their own module. + +There is no `flink-python` module in this repository. + +### Key packages in `flink-connector-mongodb/src/main/java` + +- `org.apache.flink.connector.mongodb.source` — `@PublicEvolving`: `MongoSource`, `MongoSourceBuilder`. `source.enumerator` / `source.reader` / `source.split` are `@Internal`. + - `source.enumerator.splitter` implements the partitioning strategies (`PartitionStrategy`, `@PublicEvolving`): `SINGLE` (no split), `SAMPLE` (random-sample based, the default), `SPLIT_VECTOR` (uses the `splitVector` command, standalone/replica-set only), `SHARDED` (reads chunk boundaries from `config.chunks`), `PAGINATION` (skip/limit based, works everywhere including Atlas). Each has its own `MongoXxxSplitter`. Review Comment: > SAMPLE Should we include `DEFAULT` here and describe it as the default instead of `SAMPLE`? https://github.com/apache/flink-connector-mongodb/blob/main/flink-connector-mongodb/src/main/java/org/apache/flink/connector/mongodb/table/MongoConnectorOptions.java#L76 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
