yidaimi commented on issue #10443:
URL: https://github.com/apache/seatunnel/issues/10443#issuecomment-6734583580
@SEZ9 @DanielLeens @goutamadwant I ran the requested compatibility matrix
for the
existing `connector-mongodb` sink against a **live Amazon DocumentDB 5.0.2
TLS endpoint**
(instance-based, db.r5.large, RDS global CA bundle) with claude code.
**Materials**
- Matrix and conclusions:
[compatibility-matrix.md](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/compatibility-matrix.md)
- Evidence:
[evidence-excerpts.md](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/evidence-excerpts.md)
— for every case, the sink config, whether a JVM
truststore was used, the job result and the exact exception text, copied
from the SeaTunnel job logs (credentials, endpoints and IPs replaced by
placeholders)
- Reproduction:
[run-matrix.sh](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/run-matrix.sh)
— runs all cases against any DocumentDB cluster; credentials are
passed via environment variables
Sink: `connector-mongodb-3.0.0` from Maven Central (Mongo Java driver
4.7.1). The
connection/write path is unchanged on current `dev` (only #12687, #12399,
#12600 touched
the module since v3.0.0). Some cases were also run against a local MongoDB
7 as a control,
to separate DocumentDB-specific behavior from generic sink behavior.
**Summary**
| Question | Finding |
|---|---|
| Where the failure sits | **Trust construction only.** `tlsCAFile` and
`tlsAllowInvalidCertificates` in the URI are ignored by the Java driver (PKIX
error, T01/T02); the JDK `cacerts` has no RDS CA. Only a
JVM-global `javax.net.ssl.trustStore` works (T03), which affects every job
in the process. |
| Endpoint validation | Works as expected: cluster DNS names pass (T03);
IP fails with a SAN mismatch (T04); `tlsAllowInvalidHostnames=true` is honoured
(T05). |
| Auth | SCRAM-SHA-1 and SCRAM-SHA-256 both work (T08/T09); wrong password
gives error 18 (T07). |
| Write semantics | Insert / update / delete (upsert and non-upsert)
produce correct results and BSON types are preserved (T11/T12); `DROP_DATA` and
`ERROR_WHEN_DATA_EXISTS` behave as documented (X4/T15). |
| DocumentDB source difference | It builds a per-job `SSLContext` from
`tls_ca_file` and forces `retryWrites(false)`; the sink uses
`MongoClients.create(uri)` with no hook for either. |
So the matrix confirms the gap is **per-job trust configuration**.
Following the guidance
above, I'd propose a backward-compatible MongoDB sink extension: an
optional `tls_ca_file`
building a per-job `SSLContext` (same approach as the DocumentDB source),
hostname
verification left on by default, no trust-all, no new dependency.
One unrelated finding, which I'd report separately:
`buffer-flush.max-rows` /
`buffer-flush.interval` defaults are not applied when the options are
omitted
(`flushSize` stays 0), so the sink issues one round trip per row. Against
DocumentDB this
was 56 s vs 3 s of engine time for 20,000 inserts (P01/P01b). It is not
DocumentDB-specific
and affects any remote MongoDB.
Does this scope look right before I open the implementation PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]