yidaimi commented on issue #10443:
URL: https://github.com/apache/seatunnel/issues/10443#issuecomment-6734583580

     @SEZ9 @DanielLeens @goutamadwant I ran the requested compatibility matrix 
for the
     existing `connector-mongodb` sink against a **live Amazon DocumentDB 5.0.2 
TLS endpoint**
     (instance-based, db.r5.large, RDS global CA bundle) with claude code.
   
     **Materials**
   
     - Matrix and conclusions: 
[compatibility-matrix.md](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/compatibility-matrix.md)
     - Evidence: 
[evidence-excerpts.md](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/evidence-excerpts.md)
 — for every case, the sink config, whether a JVM
     truststore was used, the job result and the exact exception text, copied 
from the SeaTunnel job logs (credentials, endpoints and IPs replaced by 
placeholders)
     - Reproduction: 
[run-matrix.sh](https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/run-matrix.sh)
 — runs all cases against any DocumentDB cluster; credentials are
     passed via environment variables
   
     Sink: `connector-mongodb-3.0.0` from Maven Central (Mongo Java driver 
4.7.1). The
     connection/write path is unchanged on current `dev` (only #12687, #12399, 
#12600 touched
     the module since v3.0.0). Some cases were also run against a local MongoDB 
7 as a control,
     to separate DocumentDB-specific behavior from generic sink behavior.
   
     **Summary**
   
     | Question | Finding |
     |---|---|
     | Where the failure sits | **Trust construction only.** `tlsCAFile` and 
`tlsAllowInvalidCertificates` in the URI are ignored by the Java driver (PKIX 
error, T01/T02); the JDK `cacerts` has no RDS CA. Only a
     JVM-global `javax.net.ssl.trustStore` works (T03), which affects every job 
in the process. |
     | Endpoint validation | Works as expected: cluster DNS names pass (T03); 
IP fails with a SAN mismatch (T04); `tlsAllowInvalidHostnames=true` is honoured 
(T05). |
     | Auth | SCRAM-SHA-1 and SCRAM-SHA-256 both work (T08/T09); wrong password 
gives error 18 (T07). |
     | Write semantics | Insert / update / delete (upsert and non-upsert) 
produce correct results and BSON types are preserved (T11/T12); `DROP_DATA` and 
`ERROR_WHEN_DATA_EXISTS` behave as documented (X4/T15). |
     | DocumentDB source difference | It builds a per-job `SSLContext` from 
`tls_ca_file` and forces `retryWrites(false)`; the sink uses 
`MongoClients.create(uri)` with no hook for either. |
   
     So the matrix confirms the gap is **per-job trust configuration**. 
Following the guidance
     above, I'd propose a backward-compatible MongoDB sink extension: an 
optional `tls_ca_file`
     building a per-job `SSLContext` (same approach as the DocumentDB source), 
hostname
     verification left on by default, no trust-all, no new dependency.
   
     One unrelated finding, which I'd report separately: 
`buffer-flush.max-rows` /
     `buffer-flush.interval` defaults are not applied when the options are 
omitted
     (`flushSize` stays 0), so the sink issues one round trip per row. Against 
DocumentDB this
     was 56 s vs 3 s of engine time for 20,000 inserts (P01/P01b). It is not 
DocumentDB-specific
     and affects any remote MongoDB.
   
     Does this scope look right before I open the implementation PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to