yidaimi opened a new issue, #13198:
URL: https://github.com/apache/seatunnel/issues/13198

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
 and found no similar issues.
   
   ### What happened
   
   `MongodbSinkOptions` declares defaults for four sink options, but 
`MongodbSinkFactory` only passes a value to `MongodbWriterOptions.Builder` when 
the option is explicitly present:
   
   ```java
   // MongodbSinkFactory (dev, lines 78-96)
   if 
(readonlyConfig.getOptional(MongodbSinkOptions.BUFFER_FLUSH_MAX_ROWS).isPresent())
 {
       
builder.withFlushSize(readonlyConfig.get(MongodbSinkOptions.BUFFER_FLUSH_MAX_ROWS));
   }
   // same pattern for BUFFER_FLUSH_INTERVAL, RETRY_MAX and RETRY_INTERVAL
   ```
   
   When an option is omitted, the builder keeps the Java field defaults instead 
of the declared ones:
   
   | Option | Declared default | Effective value when omitted |
   |---|---|---|
   | `buffer-flush.max-rows` | 1000 | 0 (`protected int flushSize;`) |
   | `buffer-flush.interval` | 30000 ms | 0 (`protected long batchIntervalMs;`) 
|
   | `retry.max` | 3 | 0 (`protected int retryMax;`) |
   | `retry.interval` | 1000 ms | 0 (`protected long retryInterval;`) |
   
   Consequences:
   
   1. With `flushSize = 0`, `MongodbWriter.isOverMaxBatchSizeLimit()` evaluates 
`bulkRequests.size() >= 0`, which is always true, so `doBulkWrite()` runs after 
**every row**: one network round trip per row instead of one per 1000 rows.
   2. With `retryMax = 0`, a failed bulk write is not retried, although the 
documented default is 3.
   
   The first point is invisible against a local server and severe against a 
remote one. Measured with the 3.0.0 sink, 20,000 inserts, nothing else changed:
   
   | Target | Options omitted | `buffer-flush.max-rows = 1000` set explicitly |
   |---|---|---|
   | Amazon DocumentDB 5.0.2 (same VPC, ~3 ms RTT) | 56 s engine time | 3 s 
engine time |
   | Local MongoDB 7 (control) | 5 s engine time | - |
   
   Evidence (cases P01 / P01b / X1a) and the reproduction script: 
https://github.com/yidaimi/seatunnel-cosmos-to-documentdb/blob/v0.2-delete_insert/test/sink-compat/evidence-excerpts.md
   
   The issue was found while preparing the DocumentDB compatibility matrix for 
#10443, but it is not DocumentDB-specific: it affects every MongoDB sink job 
that does not set these options.
   
   **Expected behavior:** omitted options use the declared defaults (1000 rows, 
30000 ms, 3 retries, 1000 ms); explicitly configured values keep working as 
today.
   
   **Proposed fix:** read the four options with `readonlyConfig.get(...)` 
unconditionally (the `Option` already carries its default), so the builder 
always receives either the configured or the declared value. Regression test: 
create the sink without any of the four options and verify the writer behaves 
with 1000 / 30000 / 3 / 1000, plus a control case with explicit values.
   
   ### SeaTunnel Version
   
   3.0.0. The factory code is unchanged on current `dev` (checked at 
b7ef4936fe).
   
   ### SeaTunnel Config
   
   ```conf
   env {
     parallelism = 1
     job.mode = "BATCH"
   }
   source {
     FakeSource {
       row.num = 20000
       schema = { fields { id = bigint, name = string } }
     }
   }
   sink {
     MongoDB {
       uri = "mongodb://<user>:<password>@<host>:27017/?retryWrites=false"
       database = "test"
       collection = "flush_defaults"
       # buffer-flush.max-rows / buffer-flush.interval / retry.max / 
retry.interval omitted
     }
   }
   ```
   
   ### Running Command
   
   ```shell
   ./bin/seatunnel.sh --config flush_defaults.conf -m local
   ```
   
   ### Error Exception
   
   ```log
   No exception. The job finishes, but writes one row per bulkWrite call (see 
the timings above) and does not retry failed bulk writes.
   ```
   
   ### Zeta or Flink or Spark Version
   
   Zeta (SeaTunnel 3.0.0)
   
   ### Java or Scala Version
   
   OpenJDK 17 (Amazon Corretto 17.0.16)
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to