yidaimi commented on PR #13200: URL: https://github.com/apache/seatunnel/pull/13200#issuecomment-6937677685
@DanielLeens Thanks for the second pass. Both blockers are addressed in `2fb8bfe41`. The production change is unchanged. **1. Idle-flush wording** (`incompatible-changes.md`, en and zh). The note no longer promises a 30-second deadline. It now says that the first row of a writer is written at once, and that the 30-second interval is checked only when a new row arrives, so it does not bound how long a partial batch waits. Buffered rows are written at the next checkpoint, at close, and at the engine timer flush when `sink.flush.interval` is set in `env` (Zeta only, disabled by default). The migration guide adds `sink.flush.interval` or a lower `buffer-flush.max-rows` for streaming jobs that need bounded visibility. The PR description is corrected the same way. **2. Integration coverage.** `MongodbIT#testSinkWithOmittedBufferAndRetryOptions` runs a FakeSource job with 1234 rows and none of the four options. It checks the exact target row count, which includes a 233-row tail below the default batch size written at job end, and the number of `insert` commands from `serverStatus` (one per bulk write). An explicit-option job (`max-rows = 1`, `interval = -1`, `retry.max = 0`) is kept as control. | Job | Unmodified `dev` | This PR | |---|---|---| | options omitted | 1234 rows, 1234 insert commands (fails) | 1234 rows, 3 insert commands (1 + 1000 + 233) | | explicit control | 1234 rows, 1234 insert commands | 1234 rows, 1234 insert commands | **UT timing (non-blocking).** `MongodbSinkOptionDefaultsTest` now sets the writer's last-send time explicitly: far in the future to isolate the size threshold, or 25 s / 30 s in the past for the interval. The retry back-off is measured between attempts instead of in total. Explicit-interval and explicit-retry-interval controls are added, and the writers are closed after the counts are taken. On unmodified `dev` the four omitted-option tests fail and the four controls pass. The fork Build on `2fb8bfe41` is green: https://github.com/yidaimi/seatunnel/actions/runs/43833403116. In `updated-modules-integration-test-part-1` (`RUN_ENGINES` flink, spark, zeta), the new test ran on all seven containers (Zeta, Flink 1.13 / 1.15 / 1.18 / 1.20, Spark 2.4 / 3.3), each logged `3 insert commands for 1234 rows`, and `MongodbIT` passed with 96 tests. Unit tests pass on ubuntu and windows with JDK 11 and 17. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
