goutamadwant opened a new pull request, #12472: URL: https://github.com/apache/seatunnel/pull/12472
### Purpose of this pull request `FileCollectReaderBehaviorTest.rediscoversFileAfterInactiveCursorClosed` fails intermittently on Windows unit-test jobs with: ``` FileCollectReaderBehaviorTest.rediscoversFileAfterInactiveCursorClosed:101 » ConditionTimeout Caused by: org.opentest4j.AssertionFailedError: expected: <1> but was: <0> ``` - `unit-test (8, windows-latest)`, #12305: https://github.com/goutamadwant/seatunnel/actions/runs/35673549024/job/106584161477 - `unit-test (11, windows-latest)`, #12258: https://github.com/merlau/seatunnel/actions/runs/35990368552/job/107605215163 **Why it fails.** The test infers two internal transitions from wall-clock arithmetic with 20–30 ms of slack: the inactive-cursor close (`close_inactive_ms = 500`) and the glob rediscovery (`glob_scan_interval_ms = 20`). Inside one `poll()`, `FileCollectReader` runs discovery before it closes idle cursors, so rediscovery needs a later poll at least one scan interval after the closing poll. A single stall of a few tens of milliseconds near the idle deadline pushes the closing poll late. The second Awaitility loop then exits before the next scan is due, and `"second\n"` is appended while the file is still untracked. With `read_from_beginning = false` and no stored position, the rediscovered file is opened at its end, so the new line is skipped. The final `untilAsserted` then times out with `expected: <1> but was: <0>`. A longer timeout cannot help, because the line is already behind the cursor. **The change.** - `FileCollectReader` gets a package-private constructor that takes a millisecond clock (`LongSupplier`). It uses that clock for the glob scan interval, idle-cursor bookkeeping and line timestamps, and passes it to `FileTailCursor`. - `FileTailCursor` gets a matching constructor overload. It has to be public, because `FileTailCursor` lives in the `cursor` subpackage. - The existing constructors delegate with `System::currentTimeMillis`, so runtime behaviour does not change. The `updatedAt` value written to the position store still uses wall-clock time. - The test steps a manual clock past `close_inactive_ms` and then one `glob_scan_interval_ms`, so each transition happens in a known poll. All the original assertions on the emitted event are kept: exactly one event, which contains `second` and not `first`. - The test also asserts that the event's `line` metadata is `1`. Line numbering only restarts if the cursor was really closed and the file rediscovered, so the test no longer passes when the idle close does nothing. This overlaps #12381, which raises the same test's Awaitility budgets from 3 s to 10 s. In the failing runs, the time is lost before the append, not in the final wait, so the larger budget does not cover this case. If #12381 merges first, I will rebase. ### Does this PR introduce _any_ user-facing change? No. The new constructors are for tests, and the default clock is `System::currentTimeMillis`, as before. ### How was this patch tested? **Forced timing window on unchanged `dev` (`deb16a3c3`).** A local-only hook, not in this PR, inserted a single 45 ms `Thread.sleep` in the first Awaitility loop, 35 ms before the idle deadline. Each run executed the single method repeatedly in one JVM through the JUnit Platform launcher: | Code | JDK 8u172 | JDK 11.0.19 | |---|--:|--:| | Unchanged `dev`, with hook | 15/20 failed | 7/20 failed | | This patch, stalls injected at the same points | 0/20 failed | 0/20 failed | On `dev`, every failure had the CI message (`expected: <1> but was: <0>` within 3 seconds). Without the hook, `dev` passed 50/50 per JDK on macOS under CPU contention. That fits a failure that needs a stall of about 20 ms or more at a specific moment, which Windows runners hit more often. For the patched run, the injected stalls were 45 ms before the close step, 700 ms after the close, and 700 ms after rediscovery. The 700 ms stalls are longer than `close_inactive_ms`, which would re-close the cursor under a wall clock. **Stress after the patch.** 200/200 passed on JDK 8 and 200/200 on JDK 11, with 20 busy-loop processes competing for CPU. **Mutation checks.** The patched test fails as expected in both cases: - `globResolver.forget(...)` removed from `closeInactiveCursors`, so the file is never rediscovered. - `closeInactiveCursors(now)` removed from `poll()`, so the cursor is never closed. **Module builds.** `./mvnw -B -o -pl seatunnel-edge-agent/seatunnel-edge-agent-connector,seatunnel-edge-agent/seatunnel-edge-agent-starter clean verify -DskipUT=false -DskipIT=true` passed on JDK 8 and JDK 11, including spotless: connector 38 tests and starter 103 tests, with 0 failures. **Not run locally:** Windows, and `seatunnel-edge-agent-e2e` (`EdgeAgentFileToEngineIT`). The E2E suite needs the Docker engine setup; it runs in this PR's `edge-agent-it` job. ### Check list * [ ] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md) * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR. * [ ] If you are contributing the connector code, please check that the following files are updated: 1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it 2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml) 3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml) 4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/) 5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
