SEPURI-SAI-KRISHNA commented on PR #13261:
URL: https://github.com/apache/seatunnel/pull/13261#issuecomment-6908829690

   @DanielLeens Thanks, both are done in `06d0de8134` and `50e1c1087a`.
   
   **Issue 2, real engine restart.** Three new cases in the existing `HttpIT`, 
on mock server paths that answer HTTP 500; each job asks Flink for 2 
fixed-delay restarts:
   
   - `testRestartedSingleSplitSourceFailsWithItsOwnError` (batch, all engines): 
every request fails; on Flink there must be 3 requests, one per attempt.
   - `testSingleSplitSourceFailingAfterACheckpointFailsWithItsOwnError` 
(streaming, Flink): 8 requests are answered, then all fail, so a checkpoint 
completes while the reader holds its split, and the restart hands back an empty 
list. It asserts a `Completed checkpoint` line for the job and 11 requests.
   - `testSingleSplitSourceRecoveringAfterTwoRestartsCompletesWithItsRow` 
(batch, Flink), the exact output: the first 2 requests fail, then one row comes 
back. The job must exit 0, the Assert sink checks exactly that row, and there 
must be 3 requests, so the split was assigned again on both restarts.
   
   The failing cases assert the HTTP error of the source in the output or 
server logs; all three assert that no `SingleSplitEnumerator.addSplitsBack` 
failure is logged.
   
   | | Flink 1.13.6 | 1.15.3 | 1.18.0 | 1.20.1 | Zeta | Spark 2.4.6 / 3.3.0 |
   | --- | --- | --- | --- | --- | --- | --- |
   | this PR | 3 of 3 pass | 3 of 3 pass | 3 of 3 pass | 3 of 3 pass | batch 
pass | batch pass |
   | dev enumerator | 3 of 3 fail | 3 of 3 fail | 3 of 3 fail | 3 of 3 fail | | 
|
   
   With the dev enumerator, each Flink JobManager log has the 
`IndexOutOfBoundsException` from `SourceCoordinator.subtaskReset`. On dev the 
first restart of a batch job looks fine, because `AbstractSingleSplitReader` 
reads without its split; the second one fails.
   
   **Issue 1, timing.** The concurrency test goes on only once the other thread 
is blocked on the enumerator's monitor (checked with `ThreadMXBean`) or has 
finished; the callback runs once and the thread is joined in `finally`. 10 of 
10 runs pass; without `synchronized` 10 of 10 fail (`expected: <1> but was: 
<2>`).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to