SEPURI-SAI-KRISHNA commented on PR #13126: URL: https://github.com/apache/seatunnel/pull/13126#issuecomment-6646659102
Thanks for the review, @DanielLeens. The fork run did not go green, and the cause is the test I added, so I pushed a commit. **What happened.** In [run 41796485966](https://github.com/SEPURI-SAI-KRISHNA/seatunnel/actions/runs/41796485966) both `updated-modules-integration-test-part-1` jobs were cancelled at the 5 hour limit. `testNullPrimaryKeyFailsTheJob` passed on Flink 1.13, 1.15, 1.18 and 1.20, then stuck on Zeta on both JDKs. The first attempt failed as intended with the NULL key error. Zeta then restarted the job (`job.retry.times` defaults to 3), and the second attempt stayed RUNNING without emitting its row. The log shows `skip already ack checkpoint id: 9223372036854775807` about 4 seconds after the first attempt triggered its final barrier. **What I do not know yet.** My guess is that `CheckpointBarrierTriggerOperation` was still retrying (2 s sleep) and reached the restarted task. That comes from reading the code and the timing; I have not proven it. I could not reproduce the hang: FakeSource into a failing Assert sink on an in-process Zeta ended FAILED in 14 of 14 runs with the default retries, with no such ack. My earlier real Cassandra jobs on Zeta also ended FAILED. So it may be a timing race, but the probe is not identical: Assert writes through `MultiTableSinkWriter` and Cassandra does not. I cannot say yet whether it is specific to this sink. If it is in the engine, a real job could hang instead of failing, and I will look into that separately. **In the new commit:** - `fake_null_primary_key_to_cassandra.conf` sets `job.retry.times = 0`, as `fake_source_and_doris_sink_timeout_error.conf` does. Without a restart the job ended FAILED in under 2 s in the same in-process setup. - Your Issue 1: the test now also asserts `Primary key column id of table sink_table is NULL`, on stdout plus stderr. The Flink logs of the old run show that text in the output on all four versions. - Your Issue 2: one WARN in the constructor when the table metadata cannot be read, and a unit test that a NULL key then reaches the server. **Not verified yet.** The class stopped at that test, so `testCassandra` with the NULL row has not run in CI on any engine, and nothing has run on Spark. The new assertion on Zeta and Spark also depends on this run. Connector unit tests pass locally (34, 0 failures) and `CassandraIT` compiles. I will report back when the run ends. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
