xgbtxy commented on PR #59763: URL: https://github.com/apache/spark/pull/59763#issuecomment-6380461213
CI follow-up for `57cc12c`: the [failed module-group artifact](https://github.com/xgbtxy/spark/actions/runs/40057615003/artifacts/12548206165) has one failing test, `KafkaRealTimeModeAggregationSuite` / `tumbling window avg`. Its stack points to the [batch-progress check after `clock.advance(1000)`](https://github.com/xgbtxy/spark/blob/57cc12c53ccceaf0a22d6b3864a684809429a853/connector/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaRealTimeModeAggregationSuite.scala#L254-L266): after 60 seconds, batchId was still 0 rather than 1. The preceding Kafka output assertion had returned normally. The input is a low-latency memory stream; its [scan builder](https://github.com/xgbtxy/spark/blob/57cc12c53ccceaf0a22d6b3864a684809429a853/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/memory.scala#L166) does not implement `SupportsPushDownAggregates`, which is required by the changed partial-aggregate rewrite. The Kafka readback likewise does not support aggregate pushdown. As a comparison, [run 40058768848](https://github.com/xgbtxy/spark/actions/runs/40058768848), against the same Apache base `0aa4f913`, passed all four tests in this Kafka suite. This comparison does not establish the cause of the timeout. After the parent workflow finished, I requested one rerun of the failed job on the unchanged commit. [Attempt 2](https://github.com/xgbtxy/spark/actions/runs/40057615003/attempts/2) is now running. The original failure and comparison artifacts are preserved. Analysis assisted by Codex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
