LuciferYang commented on PR #42581: URL: https://github.com/apache/spark/pull/42581#issuecomment-1685925739
I have two concerns about continuing to maintain this module: 1. This module has 60 UTs, but 24 are ignored due to the environment variable `ENABLE_KINESIS_TESTS` defaulting to 0. ``` [info] Tests: succeeded 34, failed 0, canceled 0, ignored 24, pending 0 [info] All tests passed. ``` However, when I run `ENABLE_KINESIS_TESTS=1 build/sbt "streaming-kinesis-asl/test" -Pkinesis-asl`, these previously ignored test cases fail. ``` [info] *** 4 SUITES ABORTED *** [error] Error: Total 36, Failed 0, Errors 4, Passed 32 [error] Error during tests: [error] org.apache.spark.streaming.kinesis.WithoutAggregationKinesisBackedBlockRDDSuite [error] org.apache.spark.streaming.kinesis.WithAggregationKinesisBackedBlockRDDSuite [error] org.apache.spark.streaming.kinesis.WithAggregationKinesisStreamSuite [error] org.apache.spark.streaming.kinesis.WithoutAggregationKinesisStreamSuite [error] (streaming-kinesis-asl / Test / test) sbt.TestsFailedException: Tests unsuccessful ``` 2. The CI does not provide sufficient check for this module. https://github.com/apache/spark/blob/a21e19b6e7ac4c4c77b39d93a2da2cbe1c88c4c8/.github/workflows/build_and_test.yml#L139 So, I think we should complete the TODO SPARK-32246 before proceeding with this update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
