dongjoon-hyun commented on PR #45619: URL: https://github.com/apache/spark/pull/45619#issuecomment-2014048519
Ya, let's continue next week because the above result looks insufficient to me. I expected the following with `ENABLE_KINESIS_TESTS=1`. ``` $ build/sbt "streaming-kinesis-asl/test" -Pkinesis-asl ... [info] KinesisInputDStreamBuilderSuite: [info] - should raise an exception if the StreamingContext is missing (12 milliseconds) [info] - should raise an exception if the stream name is missing (3 milliseconds) [info] - should raise an exception if the checkpoint app name is missing (1 millisecond) [info] - should propagate required values to KinesisInputDStream (71 milliseconds) [info] - should propagate default values to KinesisInputDStream (2 milliseconds) OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appendedaming-kinesis-asl / Test / executeTests 0s [info] - should propagate custom non-auth values to KinesisInputDStream (481 milliseconds) [info] - old Api should throw UnsupportedOperationExceptionexception with AT_TIMESTAMP (2 milliseconds) [info] WithoutAggregationKinesisBackedBlockRDDSuite: [info] - Basic reading from Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available in both block manager and Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available only in block manager, not in Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available only in Kinesis, not in block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available partially in block manager, rest in Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Test isBlockValid skips block fetching from block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Test whether RDD is valid after removing blocks from block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] KinesisReceiverSuite: [info] - process records including store and set checkpointer (25 milliseconds) [info] - split into multiple processes if a limitation is set (2 milliseconds) [info] - shouldn't store and update checkpointer when receiver is stopped (2 milliseconds) [info] - shouldn't update checkpointer when exception occurs during store (4 milliseconds) [info] - shutdown should checkpoint if the reason is TERMINATE (3 milliseconds) [info] - shutdown should not checkpoint if the reason is something other than TERMINATE (2 milliseconds) [info] - retry success on first attempt (2 milliseconds) [info] - retry success on second attempt after a Kinesis throttling exception (40 milliseconds) [info] - retry success on second attempt after a Kinesis dependency exception (49 milliseconds) [info] - retry failed after a shutdown exception (3 milliseconds) [info] - retry failed after an invalid state exception (2 milliseconds) [info] - retry failed after unexpected exception (2 milliseconds) [info] - retry failed after exhausting all retries (108 milliseconds) [info] WithAggregationKinesisBackedBlockRDDSuite: [info] - Basic reading from Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available in both block manager and Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available only in block manager, not in Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available only in Kinesis, not in block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Read data available partially in block manager, rest in Kinesis [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Test isBlockValid skips block fetching from block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Test whether RDD is valid after removing blocks from block manager [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] WithAggregationKinesisStreamSuite: [info] - RDD generation (16 milliseconds) [info] - basic operation [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - custom message handling [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Kinesis read with custom configurations (2 milliseconds) [info] - split and merge shards in a stream [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - failure recovery [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Prepare KinesisTestUtils [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] KinesisCheckpointerSuite: [info] - checkpoint is not called twice for the same sequence number (3 milliseconds) [info] - checkpoint is called after sequence number increases (1 millisecond) [info] - should checkpoint if we have exceeded the checkpoint interval (2 milliseconds) [info] - shouldn't checkpoint if we have not exceeded the checkpoint interval (0 milliseconds) [info] - should not checkpoint for the same sequence number (1 millisecond) [info] - removing checkpointer checkpoints one last time (1 millisecond) [info] - if checkpointing is going on, wait until finished before removing and checkpointing (65 milliseconds) [info] SparkAWSCredentialsBuilderSuite: [info] - should build DefaultCredentials when given no params (1 millisecond) [info] - should build BasicCredentials (0 milliseconds) [info] - should build STSCredentials (2 milliseconds) [info] - SparkAWSCredentials classes should be serializable (4 milliseconds) [info] WithoutAggregationKinesisStreamSuite: [info] - RDD generation (1 millisecond) [info] - basic operation [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - custom message handling [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Kinesis read with custom configurations (1 millisecond) [info] - split and merge shards in a stream [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - failure recovery [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] - Prepare KinesisTestUtils [enable by setting env var ENABLE_KINESIS_TESTS=1] !!! IGNORED !!! [info] Test run started (JUnit Jupiter) [info] Test org.apache.spark.streaming.kinesis.JavaKinesisInputDStreamBuilderSuite#testJavaKinesisDStreamBuilderOldApi() started [info] Test org.apache.spark.streaming.kinesis.JavaKinesisInputDStreamBuilderSuite#testJavaKinesisDStreamBuilder() started [info] Test run finished: 0 failed, 0 ignored, 2 total, 0.133s [info] ScalaTest [info] Run completed in 2 seconds, 625 milliseconds. [info] Total number of tests run: 35 [info] Suites: completed 8, aborted 0 [info] Tests: succeeded 35, failed 0, canceled 0, ignored 24, pending 0 [info] All tests passed. [info] Passed: Total 37, Failed 0, Errors 0, Passed 37, Ignored 24 [success] Total time: 39 s, completed Mar 21, 2024, 4:34:09 PM ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
