lhotari commented on issue #20661: URL: https://github.com/apache/pulsar/issues/20661#issuecomment-1610764937
@tisonkun @eolivelli I found a problem in PulsarIOSinkRunner. Instead of Awaitility, it's using [Failsafe](https://failsafe.dev/). https://github.com/apache/pulsar/blob/46bb63d6bd3f119bbad0801d801915715dc061a7/tests/integration/src/test/java/org/apache/pulsar/tests/integration/io/sinks/PulsarIOSinkRunner.java#L117-L119 This is [the first exception in the log](https://github.com/apache/pulsar/actions/runs/5388118701/jobs/9782385675?pr=20654#step:11:20191): ``` 2023-06-27T11:39:06,756 - INFO - [main:TestRetrySupport@70] - Detected test failure in test PulsarSinksTest.testElasticSearch8Sink, currentSetupNumber=1 at org.testng.Assert.failNotEquals(Assert.java:1577) at org.testng.Assert.assertEqualsImpl(Assert.java:149) at org.testng.Assert.assertEquals(Assert.java:131) at org.testng.Assert.assertEquals(Assert.java:979) at org.testng.Assert.assertEquals(Assert.java:955) at org.testng.Assert.assertEquals(Assert.java:989) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.waitForProcessingSinkMessages(PulsarIOSinkRunner.java:339) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.lambda$runSinkTester$4(PulsarIOSinkRunner.java:119) at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:262) at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48) at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66) at net.jodah.failsafe.Execution.executeSync(Execution.java:128) at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379) at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:212) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.runSinkTester(PulsarIOSinkRunner.java:118) at org.apache.pulsar.tests.integration.io.PulsarIOTestBase.testSink(PulsarIOTestBase.java:40) at org.apache.pulsar.tests.integration.io.sinks.PulsarSinksTest.testElasticSearch8Sink(PulsarSinksTest.java:75) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139) at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47) at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76) at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) ``` it retries using TestRetrySupport ``` 2023-06-27T11:39:06,771 - INFO - [main:TestRetrySupport@53] - Previous test run has failed before PulsarSinksTest.testElasticSearch8Sink, failedSetupNumber=1. Running cleanup and setup. ``` and fails on the retry: ``` java.lang.AssertionError: expected [20] but found [1] at org.testng.Assert.fail(Assert.java:110) at org.testng.Assert.failNotEquals(Assert.java:1577) at org.testng.Assert.assertEqualsImpl(Assert.java:149) at org.testng.Assert.assertEquals(Assert.java:131) at org.testng.Assert.assertEquals(Assert.java:979) at org.testng.Assert.assertEquals(Assert.java:955) at org.testng.Assert.assertEquals(Assert.java:989) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.waitForProcessingSinkMessages(PulsarIOSinkRunner.java:339) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.lambda$runSinkTester$4(PulsarIOSinkRunner.java:119) at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:262) at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48) at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66) at net.jodah.failsafe.Execution.executeSync(Execution.java:128) at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379) at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:212) at org.apache.pulsar.tests.integration.io.sinks.PulsarIOSinkRunner.runSinkTester(PulsarIOSinkRunner.java:118) at org.apache.pulsar.tests.integration.io.PulsarIOTestBase.testSink(PulsarIOTestBase.java:40) at org.apache.pulsar.tests.integration.io.sinks.PulsarSinksTest.testElasticSearch8Sink(PulsarSinksTest.java:75) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139) at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47) at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76) at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) ``` This looks simply because of slow execution and that the retry solution isn't working. Failsafe isn't clearly working as expected since the log doesn't contain any lines with "Retry ... ", that should get printed by Failsafe when it retries: https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/tests/integration/src/test/java/org/apache/pulsar/tests/integration/io/PulsarIOTestRunner.java#L45-L48 I'll replace Failsafe with Awaitility since we use Awaitility in tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
