[
https://issues.apache.org/jira/browse/SCB-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liubao updated SCB-2718:
------------------------
Fix Version/s: java-chassis-2.9.0
> Fix flaky test failure in TestOperationGenerator#apiOperationThenResponse
> --------------------------------------------------------------------------
>
> Key: SCB-2718
> URL: https://issues.apache.org/jira/browse/SCB-2718
> Project: Apache ServiceComb
> Issue Type: Bug
> Reporter: Mohsen Dehghankar
> Priority: Major
> Fix For: java-chassis-2.9.0
>
>
> h3. What is the purpose of this PR
> - This PR fixes the flaky test
> `org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse`
> - The mentioned test may fail or pass without changes made to the source code
> when it is run in different JVMs due to the non-deterministic order of the
> returned array from `getDeclaredAnnotations`. Because there is no assumption
> on the order of the returned array from this method in [Java
> documentation|https://docs.oracle.com/javase/8/docs/api/java/lang/reflect/AccessibleObject.html#getDeclaredAnnotations--].
> h3. Why the test fails
> - When processing the annotations of a method in
> `AbstractOperationGenerator#scanMethodAnnotations()`, the result of
> `method.getAnnotations()` is used which uses
> `AccessibleObject.getDeclaredAnnotations()` which does not guarantee a
> deterministic result order based on Java specifications.
> - This results in processing annotations in arbitrary order, so the expected
> assertions in both `TestOperationGenerator#apiOperationThenResponse` and
> `TestOperationGenerator#responseThenApiOperation` might fail due to this
> change in orders of annotations.
> h3. Reproduce the test failure
> - Run the test with 'NonDex' maven plugin. The command to recreate the flaky
> test failure is `mvn edu.illinois:nondex-maven-plugin:2.1-SNAPSHOT:nondex -pl
> swagger/swagger-generator/generator-core
> -Dtest=TestOperationGenerator#apiOperationThenResponse`
> - Fixing the flaky test now may prevent flaky test failures in future Java
> versions.
> h3. Expected Result
> - The tests should run successfully when run with 'NonDex' maven with the
> same command for recreating the flaky test.
> h3. Actual Result
> - We get the following error:
> ```
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.258
> s <<< FAILURE! - in
> org.apache.servicecomb.swagger.generator.core.TestOperationGenerator
> [ERROR]
> org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse
> Time elapsed: 0.25 s <<< FAILURE!
> org.opentest4j.AssertionFailedError: expected: <null> but was:
> <io.swagger.models.properties.StringProperty@8f5d1e11>
> at
> org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
> at
> org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
> at org.junit.jupiter.api.AssertNull.failNotNull(AssertNull.java:50)
> at org.junit.jupiter.api.AssertNull.assertNull(AssertNull.java:35)
> at org.junit.jupiter.api.AssertNull.assertNull(AssertNull.java:30)
> at org.junit.jupiter.api.Assertions.assertNull(Assertions.java:276)
> at
> org.apache.servicecomb.swagger.generator.core.TestOperationGenerator.apiOperationThenResponse(TestOperationGenerator.java:114)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> at
> org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
> at
> org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
> at
> org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
> at
> org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
> at
> org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
> at
> org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
> at
> org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
> at
> org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:55)
> at
> org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:223)
> at
> org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:175)
> at
> org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:139)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
> ```
> h3. Fix
> - First, we should sort the `method.getAnnotations()` result
> deterministically.
> - Secondly, there should be no different in the `assertEqual` expressions
> between these two tests: `apiOperationThenResponse` and
> `responseThenApiOperation`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
