[jira] [Closed] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same env
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Mallette closed TINKERPOP-3079. --- Fix Version/s: 4.0.0 3.6.8 3.7.3 Resolution: Fixed > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Components: structure >Affects Versions: 3.6.7 > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Assignee: Stephen Mallette >Priority: Minor > Fix For: 4.0.0, 3.6.8, 3.7.3 > > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first execution of the test > `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, > `StrategyA` is removed from global > strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), > and the test does not restore it after execution. > Therefore, in the second execution of the test, `StrategyA` is not present in > the global strategies, so assertions like > `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would > fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). > A fix is necessary since unit tests shall be self-contained. Idempotent tests > help maintain this isolation by ensuring that the state of the system under > test is consistent at the beginning of each test, regardless of previous test > runs. For example, fixing non-idempotent tests can help proactively avoid > state pollution that results in test order dependency (which could hurt > regression testing with the use of test selection / prioritization / > parallelism. > ### Failure Message in the 2nd Test Run: > ``` > java.lang.AssertionError: > at org.junit.Assert.fail(Assert.java:87) > at org.junit.Assert.assertTrue(Assert.java:42) > at org.junit.Assert.assertTrue(Assert.java:53) > at > org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) > ``` > ### Reproduce > Use the `NIOInspector` plugin that supports rerunning individual tests in the > same environment: > ``` > cd gremlin-core > mvn edu.illinois:NIOInspector:rerun > -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache > ``` > ### Proposed Fix > Handle initial strategies registration in a `setup()` method rather than in > static blocks of test classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847295#comment-17847295 ] ASF GitHub Bot commented on TINKERPOP-3079: --- spmallette closed pull request #2608: TINKERPOP-3079 Fixed non-idempotent unit test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` URL: https://github.com/apache/tinkerpop/pull/2608 > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Priority: Major > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first execution of the test > `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, > `StrategyA` is removed from global > strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), > and the test does not restore it after execution. > Therefore, in the second execution of the test, `StrategyA` is not present in > the global strategies, so assertions like > `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would > fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). > A fix is necessary since unit tests shall be self-contained. Idempotent tests > help maintain this isolation by ensuring that the state of the system under > test is consistent at the beginning of each test, regardless of previous test > runs. For example, fixing non-idempotent tests can help proactively avoid > state pollution that results in test order dependency (which could hurt > regression testing with the use of test selection / prioritization / > parallelism. > ### Failure Message in the 2nd Test Run: > ``` > java.lang.AssertionError: > at org.junit.Assert.fail(Assert.java:87) > at org.junit.Assert.assertTrue(Assert.java:42) > at org.junit.Assert.assertTrue(Assert.java:53) > at > org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) > ``` > ### Reproduce > Use the `NIOInspector` plugin that supports rerunning individual tests in the > same environment: > ``` > cd gremlin-core > mvn edu.illinois:NIOInspector:rerun > -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache > ``` > ### Proposed Fix > Handle initial strategies registration in a `setup()` method rather than in > static blocks of test classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same e
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Mallette reassigned TINKERPOP-3079: --- Component/s: structure Affects Version/s: 3.6.7 Assignee: Stephen Mallette Priority: Minor (was: Major) > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Components: structure >Affects Versions: 3.6.7 > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Assignee: Stephen Mallette >Priority: Minor > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first execution of the test > `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, > `StrategyA` is removed from global > strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), > and the test does not restore it after execution. > Therefore, in the second execution of the test, `StrategyA` is not present in > the global strategies, so assertions like > `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would > fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). > A fix is necessary since unit tests shall be self-contained. Idempotent tests > help maintain this isolation by ensuring that the state of the system under > test is consistent at the beginning of each test, regardless of previous test > runs. For example, fixing non-idempotent tests can help proactively avoid > state pollution that results in test order dependency (which could hurt > regression testing with the use of test selection / prioritization / > parallelism. > ### Failure Message in the 2nd Test Run: > ``` > java.lang.AssertionError: > at org.junit.Assert.fail(Assert.java:87) > at org.junit.Assert.assertTrue(Assert.java:42) > at org.junit.Assert.assertTrue(Assert.java:53) > at > org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) > ``` > ### Reproduce > Use the `NIOInspector` plugin that supports rerunning individual tests in the > same environment: > ``` > cd gremlin-core > mvn edu.illinois:NIOInspector:rerun > -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache > ``` > ### Proposed Fix > Handle initial strategies registration in a `setup()` method rather than in > static blocks of test classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847296#comment-17847296 ] ASF GitHub Bot commented on TINKERPOP-3079: --- spmallette commented on PR #2608: URL: https://github.com/apache/tinkerpop/pull/2608#issuecomment-2117619969 merged to 3.6-dev with 399741e7f4d628f6b72be3dca9cbb1c4fbd3f7fa - thanks! > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Priority: Major > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first execution of the test > `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, > `StrategyA` is removed from global > strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), > and the test does not restore it after execution. > Therefore, in the second execution of the test, `StrategyA` is not present in > the global strategies, so assertions like > `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would > fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). > A fix is necessary since unit tests shall be self-contained. Idempotent tests > help maintain this isolation by ensuring that the state of the system under > test is consistent at the beginning of each test, regardless of previous test > runs. For example, fixing non-idempotent tests can help proactively avoid > state pollution that results in test order dependency (which could hurt > regression testing with the use of test selection / prioritization / > parallelism. > ### Failure Message in the 2nd Test Run: > ``` > java.lang.AssertionError: > at org.junit.Assert.fail(Assert.java:87) > at org.junit.Assert.assertTrue(Assert.java:42) > at org.junit.Assert.assertTrue(Assert.java:53) > at > org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) > ``` > ### Reproduce > Use the `NIOInspector` plugin that supports rerunning individual tests in the > same environment: > ``` > cd gremlin-core > mvn edu.illinois:NIOInspector:rerun > -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache > ``` > ### Proposed Fix > Handle initial strategies registration in a `setup()` method rather than in > static blocks of test classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847207#comment-17847207 ] ASF GitHub Bot commented on TINKERPOP-3079: --- codecov-commenter commented on PR #2608: URL: https://github.com/apache/tinkerpop/pull/2608#issuecomment-2117045164 ## [Codecov](https://app.codecov.io/gh/apache/tinkerpop/pull/2608?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 80.44%. Comparing base [(`2d32517`)](https://app.codecov.io/gh/apache/tinkerpop/commit/2d32517b3bca1b00d716b3205c2abdbcd6ed3352?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`ffd57f5`)](https://app.codecov.io/gh/apache/tinkerpop/pull/2608?dropdown=coverage=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 130 commits behind head on master. Additional details and impacted files ```diff @@ Coverage Diff @@ ## master#2608 +/- ## + Coverage 76.16% 80.44% +4.28% Files 1085 27-1058 Lines 65189 5002 -60187 Branches 72890-7289 - Hits 49651 4024 -45627 + Misses12830 776 -12054 + Partials 2708 202-2506 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/tinkerpop/pull/2608?dropdown=coverage=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Priority: Major > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first execution of the test > `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, > `StrategyA` is removed from global > strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), > and the test does not restore it after execution. > Therefore, in the second execution of the test, `StrategyA` is not present in > the global strategies, so assertions like > `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would > fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). > A fix is necessary since unit tests shall be self-contained. Idempotent tests > help maintain this isolation by ensuring that the state of the system under > test is consistent at the beginning of each test, regardless of previous test > runs. For example, fixing non-idempotent tests can help proactively avoid > state pollution that results in test order dependency (which could hurt > regression testing with the use of test
[jira] [Commented] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same
[ https://issues.apache.org/jira/browse/TINKERPOP-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847194#comment-17847194 ] ASF GitHub Bot commented on TINKERPOP-3079: --- kaiyaok2 opened a new pull request, #2608: URL: https://github.com/apache/tinkerpop/pull/2608 Fixes https://issues.apache.org/jira/browse/TINKERPOP-3079 ### Brief Description of the Bug The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is non-idempotent, as it passes in the first run but fails in the second run in the same environment. The source of the problem is that the initial strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the `TestGraph` and ` TestGraphComputer` classes are static (see https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 and https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). These static blocks will only be called once during class loading. In the first execution of the test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, `StrategyA` is removed from global strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), and the test does not restore it after execution. Therefore, in the second execution of the test, `StrategyA` is not present in the global strategies, so assertions like `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). A fix is necessary since unit tests shall be self-contained. Idempotent tests help maintain this isolation by ensuring that the state of the system under test is consistent at the beginning of each test, regardless of previous test runs. For example, fixing non-idempotent tests can help proactively avoid state pollution that results in test order dependency (which could hurt regression testing with the use of test selection / prioritization / parallelism. ### Failure Message in the 2nd Test Run: ``` java.lang.AssertionError: at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) ``` ### Reproduce Use the `NIOInspector` plugin that supports rerunning individual tests in the same environment: ``` cd gremlin-core mvn edu.illinois:NIOInspector:rerun -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache ``` ### Proposed Fix Handle initial strategies registration in a `setup()` method rather than in static blocks of test classes. ### Verify the Change After the patch, all tests pass idempotently. > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is not idempotent, as it passes in the first run and fails in repeated runs > in the same environment. > - > > Key: TINKERPOP-3079 > URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 > Project: TinkerPop > Issue Type: Bug > Environment: Ubuntu 22.04, Java 17 >Reporter: Kaiyao Ke >Priority: Major > Original Estimate: 1h > Remaining Estimate: 1h > > ### Brief Description of the Bug > The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` > is non-idempotent, as it passes in the first run but fails in the second run > in the same environment. The source of the problem is that the initial > strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the > `TestGraph` and ` TestGraphComputer` classes are static (see > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 > and > https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). > These static blocks will only be called once during class loading. > In the first
[jira] [Created] (TINKERPOP-3079) The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same en
Kaiyao Ke created TINKERPOP-3079: Summary: The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is not idempotent, as it passes in the first run and fails in repeated runs in the same environment. Key: TINKERPOP-3079 URL: https://issues.apache.org/jira/browse/TINKERPOP-3079 Project: TinkerPop Issue Type: Bug Environment: Ubuntu 22.04, Java 17 Reporter: Kaiyao Ke ### Brief Description of the Bug The test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache` is non-idempotent, as it passes in the first run but fails in the second run in the same environment. The source of the problem is that the initial strategies registration (`StrategyA`, `StrategyB` and `StrategyC`) for the `TestGraph` and ` TestGraphComputer` classes are static (see https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L132 and https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L185). These static blocks will only be called once during class loading. In the first execution of the test `TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache`, `StrategyA` is removed from global strategies(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L82), and the test does not restore it after execution. Therefore, in the second execution of the test, `StrategyA` is not present in the global strategies, so assertions like `assertTrue(strategies.getStrategy(StrategyA.class).isPresent())` would fail(https://github.com/apache/tinkerpop/blob/66e5a47ffd3976d29bc7797c399b8e11c5ba810e/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/TraversalStrategiesTest.java#L77). A fix is necessary since unit tests shall be self-contained. Idempotent tests help maintain this isolation by ensuring that the state of the system under test is consistent at the beginning of each test, regardless of previous test runs. For example, fixing non-idempotent tests can help proactively avoid state pollution that results in test order dependency (which could hurt regression testing with the use of test selection / prioritization / parallelism. ### Failure Message in the 2nd Test Run: ``` java.lang.AssertionError: at org.junit.Assert.fail(Assert.java:87) at org.junit.Assert.assertTrue(Assert.java:42) at org.junit.Assert.assertTrue(Assert.java:53) at org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest.shouldAllowUserManipulationOfGlobalCache(TraversalStrategiesTest.java:77) ``` ### Reproduce Use the `NIOInspector` plugin that supports rerunning individual tests in the same environment: ``` cd gremlin-core mvn edu.illinois:NIOInspector:rerun -Dtest=org.apache.tinkerpop.gremlin.process.TraversalStrategiesTest#shouldAllowUserManipulationOfGlobalCache ``` ### Proposed Fix Handle initial strategies registration in a `setup()` method rather than in static blocks of test classes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
