LuciferYang opened a new issue, #12655:
URL: https://github.com/apache/gluten/issues/12655
### Backend
VL (Velox) — the code is backend-agnostic and lives in `gluten-core`.
### Bug description
`ComponentSuite` registers dummy components into the JVM-global component
graph, including a deliberate dependency cycle in its "Dependency cycle" test,
and never removes them. Any later suite in the same JVM that calls
`Component.sorted()` therefore fails with `UnsupportedOperationException: Cycle
detected in the component graph: B, D, C`, pointing at components that have
nothing to do with the failing test.
Nothing in `gluten-core` calls `Component.sorted()` after `ComponentSuite`
today, which is why this has not been noticed. But ten call sites in main
sources reach it — `GlutenPlugin` (5), `GlutenSessionExtensions`,
`ConventionFunc`, `HeuristicTransform`, `GlutenCostModel`, and
`BackendsApiManager` in gluten-substrait — so any suite that boots a
`SparkContext` with the Gluten plugin, or builds session extensions, will hit
it as soon as it is ordered after `ComponentSuite`. I ran into this while
adding such a suite: it passes in the natural order and fails when
`ComponentSuite` runs first.
`Component`'s graph and the `allComponentsLoaded` discovery latch are both
`object`-level state with no way to reset them, so a suite that registers
components has no way to clean up after itself.
### Gluten version
main (1.8.0-SNAPSHOT)
### Spark version
Version-agnostic (applies to spark-3.3 / 3.4 / 3.5 / 4.0 / 4.1).
### Spark configurations
N/A — a test-isolation issue, no Spark configuration involved.
### System information
N/A — logic in `gluten-core/src/main/scala/org/apache/gluten/component/`,
independent of OS/hardware.
### Relevant logs
```text
java.lang.UnsupportedOperationException: Cycle detected in the component
graph: B, D, C
at org.apache.gluten.component.Component$Graph.sorted(Component.scala:...)
at org.apache.gluten.component.Component$.sortedUnsafe(Component.scala:136)
at
org.apache.gluten.component.package$.ensureAllComponentsRegistered(package.scala:39)
at org.apache.gluten.component.Component$.sorted(Component.scala:131)
at org.apache.gluten.GlutenDriverPlugin.init(GlutenPlugin.scala:63)
```
### Fix direction
Add a testing-only reset that empties the graph, clears each component's
registration flag so it can register again, and re-arms the discovery latch;
call it from `ComponentSuite#afterAll`. Guard it with a suite that registers a
cycle, resets, and asserts the graph is empty, plus a test that runs
`ComponentSuite` in-process and asserts it cleaned up, so the fix's own call
site is covered without depending on suite execution order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
