nielspardon opened a new pull request, #12700: URL: https://github.com/apache/gluten/pull/12700
## What changes are proposed in this pull request? One increment of the Substrait proto rebase (v0.23.0 → 0.98.0), tracked by #12597. Substrait 0.98 splits Gluten's overloaded `CrossRel` — which carried both `expression` and `type` — into a pure-cartesian `CrossRel` plus a new `NestedLoopJoinRel` (`expression=4`, `type=5`) with a paired `JoinType` enum. This PR adopts that split across the vendored proto, the JVM producer, and both native consumers, in lockstep: - **Proto** (`algebra.proto`): reduce `CrossRel` to the pure-cartesian shape; add `NestedLoopJoinRel` verbatim from 0.98 (including the new `LEFT_MARK`/`RIGHT_MARK` join types). The new rel is parked at a temporary Rel-oneof field number; the final oneof-reconciliation increment relocates it to the official `nested_loop_join = 18`. Field numbers are codegen-transparent for Gluten's coupled build (single proto source, transient plans, producer + both consumers regenerate and ship together → no wire-compat constraint). - **Producer (JVM)**: `CrossRelNode` → `NestedLoopJoinRelNode`, `RelBuilder.makeNestedLoopJoinRel`, `SubstraitUtil.toNestedLoopJoinSubstrait`, and all callers (`CartesianProductExecTransformer`, `BroadcastNestedLoopJoinExecTransformer`, `JoinUtils`, `StorageJoinBuilder`). - **Velox consumer**: `SubstraitToVeloxPlan` and `SubstraitToVeloxPlanValidator` dispatch and parse `NestedLoopJoinRel` instead of `CrossRel`. - **ClickHouse consumer**: `CrossRelParser`, `CHUtil`, and `BroadcastJoinBuilder` re-point to `nested_loop_join` / `NestedLoopJoinRel_JoinType` (class/file names kept to minimize churn). The `NestedLoopJoinRel.JoinType` enum reorders the anti/semi/single values relative to the old `CrossRel.JoinType` fork, but Gluten's producers only ever emit `INNER`/`OUTER`/`LEFT`/`LEFT_SEMI` — whose proto numbers (1/2/3/5) are identical in both — and both consumers only handle those same values, so the split is semantically neutral for Gluten's code paths. Part of #12597. ## How was this patch tested? - New `SubstraitUtilJoinTypeSuite` locks the semantic-neutrality invariant: the Spark → `NestedLoopJoinRel.JoinType` mapping, the stable proto numbers (INNER=1/OUTER=2/LEFT=3/LEFT_SEMI=5), the unsupported-type → `UNRECOGNIZED` fallback, and `ordinal() == getNumber()` for every declared enum value (the raw-ordinal-over-JNI path that `StorageJoinBuilder` relies on). - Built locally: the JVM producer (`gluten-substrait`, spark-3.5) and the Velox native backend both compile cleanly; the new suite passes. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) 🤖 Generated with AI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
