nielspardon opened a new pull request, #13210: URL: https://github.com/apache/gluten/pull/13210
## What changes are proposed in this pull request? Part of #12597. This brings the vendored `Rel.rel_type` oneof onto upstream Substrait 0.98's layout. Every upstream relation now sits at its upstream number: `exchange` 15, `expand` 16, `window` 17, `ddl` 20, `reference` 21, `update` 22 and `lateral_join` 24. The upstream relation messages Gluten was missing are vendored verbatim: `LateralJoinRel`, `UpdateRel` and the top-level `NamedTable` it uses. `DdlRel` gains `common = 8` and `advanced_extension = 9`. `ReferenceRel` moves to the top level; Gluten's copy was nested inside `AggregateFunction`, which no code referenced. None of these relations is produced or consumed by Gluten, so they cost nothing at runtime. Gluten's own relations, `GenerateRel` and `WindowGroupLimitRel`, move from 17 and 20 to 1000 and 1001, following the 1000+ numbering convention in `SubstraitModifications.md`. Upstream's next relation will take 25, so parking the forks at 25/26 would have recreated the kind of collision that convention exists to prevent. Producer and consumers refer to the oneof cases by name, so neither the JVM nor Velox needs code changes. ClickHouse needs one small change: it names rel metrics with `magic_enum::enum_name(rel.rel_type_case())`, and magic_enum only reflects values up to 127 by default, so `kGenerate` and `kWindowGroupLimit` would have been reported with an empty name. `SerializedPlanParser.cpp` widens magic_enum's range for `substrait::Rel::RelTypeCase` to 0–1001 to keep those names. Compatibility: the oneof renumbering is wire-incompatible, but there is no wire-compatibility constraint: Gluten plans are transient, and the JAR and native library are generated from one proto source and ship together. Both must be rebuilt from the same revision. `RelRoot.output_schema` is left as is here. Replacing it, so that the plan carries the expected output types itself, is a behavioural ClickHouse change and will follow separately. ## How was this patch tested? A new `RelProtoSuite` pins the oneof by descriptor. It checks every relation's tag and that `rel_type` holds exactly the standard plus Gluten-local relations, so a field grafted in later also fails it. It also checks that `ReferenceRel` is top level and that `DdlRel` has `common` and `advanced_extension`. A renumber round-trips cleanly through the generated classes, so these assert on the descriptors rather than on behaviour. Locally, `protoc` compiles all protos, including C++ and Java generation, and the `gluten-substrait` JVM build and the new suite pass. The ClickHouse native build needs the Linux toolchain, so the `enum_range` specialization is verified by inspection here and by CI. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) 🤖 Generated with AI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
