Dear Yunhong and the Fluss community, After a thorough review of the FIP-36 design document and a comparative analysis with the Apache Iceberg Variant Type specification, I am casting my vote:
+1 (binding) — I strongly support this proposal. 1. Rationale: Alignment with Industry Standards FIP-36's three-tier architecture — Parquet Variant Binary Encoding, Arrow Columnar Storage, and Shredding Optimization — represents a well-considered design that aligns directly with the broader open-source ecosystem. Having studied the Iceberg Variant implementation in parallel, I find the convergence between the two projects particularly compelling: Binary Encoding Layer: Both FIP-36 and Iceberg adopt the Parquet Variant Binary Encoding specification, utilizing the identical metadata + value two-field structure [1][2]. This shared foundation ensures that Variant data written by Fluss can be read by any Parquet-compatible engine (Spark, Trino, Snowflake) without format translation overhead. Shredding Mechanism: The core optimization — extracting frequently accessed fields into typed sub-columns (typed_value) while preserving unextracted data in a residual binary field — is conceptually identical across both designs. Iceberg's production benchmarks show 30x faster reads compared to raw JSON strings and 8x faster compared to non-shredded binary [3], which validates the performance thesis behind FIP-36. Ecosystem Cohesion: With Spark 4.0, Flink 2.1 (FLIP-521), and Iceberg Table Spec V3 all converging on Variant as a first-class type, FIP-36 positions Fluss at the center of this emerging standard rather than on the periphery [4][5]. 2. Strengths of the FIP-36 Design I would like to highlight several design decisions that I find particularly well-executed: Arrow-native Memory Representation: The choice to represent Variant values as StructVector<metadata: VarBinaryVector, value: VarBinaryVector> in Arrow is a pragmatic decision that leverages Fluss's existing zero-copy read path and avoids unnecessary serialization overhead between the storage layer and the query engine. Server-side Selective Column Reading: The Shredding design enables column-level pruning at the server side, which is a meaningful optimization beyond what simple file-level statistics can achieve. This is especially valuable for Fluss's streaming-oriented workloads where latency sensitivity is high. Backward Compatibility: The proposal's approach to introducing Variant as an additive type without breaking existing string-based JSON storage is a prudent migration path. 3. Constructive Questions and Suggestions While I am fully supportive, I would like to raise a few points for the community's consideration as the implementation progresses: Q1: Interoperability with Iceberg Tables. Given that both Fluss and Iceberg now target the same Parquet Variant binary format, has the design considered the scenario where Fluss serves as a streaming ingestion layer into Iceberg-managed lakehouse tables? Specifically, would a Fluss-written Variant column be directly consumable by an Iceberg V3 reader without metadata reconciliation? If so, this could be a powerful integration story for the Fluss + Iceberg ecosystem. Q2: Shredding Metadata Management. In Iceberg's implementation, Shredding decisions are persisted in table-level metadata (the shredded_fields property in the schema). How does FIP-36 plan to manage Shredding metadata — will it be table-level, partition-level, or dynamically inferred at query time? The choice has implications for schema evolution and cross-version compatibility. Q3: Write-Amplification Trade-off. Iceberg's Shredding implementation reports a 20%-50% write throughput degradation due to schema inference and field extraction overhead [3]. For a streaming-first system like Fluss, this is a non-trivial concern. Are there plans to make Shredding optional or configurable per-column, allowing users to opt into the optimization only for latency-tolerant workloads? Q4: Non-JVM Language Support. The Iceberg community has recently invested in iceberg-cpp with Arrow-based Variant support, enabling Python (PyIceberg) and Go (iceberg-go) clients [6]. Given Fluss's growing multi-language client ecosystem, is there a roadmap for exposing Variant types through the C++/Python client libraries? These are not blockers but rather areas where I believe cross-project collaboration could yield significant dividends. 4. Conclusion FIP-36 is a timely and technically sound proposal that brings Fluss into alignment with the emerging industry standard for semi-structured data processing. The design is well-scoped, the performance thesis is validated by independent benchmarks from the Iceberg ecosystem, and the integration path with Spark/Flink/Parquet is clear. I look forward to seeing this feature land in Fluss. Best regards, ForwardXu [1] apache.org - Parquet Variant Binary Encoding Specification (2025-08-15) <https://github.com/apache/parquet-format/blob/master/VariantEncoding.md> [2] cwiki.apache.org - FIP-36: Support Variant Type and Shredding <https://cwiki.apache.org/confluence/spaces/FLUSS/pages/406623649/FIP-36+Support+Variant+Type+and+Shredding> [3] databricks.com - Iceberg Shredding Semi-Structured Data Optimization (2025-06-20) <https://www.databricks.com/> [4] apache.org - Parquet Variant Shredding Specification (2026-02-10) <https://github.com/apache/parquet-format/blob/master/VariantShredding.md> [5] apache.org - Iceberg Improvement Proposal (IIP): Variant Type Support (2024-10-09) <https://github.com/apache/iceberg/issues/10831> [6] medium.com - Iceberg V3 Ratification and Variant Type GA Progress (2026-06-15) <https://medium.com/> yunhong Zheng <[email protected]> 于2026年7月6日周一 11:57写道： > Hi, all. > > I'd like to start a vote on FIP-36: Support Variant Type and Shredding > [1]. You can find the discussion on it in here [2]. The vote will last > for at least 72 hours unless there is objection or insufficient votes. > > [1] > https://cwiki.apache.org/confluence/spaces/FLUSS/pages/406623649/FIP-36+Support+Variant+Type+and+Shredding > [2] https://lists.apache.org/thread/m4wrtxoxrkg0zbgoy7p1yr3o15onylcm > > > Best regards, > Yunhong >
