Hey Arnav, Thanks for this proposal, this looks great overall.
One question for my understanding: have you considered, instead of adding a new dedicated symbol page type, re-using the existing dict page mechanism? It seems a symbol table essentially is a specific kind of dictionary, so I'm wondering whether it could be stored as such. This would reduce the surface of the spec change quite a bit, only new encoding enums (FSST, FSST_16) plus the FSST data page layout. Writers would have to ensure that they don't store more than 255 (or 65,535, respectively) dictionary entries, each at most 8 (or 16) bytes. If it has been considered before and been decided against, it might be worth being added under a "Rejected alternatives" appendix? Thanks, --Gunnar On Thu, 13 Aug 2026 at 02:56, Arnav Balyan <[email protected]> wrote: > Hi Curt, > Thanks for taking a look! You’re reading it correctly. > The existing C++ and Rust work began while the specification was still in > progress. > I’m waiting for the final comments before finalizing the spec. Once the > spec has been finalized and voted upon, the next step will be to update and > complete the implementations so they conform to it. > > Thanks and Regards, > Arnav > > On Thu, Aug 13, 2026 at 5:19 AM Curt Hagenlocher <[email protected]> > wrote: > > > It seems to me that neither the C++ PoC nor the Rust PoC are > > consistent with what's in the current specification. Am I reading > > something incorrectly? > > > > On Tue, Aug 11, 2026 at 11:45 PM Arnav Balyan <[email protected]> > > wrote: > > > > > > Hello everyone, > > > > > > I wanted to provide a quick update on the FSST encoding proposal for > > > Parquet. > > > > > > Over the last several months, we have incorporated the feedback > received > > > from the community and finalized the design. Thank you to everyone who > > > reviewed the proposal and helped resolve the open questions. > > > > > > At this point, we are planning to move the proposal from the > Design/Draft > > > phase to the Implementation phase. Before doing so, I would like to ask > > for > > > one final round of feedback and ensure that everyone who wanted to > review > > > the specification has had an opportunity to do so. > > > > > > Implementation work is also underway: > > > - Devan Benz has started an Arrow Rust > > > <https://github.com/apache/arrow-rs/pull/10153> implementation. > > > - I have an existing Arrow C++ < > > https://github.com/apache/arrow/pull/48232> > > > PoC for FSST. > > > > > > For the initial interoperability work, it would be particularly helpful > > to > > > have implementations in Parquet Java and Arrow Go as well. > > > > > > If anyone is interested in owning or contributing to the Parquet Java > or > > > Arrow Go implementation, please let us know. Contributions to the > > existing > > > Rust and C++ implementations and to cross language interoperability > > testing > > > would also be greatly appreciated. > > > > > > Artifacts: > > > > > > - FSST specification: > > > > > > > > > https://docs.google.com/document/d/1Xg2b8HR19QnI3nhtQUDWZJhCLwJzW6y9tU1ziiLFZrM/edit > > > > > > - Tracking issue: > > > https://github.com/apache/parquet-format/issues/531 > > > > > > - Original FSST mailing-list discussion: > > > https://lists.apache.org/thread/15w66hvm516641btrljmtqhb4m38wn3s > > > > > > - FSST and OnPair discussion: > > > https://lists.apache.org/thread/hfoltdl6o6txc3zp4680nns1mh29h0r8 > > > > > > - Arrow C++ PoC: > > > https://github.com/apache/arrow/pull/48232 > > > > > > - Arrow Rust implementation: > > > https://github.com/apache/arrow-rs/pull/10153 > > > > > > Based on any new comments/questions, the next steps will be to: > > > 1. Address any feedback from this final review. > > > 2. Open a PR updating the proposal status from Draft/PoC to > > Implementation. > > > 3. Continue the java/go implementations and establish cross language > > tests. > > > 4. Prepare for the formal vote once the implementation requirements > have > > > been met. > > > > > > Please share any remaining design feedback or implementation interest > on > > > this thread or the spec doc. > > > > > > Thanks again to everyone involved for the thoughtful reviews, > benchmarks, > > > and implementation work. I really appreciate all the help in bringing > the > > > proposal to this stage. > > > > > > Warm regards, > > > Arnav > > >
