Hi Steven, Thank you for putting this together. I have a naive question, is Parquet the correct layer to build Reader performance guardrails that Writers choose to ignore or should that be at an application level such as Iceberg?
On Fri, Jun 26, 2026 at 12:05 PM Steve Loughran <[email protected]> wrote: > I've got a PR up to harden variant parsing in parquet-jave, trying to some > shallow validation of inputs > > https://github.com/apache/parquet-java/pull/3562 > ...with output saved and submitted as bad data to the test archive > https://github.com/apache/parquet-testing/pull/113 > > Neelesh Salian has a matching PR for iceberg java > https://github.com/apache/iceberg/pull/16568 > > Where there's an interesting question: what is a sensible depth limit for > variants? Too deep a variant creates stack problems, too shallow and you > can't represent data. > > my parquet-variant PR just took the 500 element depth from the json parsing > in org.apache.parquet.variant.VariantJsonParser, but really, it would be > good to have a consensus on what is a sensible depth for nested variants > which every library can expect other readers to cope with. > > Looking at jackson, there's a issue to link to all their issues, including > what's considered a CVE because it was a bit too brittle > https://github.com/FasterXML/jackson-core/issues/637 > > And it seems like the current depth is 1000, though we should check with PJ > Fanning there https://github.com/FasterXML/jackson-core/pull/943 > > > 1. What depth do people think is good? > 2. What limits do the other variant parsers apply? > > I don't care what the actual number is, what is important is that consensus > and everyone confident it's a good number. > > -Steve >
