vinooganesh commented on PR #100:
URL: https://github.com/apache/parquet-testing/pull/100#issuecomment-4538468102
Hey all, quick heads up, I just opened a stacked PR
(prtkgaur/parquet-testing#1) with six Java-written ALP fixtures from
apache/parquet-java#3397 to broaden the cross-language test surface here.
Four of them are dataset-derived (~960 KB combined) and cover both page
versions (V1/V2) × both ALP vector sizes (1024/4096) for float and double,
using the existing arade and spotify1 source data. The other two are a small
corner-case fixture (~60 KB) targeting the design list in #105 — vectors with
no exceptions, one exception, all-NaN, NaN+Inf+(-0.0), constant (bit_width=0),
differing exponents per vector, and optional columns with nulls, both f32 and
f64 — plus a sidecar _expect.csv ground truth emitted directly from the
construction recipe, so the corner-case file is independently verifiable
without rerunning the Java generator.
I've already run Arrow C++ ALP from apache/arrow#48345 against all six and
bit-compared every value against the _expect.csv truth — 342K values, 0
mismatches. The full local matrix (17 fixtures across all V1/V2 × 1024/4096
combinations) comes out to 1.59M values verified, also 0 mismatches.
On size: @alamb raised concerns in #105 about test data bloating CI
checkouts, so I deliberately kept this PR small — ~1.1 MB total. I have 11
more fixtures locally from the full matrix (~2.5 MB more) if anyone wants
additional coverage, but figured the four representative variants plus the
corner cases were enough to verify the axes work without growing the repo
unnecessarily.
Happy to add or trim, let me know what would be most useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
