yadavay-amzn opened a new pull request, #3607: URL: https://github.com/apache/parquet-java/pull/3607
Closes #3601. ## Summary Caches the result of `CorruptStatistics.shouldIgnoreStatistics` to avoid redundant version string parsing. The `createdBy` string is constant per file but was parsed R×C times (row groups × columns) during footer reading. ## Changes - Added a bounded `ConcurrentHashMap<String, Boolean>` cache (max 64 entries) keyed on `createdBy` - Cheap type check (BINARY/FIXED_LEN_BYTE_ARRAY) still runs before cache lookup - When cache is full, computes directly without storing (best-effort cap) ## Testing - `testCachingBehavior`: verifies cache is populated (cacheSize grows from 0→1→2) - `testCorrectnessWhenCacheIsFull`: fills cache to 64 entries, verifies correct results on the cache-bypass path (65th+ distinct strings) - All existing `CorruptStatisticsTest` tests pass (correctness preserved) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
