mapleFU commented on PR #41346: URL: https://github.com/apache/arrow/pull/41346#issuecomment-2099900291
Sigh, adding checks is easy, but maybe reasoning them would be a little tricky, and https://github.com/apache/arrow/commit/77fc23fcae0331da3adf94619a381a371a6e414f adds some ad-hoc checkings. Not ad-hoc checkings are written everywhere in our system. Maybe I can summarize them as: 1. Levels Count / num_values not matches the `num_values()` in Page Header. `LevelDecoder` currently doesn't checks this ðŸ˜. But we will check the decoded `def-level` should equal to `rep-level`, and we can check `available_values_current_page()` 2. storing value not matches the level, like in `ReadBatch`, we parsed that we have 10 non-null values, but only reads 5. This should be regard as "value not matches level" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
