Weijun-H commented on issue #9329:
URL: https://github.com/apache/arrow-rs/issues/9329#issuecomment-3940109239
I agree that realistically we want both options, and that different users
will prioritize correctness vs throughput.
To make this explicit and avoid implicit behavior changes, I think this
should be modeled as a policy choice rather than a boolean flag. Concretely,
something like:
```
enum SkipValidation {
Full, // correctness-first
Structural, // safe + fast (default)
BestEffort, // speed-first
}
```
This gives us a safe default (Structural), allows opt-in to maximum
correctness or maximum speed, and keeps the trade-offs explicit and extensible.
We can also constrain combinations, e.g. disallow `BestEffort when
strict_mode=true`.
From an implementation perspective, Structural can still be very fast (no
allocations, no AST, no number/string decoding), and it is straightforward to
fuzz and benchmark independently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
