Rich-T-kid commented on PR #11013: URL: https://github.com/apache/arrow-rs/pull/11013#issuecomment-5591676214
> If we can find an example query or plausible real world benchmark where it makes a difference then I think it is more compelling nothing comes to mind. I think any performance wins we get here would be insignificant compared to the IO cost. a reason for this is that the "check" is actually pretty cheap https://github.com/apache/arrow-rs/blob/4a0621d2a770ba893479b63354d2b503081bec73/parquet/src/arrow/buffer/offset_buffer.rs#L68 I initially thought it'd be a full string validation on each push but its just checks the first byte. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
