tustvold commented on issue #9211: URL: https://github.com/apache/arrow-rs/issues/9211#issuecomment-3765513123
Broadly speaking I agree with Andrew's points. One thing to perhaps add is that Avro is inherently a tricky format to decide efficiently, with the encodings being inherently problematic for branch predictors. Similarly the conversion from a row-representation to arrow is a non-trivial component of decode (at least for JSON) and isn't something a JIT will be able to accelerate much as it is already vectorized. That's all to say, I'm not sure how big the return will actually be. Perhaps it might be worth comparing the current decoder vs a hand-written decoder for a given schema of interest, effectively to determine the upper-bound of what is actually possible? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
