rluvaton commented on issue #9083: URL: https://github.com/apache/arrow-rs/issues/9083#issuecomment-3764170446
> > you are right but in my tests I saw that it is faster to (1) encode to this format (2) partition and (3) decode back rather than to encode it using specialized implementation per type > > What tests are you referring to? Maybe I would understand better if you could describe what your test was doing > Some spark workload of a client > > @alamb in the case when we already need to convert to rows (in the fallback implementation) the ordering doesn't matter and we can use this new row format which will make it faster. > > Right -- I am saying this proposal would make the fallback faster. I was suggesting avoiding the fallback entirely (with specialized implementation) Sometimes you either can't or don't wanna bother in implementing a specialized implementation, for example struct of lists of structs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
