[
https://issues.apache.org/jira/browse/ARROW-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991777#comment-16991777
]
Neville Dipale commented on ARROW-5303:
---------------------------------------
Hi [~andy-thomason], would null checks in arrays affect the performance? I
tried [godbolt output|[https://godbolt.org/z/JKDXyb]], I don't know asm, so I
can't tell from the instructions if there's some optimisation taking place.
Perhaps instead of always checking if data is null, then appending a null byte,
we could run the cast on all values (as null values would still return
something), then just apply the null mask? This might only work on casts that
don't flow, like going from \{u|i}X to \{u|i}Y where X < Y.
I haven't seen other users of Arrow's Rust impl outside of DataFusion, so I
don't know if the cast kernel's performance is fine on large enough datasets,
or whether this ticket is worth pursuing in the short-term.
> [Rust] Add SIMD vectorization of numeric casts
> ----------------------------------------------
>
> Key: ARROW-5303
> URL: https://issues.apache.org/jira/browse/ARROW-5303
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust
> Affects Versions: 0.13.0
> Reporter: Neville Dipale
> Priority: Minor
>
> To improve the performance of cast kernels, we need SIMD support in numeric
> casts.
> An initial exploration shows that we can't trivially add SIMD casts between
> our Arrow T::Simd types, because `packed_simd` only supports a cast between
> T::Simd types that have the same number of lanes.
> This means that adding casts from f64 to i64 (same lane length) satisfies the
> bound trait `where TO::Simd : packed_simd::FromCast<FROM::Simd>`, but f64 to
> i32 (different lane length) doesn't.
> We would benefit from investigating work-arounds to this limitation. Please
> see
> [github::nevi_me::arrow/\{branch:simd-cast}/../kernels/cast.rs|[https://github.com/nevi-me/arrow/blob/simd-cast/rust/arrow/src/compute/kernels/cast.rs#L601]]
> for an example implementation that's limited by the differences in lane
> length.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
