cyb70289 commented on a change in pull request #12504:
URL: https://github.com/apache/arrow/pull/12504#discussion_r814418677
##########
File path: cpp/src/arrow/csv/writer.cc
##########
@@ -328,6 +359,31 @@ class QuotedColumnPopulator : public ColumnPopulator {
}
private:
+ // Returns true if there's no quote in the string array
+ // similar to std::find, but with much better performance
+ static bool NoQuoteInArray(const StringArray& array) {
+ const uint8_t* const data = array.raw_data() + array.value_offset(0);
+ const int64_t buffer_size = array.total_values_length();
+ int64_t offset = 0;
+#if defined(ARROW_HAVE_SSE4_2) || defined(ARROW_HAVE_NEON)
+ using simd_batch = xsimd::make_sized_batch_t<uint8_t, 16>;
+ while ((offset + 16) <= buffer_size) {
+ const auto v = simd_batch::load_unaligned(data + offset);
Review comment:
No obvious difference for aligned access per my test, probably because
this code is not the hotspot.
That said, unaligned loading does have observable penalty for trivial code,
e.g., https://quick-bench.com/q/MLvhY9_3IRZK0UR66DL1KFOF0xc
We don't have standard for using xsimd.
Yes we can leverage compiler auto-vectorization to avoid explicit simd code.
Like your example, compiler is able to vectorize `std::count` like code pretty
well. One reason I'm using xsimd here is there's already a similar optimization
in csv writer to find string cell with illegal chars:
https://github.com/apache/arrow/blob/master/cpp/src/arrow/csv/writer.cc#L213-L249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
