On Monday, 25 August 2014 at 21:53:50 UTC, Ola Fosheim Grøstad
wrote:
I presume you can load 16 bytes and do BITWISE-AND on the MSB,
then match against string-end and carefully use this to boost
performance of simultanous UTF validation, escape-scanning, and
string-end scan. A bit tricky, of course.
I think it is doable and worth it…
https://software.intel.com/sites/landingpage/IntrinsicsGuide/
e.g.:
__mmask16 _mm_cmpeq_epu8_mask (__m128i a, __m128i b)
__mmask32 _mm256_cmpeq_epu8_mask (__m256i a, __m256i b)
__mmask64 _mm512_cmpeq_epu8_mask (__m512i a, __m512i b)
__mmask16 _mm_test_epi8_mask (__m128i a, __m128i b)
etc.
So you can:
1. preload registers with "\\\\\\\\…" , "\"\"…" and "\0\0\0…"
2. then compare signed/unsigned/equal whatever.
3. then load 16,32 or 64 bytes of data and stream until the masks
trigger
4. tests masks
5. resolve any potential issues, goto 3
