urlyy commented on issue #1547:
URL: https://github.com/apache/fury/issues/1547#issuecomment-2211618829
Hi, I've created a basic demo. Since Rust's string encoding is UTF-8, we can
directly use the String's API to convert UTF-16 encoded data into a string.
However, this method doesn't utilize SIMD. So, I'm wondering what else needs to
be done on top of this.
```rust
let bytes = [
0b01101000, // 'h'
0b00000000,
0b01100101, // 'e'
0b00000000,
0b01101100, // 'l'
0b00000000,
0b01101100, // 'l'
0b00000000,
0b01101111, // 'o'
0b00000000,
0b00010110, // '世' in UTF-16 little-endian
0b01001110,
0b01001100, // '界' in UTF-16 little-endian
0b01110101,
];
let utf16_vec: Vec<u16> = bytes
.chunks_exact(2)
.map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]]))
.collect();
let utf16_string = String::from_utf16(&utf16_vec).expect("Invalid UTF-16
sequence");
println!("{}", utf16_string);
```
I'm not familiar with high-performance computing and I've only found the
[std::simd](https://doc.rust-lang.org/beta/std/simd/index.html) library,
however, it is a **nightly-only experimental** API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
