mbutrovich commented on code in PR #12289:
URL: https://github.com/apache/arrow-rs/pull/12289#discussion_r4602903053


##########
arrow-row/src/lib.rs:
##########
@@ -1101,6 +1102,35 @@ impl RowConverter {
 
         let write_offset = rows.num_rows();
         let lengths = row_lengths(columns, &encoders);
+        if let LengthTracker::Fixed { length, num_rows } = lengths

Review Comment:
   > That key never takes the strided path. Could the cost come from the 
strided code being inlined into `append`?
   
   I don't think so. `append` calls `encode_column` out of line, once per 
column 
([lib.rs](https://github.com/apache/arrow-rs/blob/24c1ede738e7f52f3f90acaa2924d659a5373557/arrow-row/src/lib.rs#L1137-L1146)).
 That holds in release builds for `aarch64-unknown-linux-gnu` and 
`x86_64-unknown-linux-gnu`, and in a thin-LTO `row_format` binary. So a larger 
`append` adds a cost per column call, 53 per batch. A 3% loss on `4096 53 
columns` on Neoverse-V2 is about 25 us per batch, which is too much to come 
from 53 calls.
   
   The change on that key's per-row path is `copy_short`. Its 13 string columns 
have random lengths of 0 to 10, 10 to 20, 20 to 30, and 0 to 50 bytes 
([row_format.rs](https://github.com/apache/arrow-rs/blob/24c1ede738e7f52f3f90acaa2924d659a5373557/arrow/benches/row_format.rs#L138-L172)).
 Its speedup follows the random-length string case on each server:
   
   | Case | Neoverse-V2 (default / aligned) | Xeon (default / aligned) |
   |---|---:|---:|
   | `8192 string(1..40, 0)` | 1.04x / 0.99x | 1.21x / 1.17x |
   | `4096 53 columns` | 0.97x / 0.98x | 1.06x / 1.09x |
   | `8192 53 columns` | 0.98x / 0.99x | 1.06x / 1.09x |
   
   So `copy_short` costs up to 3% on Neoverse-V2 for random short strings. I 
replied about whether to keep it on the other thread.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to