slow-J commented on code in PR #16631:
URL: https://github.com/apache/lucene/pull/16631#discussion_r3968494047
##########
lucene/core/src/java25/org/apache/lucene/internal/vectorization/PanamaVectorUtilSupport.java:
##########
@@ -584,6 +588,24 @@ private static int int4DotProductBody(ByteVectorLoader a,
ByteVectorLoader b, in
return sum;
}
+ @Override
+ public void int4Unpack(byte[] packed, byte[] unpacked) {
+ final int len = packed.length;
+ final int bound = BYTE_SPECIES_FULL.loopBound(len);
+ int i = 0;
+ for (; i < bound; i += BYTE_SPECIES_FULL.length()) {
+ ByteVector v = ByteVector.fromArray(BYTE_SPECIES_FULL, packed, i);
+ // LSHR is a logical shift within the byte lane, so the high nibble
needs no mask.
+ v.lanewise(LSHR, 4).intoArray(unpacked, i);
+ v.lanewise(VectorOperators.AND, (byte) 0x0F).intoArray(unpacked, len +
i);
+ }
+ // scalar tail
+ for (; i < len; i++) {
+ unpacked[i] = (byte) ((packed[i] >> 4) & 0x0F);
+ unpacked[len + i] = (byte) (packed[i] & 0x0F);
+ }
Review Comment:
Same as above (Graviton2, `BYTE_SPECIES_FULL.length()` = 16), ops/us
Tested 2 quick ideas for the tail:
1. **Cascade** - step the tail down through narrower species, then falling
back to scalar. See
https://gist.github.com/slow-J/aa5042b21261bda5fc2e79a6557a62e3 (hope this
works to share)
2. **Masked** - one masked full-width iteration instead of the scalar loop:
```java
...
VectorMask<Byte> m = BYTE_SPECIES_FULL.indexInRange(i, len);
ByteVector v = ByteVector.fromArray(BYTE_SPECIES_FULL, packed, i, m);
...
```
| size | tail | Scalar | Vector (PR) | Cascade | Masked |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 1024 | 0 | 2.154 ± 0.003 | 27.475 ± 1.079 | 27.246 ± 0.971 | 27.133 ±
0.890 |
| 512 | 0 | 4.161 ± 0.004 | 44.962 ± 1.833 | 43.585 ± 1.091 | 44.217 ± 1.785
|
| 702 | 15 | 3.093 ± 0.003 | 21.255 ± 0.229 | 24.608 ± 0.455 | 6.126 ± 0.032
|
| 300 | 6 | 7.184 ± 0.012 | 38.920 ± 0.552 | 37.221 ± 0.299 | 7.762 ± 0.065 |
Not sure why the second version performed so bad?
Do you think its worth the complexity of adding the cascading logic here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
