mkaravel commented on a change in pull request #34056:
URL: https://github.com/apache/spark/pull/34056#discussion_r716989650
##########
File path:
common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java
##########
@@ -101,4 +101,95 @@ public static long getPrefix(byte[] bytes) {
}
return result;
}
+
+ // Return the bitwise AND of two byte arrays. The byte length of the result
is equal to the
+ // maximum byte length of the two inputs. The two input byte arrays are
aligned with respect
+ // to their least significant (right-most) bytes.
+ public static byte[] bitwiseAnd(byte[] bytes1, byte[] bytes2) {
+ if (bytes1 == null || bytes2 == null) return null;
+ // Compute the length of the result (maximum of the lengths of the inputs).
+ final int len1 = bytes1.length;
+ final int len2 = bytes2.length;
+ final int maxLen = Math.max(len1, len2);
+ if (maxLen == 0) {
+ return EMPTY_BYTE;
+ }
+ final byte[] result = new byte[maxLen];
+ final int minLen = Math.min(len1, len2);
+ // Initialize the first `maxLen - minLen` bytes to 0.
+ Platform.setMemory(result, Platform.BYTE_ARRAY_OFFSET, maxLen - minLen,
(byte)0);
+ // Compute the right-most minLen bytes of the result.
+ for (int j = 0; j < minLen; ++j) {
+ result[maxLen - 1 - j] = (byte)(bytes1[len1 - 1 - j] & bytes2[len2 - 1 -
j]);
+ }
+ return result;
+ }
+
+ // Return the bitwise OR of two byte arrays. The byte length of the result
is equal to the
+ // maximum byte length of the two inputs. The two input byte arrays are
aligned with respect
+ // to their least significant (right-most) bytes.
+ public static byte[] bitwiseOr(byte[] bytes1, byte[] bytes2) {
+ if (bytes1 == null || bytes2 == null) return null;
+ // Compute the length of the result (maximum of the lengths of the inputs).
+ final int len1 = bytes1.length;
+ final int len2 = bytes2.length;
+ final int maxLen = Math.max(len1, len2);
+ if (maxLen == 0) {
+ return EMPTY_BYTE;
+ }
+ final byte[] result = new byte[maxLen];
+ final int minLen = Math.min(len1, len2);
+ // Copy the first `maxLen - minLen` bytes of the longer byte array into
the result buffer.
+ final byte[] maxLenBytes = (len1 == maxLen) ? bytes1 : bytes2;
+ Platform.copyMemory(
+ maxLenBytes, Platform.BYTE_ARRAY_OFFSET,
+ result, Platform.BYTE_ARRAY_OFFSET,
+ maxLen - minLen);
Review comment:
Yes, it is a no-op. See the code here:
https://github.com/apache/spark/blob/d03999ab8846d4897d2ce95ca21a7feed45f292b/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L249
I therefore think there is no need to add an explicit `if-else`. I have also
added a few more test cases with equal-length inputs that cover this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
