srowen commented on a change in pull request #26548: [SPARK-29918][SQL]
RecordBinaryComparator should check endianness when compared by long
URL: https://github.com/apache/spark/pull/26548#discussion_r347413364
##########
File path:
sql/core/src/test/java/test/org/apache/spark/sql/execution/sort/RecordBinaryComparatorSuite.java
##########
@@ -321,4 +321,69 @@ public void
testBinaryComparatorWhenOnlyTheLastColumnDiffers() throws Exception
assert(compare(0, 1) < 0);
}
+
+ @Test
+ public void
testBinaryComparatorGiveSameResultWhenComparedByteByByteAndComparedByLong()
+ throws Exception {
+ int numFields = 1;
+
+ UnsafeRow row1 = new UnsafeRow(numFields);
+ byte[] data1 = new byte[100];
+ row1.pointTo(data1, computeSizeInBytes(numFields * 8));
+ row1.setLong(0, 0x0800000000000000L);
+
+ UnsafeRow row2 = new UnsafeRow(numFields);
+ byte[] data2 = new byte[100];
+ row2.pointTo(data2, computeSizeInBytes(numFields * 8));
+ row2.setLong(0, 0x0000008000000000L);
+
+ UnsafeRow row3 = new UnsafeRow(numFields);
+ byte[] data3 = new byte[100];
+ row3.pointTo(data3, computeSizeInBytes(numFields * 8));
+ row3.setLong(0, 0x0000008000000000L);
+
+ insertRow(row1);
+ insertRow(row2);
+ insertRow(row3);
+
+ // the bytes in row2 and row3 are the same.
+ // the base offset of row1 is 20, row2 is 40, row3 is 60
+ // so the RecordBinaryComparator will compare row1 and row2 with two long
comparison directly,
+ // while the comparison between row1 and row3 is started with 4 bytes
byte-by-byte comparison,
+ // followed by a long comparison, and lastly 4 bytes byte-by-byte
comparison.
+ assert(compare(0, 1) < 0);
Review comment:
Just assert the comparison is the same, here and below.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
