Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16403#discussion_r94012605
--- Diff:
common/unsafe/src/test/java/org/apache/spark/unsafe/PlatformUtilSuite.java ---
@@ -74,4 +76,71 @@ public void memoryDebugFillEnabledInTest() {
Platform.getByte(offheap.getBaseObject(), offheap.getBaseOffset()),
MemoryAllocator.MEMORY_DEBUG_FILL_CLEAN_VALUE);
}
+
+ /**
+ * This test ensures that double access is handled correctly for the
host platform.
+ * On platforms that require double access to be aligned to 8-byte
boundaries, this
+ * test should pass and not cause the JVM to segfault.
+ */
+ @Test
+ public void unalignedDoublePutAndGet() {
+ MemoryBlock testBuffer = MemoryBlock.fromLongArray(new long[20]);
+
+ // write a double to an unaligned location
+ long unalignedOffset = testBuffer.getBaseOffset() + 3;
+ // double check unalignment just to be sure:
+ if (unalignedOffset%8 == 0) {
+ ++unalignedOffset;
+ }
+
+ Platform.putDouble(testBuffer.getBaseObject(), unalignedOffset,
3.14159);
+
+ double foundDouble = Platform.getDouble(testBuffer.getBaseObject(),
unalignedOffset);
+
+ Assert.assertEquals(3.14159, foundDouble, 0.000001);
+ }
+
+ /**
+ * The goal of this test is to ensure that doubles written to a memory
location
+ * using the long-buffered functor can be read back using the direct
functor,
+ * and vice-versa. This is to ensure binary output such as parquet files
written
+ * from one platform will be readable on another.
+ */
+ @Test
+ public void mixedDoubleFunctors() {
+ sun.misc.Unsafe unsafe;
+ try {
+ Field unsafeField = Unsafe.class.getDeclaredField("theUnsafe");
+ unsafeField.setAccessible(true);
+ unsafe = (sun.misc.Unsafe) unsafeField.get(null);
+
+ MemoryBlock testBuffer = MemoryBlock.fromLongArray(new long[20]);
+
+ // ensure offset is aligned so test can run on any host platform.
+ long alignedOffset = testBuffer.getBaseOffset();
+ if (alignedOffset%8 != 0) {
+ alignedOffset += 8 - alignedOffset%8;
+ }
+
+ DoubleAccessFunctor buffered = new DoubleAccessBuffered();
+ DoubleAccessFunctor direct = new DoubleAccessDirect();
+
+ final double direct2bufferedValue = 3.14159;
+ final double buffered2directValue = 2.71828;
+
+ buffered.putDouble(unsafe, testBuffer.getBaseObject(),
alignedOffset, buffered2directValue);
+ Assert.assertEquals(
+ buffered2directValue,
+ direct.getDouble(unsafe, testBuffer.getBaseObject(),
alignedOffset),
+ 0.000001);
+
+ direct.putDouble(unsafe, testBuffer.getBaseObject(), alignedOffset,
direct2bufferedValue);
+ Assert.assertEquals(
+ direct2bufferedValue,
+ buffered.getDouble(unsafe, testBuffer.getBaseObject(),
alignedOffset),
+ 0.000001);
+ } catch (Throwable cause) {
+ // can't run test because Unsafe is unavailable.
--- End diff --
You can't do this in a test, it will catch any error. Construct the test to
deal with this case explicitly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
