Github user sachouche commented on a diff in the pull request:
https://github.com/apache/drill/pull/1060#discussion_r164525752
--- Diff:
exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java ---
@@ -0,0 +1,186 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.util;
+
+import java.lang.reflect.Field;
+import java.nio.ByteOrder;
+
+import sun.misc.Unsafe;
+
+/** Exposes advanced Memory Access APIs for Little-Endian / Unaligned
platforms */
+@SuppressWarnings("restriction")
+public final class MemoryUtils {
+
+ // Ensure this is a little-endian hardware */
+ static {
+ if (ByteOrder.nativeOrder() != ByteOrder.LITTLE_ENDIAN) {
+ throw new IllegalStateException("Drill only runs on LittleEndian
systems.");
+ }
+ }
+
+ /** Java's unsafe object */
+ private static Unsafe UNSAFE;
+
+ static {
+ try {
+ Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
+ theUnsafe.setAccessible(true);
+ UNSAFE = (Unsafe) theUnsafe.get(null);
+ }
+ catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ /** Byte arrays offset */
+ private static final long BYTE_ARRAY_OFFSET =
UNSAFE.arrayBaseOffset(byte[].class);
+
+ /** Number of bytes in a long */
+ public static final int LONG_NUM_BYTES = 8;
+ /** Number of bytes in an int */
+ public static final int INT_NUM_BYTES = 4;
+ /** Number of bytes in a short */
+ public static final int SHORT_NUM_BYTES = 2;
+
+//----------------------------------------------------------------------------
+// APIs
+//----------------------------------------------------------------------------
+
+ /**
+ * @param data source byte array
+ * @param index index within the byte array
+ * @return short value starting at data+index
+ */
+ public static short getShort(byte[] data, int index) {
--- End diff --
Vlad,
Can you please clarify your point? Are you suggesting that every memory
access has to be via DrillBuf even if the data is loaded in Java heap and the
developer is performing an optimization. If this is your goal, then I would
think the danger is that the DrillBuf class will be cluttered and contracts
hard to understand:
- Notice that DrillBuf performs expensive sanity checks (boundary and
reference count checks); this is due to Drill memory optimizations for
minimizing the overall memory usage.
- When the code is processing already load data, these checks are not
applicable anymore as this data is private.
- I could have easily moved the new APIs there but I strongly felt this was
inappropriate..
---
