ekaterinadimitrova2 commented on code in PR #2488:
URL: https://github.com/apache/cassandra/pull/2488#discussion_r1264099959
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -356,12 +366,17 @@ public void writeMarshallablePayload(WireOut wire)
@Override
public int weight()
{
- return Ints.checkedCast(ObjectSizes.sizeOf(query)) +
super.weight();
+ return Ints.checkedCast(EMPTY_SIZE + ObjectSizes.sizeOf(query) +
super.fielsdSize());
Review Comment:
```suggestion
return Ints.checkedCast(EMPTY_SIZE + ObjectSizes.sizeOf(query) +
super.fieldsSize());
```
##########
src/java/org/apache/cassandra/utils/ObjectSizes.java:
##########
@@ -105,15 +98,12 @@ public static long sizeOfReferenceArray(int length)
*/
public static long sizeOfArray(Object[] objects)
{
- if (objects == null)
- return 0;
-
- return sizeOfReferenceArray(objects.length);
+ return meter.measureArray(objects);
Review Comment:
If we substitute `sizeOfReferenceArray` with `meter.measureArray` here, why
don't we do it everywhere?
Did you opt out of that as it will be easier for people to read it around
the codebase, or you did not think of that?
So effectively, we also have the null check in `sizeOfReferenceArray,` where
it was not added before.
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
Review Comment:
```suggestion
ByteBuffer buffer = ByteBuffer.allocate(10).duplicate().limit(8);
```
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSizeExcludingData(buffer, 8);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
+ checkBufferSizeExcludingData(buffer, 9);
- // heap buffer being a prefix slice of other buffer
- buffers = new ByteBuffer[]{ prefixBuf8 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being a suffix slice of other buffer
- buffers = new ByteBuffer[]{ suffixBuf9 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being an infix slice of other buffer
- buffers = new ByteBuffer[]{ infixBuf7 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
+ // heap buffer being an infix slab
+ buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().position(1).limit(8);
+ checkBufferSizeExcludingData(buffer, 7);
}
+ private void checkBufferSizeExcludingData(ByteBuffer buffer, int dataSize)
+ {
+ assertEquals(meter.measureDeep(buffer, SLAB_ALLOCATION_NO_SLICE) -
dataSize, ObjectSizes.sizeOnHeapExcludingDataOf(buffer));
+ };
+
@Test
public void testSizeOnHeapOf()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSize(ByteBuffer.allocate(0));
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_SIZE);
+ checkBufferSize(ByteBuffer.allocate(10));
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(0));
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(10));
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_2_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE + EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
Review Comment:
```suggestion
ByteBuffer buffer = ByteBuffer.allocate(10).duplicate().limit(8);
```
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -508,16 +545,14 @@ public void release()
queryOptionsBuffer.release();
}
- @Override
- public int weight()
+ /**
+ * Returns the sum of the non primitive fields sizes.
+ * @return the sum of the non primitive fields sizes.
+ */
+ protected long fielsdSize()
{
- return OBJECT_HEADER_SIZE
- + Long.BYTES
// queryStartTime
- + Integer.BYTES
// protocolVersion
- + OBJECT_REFERENCE_SIZE + EMPTY_BYTEBUF_SIZE +
queryOptionsBuffer.capacity() // queryOptionsBuffer
- + Long.BYTES
// generatedTimestamp
- + Long.BYTES
// generatedNowInSeconds
- + OBJECT_REFERENCE_SIZE +
Ints.checkedCast(ObjectSizes.sizeOf(keyspace)); // keyspace
+ return EMPTY_BYTEBUF_SIZE + queryOptionsBuffer.capacity() //
queryOptionsBuffer
+ + ObjectSizes.sizeOf(keyspace); //
keyspace
Review Comment:
Plus I think it will be good to explain what and why measure in the javadoc
for both methods.
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -380,25 +395,33 @@ public Batch(BatchStatement.Type batchType,
this.values = values;
this.batchType = batchType;
- int weight = super.weight();
-
- // weight, queries, values, batch type
- weight += Integer.BYTES + // cached weight
- 2 * EMPTY_LIST_SIZE + // queries + values lists
- 3 * OBJECT_REFERENCE_SIZE; // batchType and two lists
references
-
+ // We assume that all the lists are ArrayLists and that the size
of each underlying array is the one of the list
Review Comment:
I did not find anything that was not an ArrayList so I don't think it is
just an assumption, more like strong requirement.
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -508,16 +545,14 @@ public void release()
queryOptionsBuffer.release();
}
- @Override
- public int weight()
+ /**
+ * Returns the sum of the non primitive fields sizes.
+ * @return the sum of the non primitive fields sizes.
+ */
+ protected long fielsdSize()
{
- return OBJECT_HEADER_SIZE
- + Long.BYTES
// queryStartTime
- + Integer.BYTES
// protocolVersion
- + OBJECT_REFERENCE_SIZE + EMPTY_BYTEBUF_SIZE +
queryOptionsBuffer.capacity() // queryOptionsBuffer
- + Long.BYTES
// generatedTimestamp
- + Long.BYTES
// generatedNowInSeconds
- + OBJECT_REFERENCE_SIZE +
Ints.checkedCast(ObjectSizes.sizeOf(keyspace)); // keyspace
+ return EMPTY_BYTEBUF_SIZE + queryOptionsBuffer.capacity() //
queryOptionsBuffer
+ + ObjectSizes.sizeOf(keyspace); //
keyspace
Review Comment:
Wondering why was there the `Ints.checkedCast`? Precaution? Do we need it
back?
##########
src/java/org/apache/cassandra/utils/ObjectSizes.java:
##########
@@ -134,65 +124,89 @@ public static long sizeOnHeapOf(ByteBuffer[] array)
/**
* Amount of non-data heap memory consumed by the array of byte buffers.
It sums memory consumed
- * by the array itself and for each included byte buffer using {@link
#sizeOnHeapExcludingData(ByteBuffer)}.
+ * by the array itself and for each included byte buffer using {@link
#sizeOnHeapExcludingDataOf(ByteBuffer)}.
*/
- public static long sizeOnHeapExcludingData(ByteBuffer[] array)
+ public static long sizeOnHeapExcludingDataOf(ByteBuffer[] array)
{
if (array == null)
return 0;
long sum = sizeOfArray(array);
for (ByteBuffer b : array)
- sum += sizeOnHeapExcludingData(b);
+ sum += sizeOnHeapExcludingDataOf(b);
return sum;
}
/**
- * @return heap memory consumed by the byte buffer. If it is a slice, it
counts the data size, but it does not
- * include the internal array overhead.
+ * Measures the heap memory used by the specified byte buffer. If the
buffer is a slab only the data size will be
+ * counted but not the internal overhead. A SLAB is assumed to be created
by: {@code buffer.duplicate().position(start).limit(end)} without the use of
{@code slice()}.
+ * <p>This method makes a certain amount of assumptions:
+ * <ul>
+ * <li>That slabs are always created using: {@code
buffer.duplicate().position(start).limit(end)} and not through slice</li>
+ * <li>That the input buffers are not read-only buffers</li>
+ * <li>That the direct buffers that are not slab are not
duplicates</li>
+ * </ul>
+ * Non-respect of those assumptions can lead to an invalid value being
returned.
+ * @param buffer the buffer to measure
+ * @return the heap memory used by the specified byte buffer.
*/
public static long sizeOnHeapOf(ByteBuffer buffer)
{
if (buffer == null)
return 0;
- if (buffer.isDirect())
- return DIRECT_BUFFER_HEAP_SIZE;
+ assert !buffer.isReadOnly();
- int arrayLen = buffer.array().length;
- int bufLen = buffer.remaining();
+ // We assume here that slabs are always created using:
buffer.duplicate().position(start).limit(end) and not through slice
+ if (ByteBufferMode.SLAB_ALLOCATION_NO_SLICE.isSlab(buffer))
+ {
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_SHALLOW_SIZE; // We ignore the underlying
buffer
+
+ return HEAP_BUFFER_SHALLOW_SIZE + buffer.remaining(); // We ignore
the array overhead
+ }
- // if we're only referencing a sub-portion of the ByteBuffer, don't
count the array overhead (assume it is SLAB
- // allocated - the overhead amortized over all the allocations is
negligible and better to undercount than over)
- if (arrayLen > bufLen)
- return EMPTY_HEAP_BUFFER_SIZE + bufLen;
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_DEEP_SIZE; // That might not be true if the
buffer is a view of another buffer so we could undercount
- return EMPTY_HEAP_BUFFER_SIZE + (arrayLen == 0 ? EMPTY_BYTE_ARRAY_SIZE
: sizeOfArray(arrayLen, 1));
+ return HEAP_BUFFER_SHALLOW_SIZE + meter.measureArray(buffer.array());
Review Comment:
We don't need EMPTY_BYTE_ARRAY_SIZE, anymore? Isn't it gonna return 0? Do we
have a test case or so?
##########
src/java/org/apache/cassandra/utils/ObjectSizes.java:
##########
@@ -105,15 +98,12 @@ public static long sizeOfReferenceArray(int length)
*/
public static long sizeOfArray(Object[] objects)
{
- if (objects == null)
- return 0;
-
- return sizeOfReferenceArray(objects.length);
+ return meter.measureArray(objects);
}
- private static long sizeOfArray(int length, long elementSize)
+ private static long sizeOfArray(int length, int elementSize)
Review Comment:
So the `getReferenceSize` was always int, and long was for nothing here...
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -508,16 +545,14 @@ public void release()
queryOptionsBuffer.release();
}
- @Override
- public int weight()
+ /**
+ * Returns the sum of the non primitive fields sizes.
+ * @return the sum of the non primitive fields sizes.
Review Comment:
```suggestion
* Returns the sum of the non-primitive fields' sizes.
* @return the sum of the non-primitive fields' sizes.
```
##########
src/java/org/apache/cassandra/utils/ObjectSizes.java:
##########
@@ -134,65 +124,89 @@ public static long sizeOnHeapOf(ByteBuffer[] array)
/**
* Amount of non-data heap memory consumed by the array of byte buffers.
It sums memory consumed
- * by the array itself and for each included byte buffer using {@link
#sizeOnHeapExcludingData(ByteBuffer)}.
+ * by the array itself and for each included byte buffer using {@link
#sizeOnHeapExcludingDataOf(ByteBuffer)}.
*/
- public static long sizeOnHeapExcludingData(ByteBuffer[] array)
+ public static long sizeOnHeapExcludingDataOf(ByteBuffer[] array)
{
if (array == null)
return 0;
long sum = sizeOfArray(array);
for (ByteBuffer b : array)
- sum += sizeOnHeapExcludingData(b);
+ sum += sizeOnHeapExcludingDataOf(b);
return sum;
}
/**
- * @return heap memory consumed by the byte buffer. If it is a slice, it
counts the data size, but it does not
- * include the internal array overhead.
+ * Measures the heap memory used by the specified byte buffer. If the
buffer is a slab only the data size will be
+ * counted but not the internal overhead. A SLAB is assumed to be created
by: {@code buffer.duplicate().position(start).limit(end)} without the use of
{@code slice()}.
+ * <p>This method makes a certain amount of assumptions:
+ * <ul>
+ * <li>That slabs are always created using: {@code
buffer.duplicate().position(start).limit(end)} and not through slice</li>
+ * <li>That the input buffers are not read-only buffers</li>
+ * <li>That the direct buffers that are not slab are not
duplicates</li>
+ * </ul>
+ * Non-respect of those assumptions can lead to an invalid value being
returned.
+ * @param buffer the buffer to measure
+ * @return the heap memory used by the specified byte buffer.
*/
public static long sizeOnHeapOf(ByteBuffer buffer)
{
if (buffer == null)
return 0;
- if (buffer.isDirect())
- return DIRECT_BUFFER_HEAP_SIZE;
+ assert !buffer.isReadOnly();
- int arrayLen = buffer.array().length;
- int bufLen = buffer.remaining();
+ // We assume here that slabs are always created using:
buffer.duplicate().position(start).limit(end) and not through slice
+ if (ByteBufferMode.SLAB_ALLOCATION_NO_SLICE.isSlab(buffer))
+ {
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_SHALLOW_SIZE; // We ignore the underlying
buffer
+
+ return HEAP_BUFFER_SHALLOW_SIZE + buffer.remaining(); // We ignore
the array overhead
+ }
- // if we're only referencing a sub-portion of the ByteBuffer, don't
count the array overhead (assume it is SLAB
- // allocated - the overhead amortized over all the allocations is
negligible and better to undercount than over)
- if (arrayLen > bufLen)
- return EMPTY_HEAP_BUFFER_SIZE + bufLen;
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_DEEP_SIZE; // That might not be true if the
buffer is a view of another buffer so we could undercount
- return EMPTY_HEAP_BUFFER_SIZE + (arrayLen == 0 ? EMPTY_BYTE_ARRAY_SIZE
: sizeOfArray(arrayLen, 1));
+ return HEAP_BUFFER_SHALLOW_SIZE + meter.measureArray(buffer.array());
}
/**
- * @return non-data heap memory consumed by the byte buffer. If it is a
slice, it does not include the internal
- * array overhead.
+ * Measures the heap memory used by the specified byte buffer excluding
the data. If the buffer shallow size will be counted.
+ * A SLAB is assumed to be created by: {@code
buffer.duplicate().position(start).limit(end)} without the use of {@code
slice()}.
+ * <p>This method makes a certain amount of assumptions:
+ * <ul>
+ * <li>That slabs are always created using: {@code
buffer.duplicate().position(start).limit(end)} and not through slice</li>
+ * <li>That the input buffers are not read-only buffers</li>
+ * <li>That the direct buffers that are not slab are not
duplicates</li>
+ * </ul>
+ * Non-respect of those assumptions can lead to an invalid value being
returned. T
+ * @param buffer the buffer to measure
+ * @return the heap memory used by the specified byte buffer excluding the
data..
*/
- public static long sizeOnHeapExcludingData(ByteBuffer buffer)
+ public static long sizeOnHeapExcludingDataOf(ByteBuffer buffer)
{
if (buffer == null)
return 0;
- if (buffer.isDirect())
- return DIRECT_BUFFER_HEAP_SIZE;
+ assert !buffer.isReadOnly();
- int arrayLen = buffer.array().length;
- int bufLen = buffer.remaining();
+ // We assume here that slabs are always created using:
buffer.duplicate().position(start).limit(end) and not through slice
+ if (ByteBufferMode.SLAB_ALLOCATION_NO_SLICE.isSlab(buffer))
+ {
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_SHALLOW_SIZE; // We ignore the underlying
buffer
- // if we're only referencing a sub-portion of the ByteBuffer, don't
count the array overhead (assume it is SLAB
- // allocated - the overhead amortized over all the allocations is
negligible and better to undercount than over)
- if (arrayLen > bufLen)
- return EMPTY_HEAP_BUFFER_SIZE;
+ return HEAP_BUFFER_SHALLOW_SIZE; // We ignore the array overhead
+ }
+
+ if (buffer.isDirect())
+ return DIRECT_BUFFER_DEEP_SIZE; // That might not be true if the
buffer is a view of another buffer so we could undercount
- // If buffers are dedicated, account for byte array size and any
padding overhead
- return EMPTY_HEAP_BUFFER_SIZE + (arrayLen == 0 ? EMPTY_BYTE_ARRAY_SIZE
: (sizeOfArray(arrayLen, 1) - arrayLen));
+ byte[] bytes = buffer.array();
+ return HEAP_BUFFER_SHALLOW_SIZE + meter.measureArray(bytes) -
bytes.length;
Review Comment:
Same question about EMPTY_BYTE_ARRAY_SIZE
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
Review Comment:
My assumption is you intentionally created the meter with `INSTRUMENTATION`
and not `INSTRUMENTATION_AND_SPEC` as in `ObjectSizes`, for additional
verification. Can we add a comment so people realize that and do not consider
it a bug and change it to `INSTRUMENTATION_AND_SPEC`?
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSizeExcludingData(buffer, 8);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
+ checkBufferSizeExcludingData(buffer, 9);
- // heap buffer being a prefix slice of other buffer
- buffers = new ByteBuffer[]{ prefixBuf8 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being a suffix slice of other buffer
- buffers = new ByteBuffer[]{ suffixBuf9 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being an infix slice of other buffer
- buffers = new ByteBuffer[]{ infixBuf7 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
+ // heap buffer being an infix slab
+ buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().position(1).limit(8);
Review Comment:
```suggestion
buffer = ByteBuffer.allocate(10).duplicate().position(1).limit(8);
```
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSizeExcludingData(buffer, 8);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
Review Comment:
```suggestion
buffer = ByteBuffer.allocate(10).duplicate().position(1);
```
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSizeExcludingData(buffer, 8);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
+ checkBufferSizeExcludingData(buffer, 9);
- // heap buffer being a prefix slice of other buffer
- buffers = new ByteBuffer[]{ prefixBuf8 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being a suffix slice of other buffer
- buffers = new ByteBuffer[]{ suffixBuf9 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being an infix slice of other buffer
- buffers = new ByteBuffer[]{ infixBuf7 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
+ // heap buffer being an infix slab
+ buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().position(1).limit(8);
+ checkBufferSizeExcludingData(buffer, 7);
}
+ private void checkBufferSizeExcludingData(ByteBuffer buffer, int dataSize)
+ {
+ assertEquals(meter.measureDeep(buffer, SLAB_ALLOCATION_NO_SLICE) -
dataSize, ObjectSizes.sizeOnHeapExcludingDataOf(buffer));
+ };
+
@Test
public void testSizeOnHeapOf()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSize(ByteBuffer.allocate(0));
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_SIZE);
+ checkBufferSize(ByteBuffer.allocate(10));
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(0));
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(10));
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_2_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE + EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSize(buffer);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_2_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
Review Comment:
```suggestion
buffer = ByteBuffer.allocate(10).duplicate().position(1);
```
##########
test/unit/org/apache/cassandra/db/memtable/MemtableSizeTestBase.java:
##########
@@ -39,16 +39,20 @@
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.utils.FBUtilities;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
// Note: This test can be run in idea with the allocation type configured in
the test yaml and memtable using the
// value memtableClass is initialized with.
@RunWith(Parameterized.class)
public abstract class MemtableSizeTestBase extends CQLTester
{
- // Note: To see a printout of the usage for each object, add
.enableDebug() here (most useful with smaller number of
+ // Note: To see a printout of the usage for each object, add
.printVisitedTree() here (most useful with smaller number of
// partitions).
- static MemoryMeter meter = new MemoryMeter().ignoreKnownSingletons()
-
.withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE);
+ static MemoryMeter meter = MemoryMeter.builder()
Review Comment:
Why do we need a new meter, and we do not do `ObjectSizes.measureDeep`
instead? CC @blambov
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSizeExcludingData(buffer, 8);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_2_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
+ checkBufferSizeExcludingData(buffer, 9);
- // heap buffer being a prefix slice of other buffer
- buffers = new ByteBuffer[]{ prefixBuf8 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being a suffix slice of other buffer
- buffers = new ByteBuffer[]{ suffixBuf9 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
-
- // heap buffer being an infix slice of other buffer
- buffers = new ByteBuffer[]{ infixBuf7 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE);
+ // heap buffer being an infix slab
+ buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().position(1).limit(8);
+ checkBufferSizeExcludingData(buffer, 7);
}
+ private void checkBufferSizeExcludingData(ByteBuffer buffer, int dataSize)
+ {
+ assertEquals(meter.measureDeep(buffer, SLAB_ALLOCATION_NO_SLICE) -
dataSize, ObjectSizes.sizeOnHeapExcludingDataOf(buffer));
+ };
+
@Test
public void testSizeOnHeapOf()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSize(ByteBuffer.allocate(0));
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_SIZE);
+ checkBufferSize(ByteBuffer.allocate(10));
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(0));
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSize(ByteBuffer.allocateDirect(10));
- // two different empty byte buffers
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0),
ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_2_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE + EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a prefix slab
+ ByteBuffer buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().limit(8);
+ checkBufferSize(buffer);
- // two different non-empty byte buffers
- buffers = new ByteBuffer[]{ buf10, ByteBuffer.allocateDirect(500) };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_2_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_SIZE +
EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ // heap buffer being a suffix slab
+ buffer = (ByteBuffer) ByteBuffer.allocate(10).duplicate().position(1);
+ checkBufferSize(buffer);
- // heap buffer being a prefix slice of other buffer
- buffers = new ByteBuffer[]{ prefixBuf8 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + 8);
-
- // heap buffer being a suffix slice of other buffer
- buffers = new ByteBuffer[]{ suffixBuf9 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + 9);
-
- // heap buffer being an infix slice of other buffer
- buffers = new ByteBuffer[]{ infixBuf7 };
-
assertThat(ObjectSizes.sizeOnHeapOf(buffers)).isEqualTo(REF_ARRAY_1_SIZE +
EMPTY_HEAP_BUFFER_RAW_SIZE + 7);
+ // heap buffer being an infix slab
+ buffer = (ByteBuffer)
ByteBuffer.allocate(10).duplicate().position(1).limit(8);
Review Comment:
```suggestion
buffer = ByteBuffer.allocate(10).duplicate().position(1).limit(8);
```
##########
test/unit/org/apache/cassandra/utils/ObjectSizesTest.java:
##########
@@ -22,128 +22,81 @@
import org.junit.Test;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.Guess;
-import static org.assertj.core.api.Assertions.assertThat;
+import static
org.github.jamm.MemoryMeter.ByteBufferMode.SLAB_ALLOCATION_NO_SLICE;
+import static org.junit.Assert.assertEquals;
public class ObjectSizesTest
{
- private static final MemoryMeter meter = new
MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_UNSAFE).omitSharedBufferOverhead().ignoreKnownSingletons();
-
- private static final long EMPTY_HEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocate(0));
- private static final long EMPTY_OFFHEAP_BUFFER_RAW_SIZE =
meter.measure(ByteBuffer.allocateDirect(0));
- private static final ByteBuffer[] EMPTY_BYTE_BUFFER_ARRAY = new
ByteBuffer[0];
-
- public static final long REF_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_1_SIZE =
MemoryLayoutSpecification.sizeOfArray(1,
MemoryLayoutSpecification.SPEC.getReferenceSize());
- public static final long REF_ARRAY_2_SIZE =
MemoryLayoutSpecification.sizeOfArray(2,
MemoryLayoutSpecification.SPEC.getReferenceSize());
-
- public static final long BYTE_ARRAY_0_SIZE =
MemoryLayoutSpecification.sizeOfArray(0, 1);
- public static final long BYTE_ARRAY_10_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1);
- public static final long BYTE_ARRAY_10_EXCEPT_DATA_SIZE =
MemoryLayoutSpecification.sizeOfArray(10, 1) - 10;
-
- private ByteBuffer buf10 = ByteBuffer.allocate(10);
- private ByteBuffer prefixBuf8 = buf10.duplicate();
- private ByteBuffer suffixBuf9 = buf10.duplicate();
- private ByteBuffer infixBuf7 = buf10.duplicate();
-
- {
- prefixBuf8.limit(8);
-
- suffixBuf9.position(1);
- suffixBuf9 = suffixBuf9.slice();
-
- infixBuf7.limit(8);
- infixBuf7.position(1);
- infixBuf7 = infixBuf7.slice();
- }
+ private static final MemoryMeter meter = MemoryMeter.builder()
+
.withGuessing(Guess.INSTRUMENTATION, Guess.UNSAFE)
+ .build();
@Test
public void testSizeOnHeapExcludingData()
{
- // empty array of byte buffers
- ByteBuffer[] buffers = EMPTY_BYTE_BUFFER_ARRAY;
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_0_SIZE);
-
// single empty heap buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocate(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_0_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(0), 0);
// single non-empty heap buffer
- buffers = new ByteBuffer[]{ buf10 };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_HEAP_BUFFER_RAW_SIZE + BYTE_ARRAY_10_EXCEPT_DATA_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocate(10), 10);
// single empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(0) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(0), 0);
// single non-empty direct buffer
- buffers = new ByteBuffer[]{ ByteBuffer.allocateDirect(10) };
-
assertThat(ObjectSizes.sizeOnHeapExcludingData(buffers)).isEqualTo(REF_ARRAY_1_SIZE
+ EMPTY_OFFHEAP_BUFFER_RAW_SIZE);
+ checkBufferSizeExcludingData(ByteBuffer.allocateDirect(10), 0);
- // two different empty byte buffers
Review Comment:
Why not measuring arrays anymore?
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -441,6 +464,7 @@ private static abstract class AbstractLogEntry extends
BinLog.ReleaseableWriteMa
{
private final long queryStartTime;
private final int protocolVersion;
+ @Unmetered
private final ByteBuf queryOptionsBuffer;
Review Comment:
Is this `@Unmetered` some leftover? We measure `queryOptionsBuffer` by
getting its capacity directly.
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -508,16 +545,14 @@ public void release()
queryOptionsBuffer.release();
}
- @Override
- public int weight()
+ /**
+ * Returns the sum of the non primitive fields sizes.
+ * @return the sum of the non primitive fields sizes.
+ */
+ protected long fielsdSize()
{
- return OBJECT_HEADER_SIZE
- + Long.BYTES
// queryStartTime
- + Integer.BYTES
// protocolVersion
- + OBJECT_REFERENCE_SIZE + EMPTY_BYTEBUF_SIZE +
queryOptionsBuffer.capacity() // queryOptionsBuffer
- + Long.BYTES
// generatedTimestamp
- + Long.BYTES
// generatedNowInSeconds
- + OBJECT_REFERENCE_SIZE +
Ints.checkedCast(ObjectSizes.sizeOf(keyspace)); // keyspace
+ return EMPTY_BYTEBUF_SIZE + queryOptionsBuffer.capacity() //
queryOptionsBuffer
+ + ObjectSizes.sizeOf(keyspace); //
keyspace
Review Comment:
I think we need also to add EMPTY_SIZE for the
`BinAuditLogger#Message#weight()`
##########
src/java/org/apache/cassandra/fql/FullQueryLogger.java:
##########
@@ -508,16 +545,14 @@ public void release()
queryOptionsBuffer.release();
}
- @Override
- public int weight()
+ /**
+ * Returns the sum of the non primitive fields sizes.
+ * @return the sum of the non primitive fields sizes.
+ */
+ protected long fielsdSize()
{
- return OBJECT_HEADER_SIZE
- + Long.BYTES
// queryStartTime
- + Integer.BYTES
// protocolVersion
- + OBJECT_REFERENCE_SIZE + EMPTY_BYTEBUF_SIZE +
queryOptionsBuffer.capacity() // queryOptionsBuffer
- + Long.BYTES
// generatedTimestamp
- + Long.BYTES
// generatedNowInSeconds
- + OBJECT_REFERENCE_SIZE +
Ints.checkedCast(ObjectSizes.sizeOf(keyspace)); // keyspace
+ return EMPTY_BYTEBUF_SIZE + queryOptionsBuffer.capacity() //
queryOptionsBuffer
+ + ObjectSizes.sizeOf(keyspace); //
keyspace
Review Comment:
It seems the weight was a deep size, and to differentiate it from what you
decided to weigh here, you changed the name? Is this correct? So only I need to
know why the primitives were excluded if that was the case.
##########
src/java/org/apache/cassandra/utils/ObjectSizes.java:
##########
@@ -24,23 +24,25 @@
import java.net.UnknownHostException;
import java.nio.ByteBuffer;
-import org.github.jamm.MemoryLayoutSpecification;
import org.github.jamm.MemoryMeter;
+import org.github.jamm.MemoryMeter.ByteBufferMode;
+import org.github.jamm.MemoryMeter.Guess;
+
+import static org.github.jamm.MemoryMeterStrategy.MEMORY_LAYOUT;
+import static org.github.jamm.utils.ArrayMeasurementUtils.computeArraySize;
/**
* A convenience class for wrapping access to MemoryMeter
*/
public class ObjectSizes
Review Comment:
So after seeing the InMemoryTrie not using the `ObjectSizes` wrapper I have
a question... should we always use the wrapper and make it clear in the javadoc
here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
