rpuch commented on code in PR #787:
URL: https://github.com/apache/ignite-3/pull/787#discussion_r852951153
##########
modules/rocksdb-common/src/main/java/org/apache/ignite/internal/rocksdb/ColumnFamily.java:
##########
@@ -41,7 +41,8 @@ public class ColumnFamily {
private final String cfName;
/** Column family handle. */
- private final ColumnFamilyHandle cfHandle;
+ // Temporarily made public until the integration with MV-store is
completed.
+ public final ColumnFamilyHandle cfHandle;
Review Comment:
Do we really want to make a field public?
##########
modules/storage-api/src/main/java/org/apache/ignite/internal/storage/IgniteRowId.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.storage;
+
+import java.nio.ByteBuffer;
+
+/**
+ * Interface that represents row id in primary index of the table.
+ *
+ * @see MvPartitionStorage
+ */
+public interface IgniteRowId extends Comparable<IgniteRowId> {
+ /**
+ * Maximum possible row id size in bytes. If PK columns exceed this size,
then UUID-based row id should be used.
+ */
+ final int MAX_ROW_ID_SIZE = 16;
+
+ /**
+ * Writes row id into a byte buffer. Binary row representation should
match natural order defined by {@link #compareTo(Object)} when
+ * comparing lexicographically.
+ *
+ * @param buf Output byte buffer with {@link
java.nio.ByteOrder#LITTLE_ENDIAN} byte order.
+ * @param signedBytesCompare Defines properties of a target binary
comparator. {@code true} if bytes are compared as signed values,
+ * {@code false} if unsigned.
+ */
+ void writeTo(ByteBuffer buf, boolean signedBytesCompare);
+
+ /**
+ * Compares row id with a byte buffer, previously ritten by a {@link
#writeTo(ByteBuffer, boolean)} method.
+ *
+ * @param buf Input byte buffer with {@link
java.nio.ByteOrder#LITTLE_ENDIAN} byte order.
+ * @param signedBytesCompare Defines properties of a binary comparator.
{@code true} if bytes are compared as signed values,
+ * {@code false} if unsigned.
+ */
Review Comment:
Let's also add a `@return` explaining what this method returns
##########
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbMvPartitionStorage.java:
##########
@@ -0,0 +1,489 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.storage.rocksdb;
+
+import static java.lang.ThreadLocal.withInitial;
+import static java.nio.ByteBuffer.allocateDirect;
+import static java.nio.ByteOrder.BIG_ENDIAN;
+import static org.apache.ignite.internal.storage.IgniteRowId.MAX_ROW_ID_SIZE;
+import static org.apache.ignite.internal.util.ArrayUtils.BYTE_EMPTY_ARRAY;
+
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.UUID;
+import org.apache.ignite.internal.schema.BinaryRow;
+import org.apache.ignite.internal.schema.ByteBufferRow;
+import org.apache.ignite.internal.storage.IgniteRowId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.StorageException;
+import org.apache.ignite.internal.storage.TxIdMismatchException;
+import org.apache.ignite.internal.tx.Timestamp;
+import org.apache.ignite.internal.util.Cursor;
+import org.apache.ignite.internal.util.GridUnsafe;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.lang.IgniteInternalException;
+import org.jetbrains.annotations.Nullable;
+import org.rocksdb.ColumnFamilyHandle;
+import org.rocksdb.ReadOptions;
+import org.rocksdb.RocksDB;
+import org.rocksdb.RocksDBException;
+import org.rocksdb.RocksIterator;
+import org.rocksdb.Slice;
+import org.rocksdb.WriteOptions;
+
+/**
+ * Mult-versioned partition storage implementation based on RocksDB. Stored
data has the following format:
+ * <pre><code>
+ * | partId (2 bytes, BE) | rowId ({@link #igniteRowIdSize} bytes)
|</code></pre>
+ * or
+ * <pre><code>
+ * | partId (2 bytes, BE) | rowId ({@link #igniteRowIdSize} bytes) | timestamp
(16 bytes, DESC) |</code></pre>
+ * depending on transaction status. Pending transactions data doesn't have a
timestamp assigned.
+ *
+ * <p/>BE means Big Endian, meaning that lexicographical bytes order matches a
natural order of partitions.
+ *
+ * <p/>DESC means that timestamps are sorted from newest to oldest (N2O).
Please refer to {@link #putTimestamp(ByteBuffer, Timestamp)} to
+ * see how it's achieved. Missing timestamp could be interpreted as a moment
infinitely far away in the future.
+ */
+public class RocksDbMvPartitionStorage implements MvPartitionStorage {
+ /** Position of row id inside of the key. */
+ private static final int ROW_ID_OFFSET = Short.BYTES;
+
+ /** Timestamp size in bytes. */
+ private static final int TIMESTAMP_SIZE = 2 * Long.BYTES;
+
+ /** Maximum possible size of the key. */
+ private static final int MAX_KEY_SIZE = /* partId */ ROW_ID_OFFSET + /*
rowId */ MAX_ROW_ID_SIZE + /* timestamp */ TIMESTAMP_SIZE;
+
+ /** Threadlocal direct buffer instance to read keys from RocksDB. */
+ private static final ThreadLocal<ByteBuffer> MV_KEY_BUFFER =
withInitial(() -> allocateDirect(MAX_KEY_SIZE).order(BIG_ENDIAN));
+
+ /** Threadlocal ob-heap byte buffer instance to use for key manipulations.
*/
Review Comment:
on-heap
##########
modules/storage-api/src/main/java/org/apache/ignite/internal/storage/UuidIgniteRowId.java:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.storage;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.util.UUID;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * UUID-based ignite row id implementation.
+ */
+public class UuidIgniteRowId implements IgniteRowId {
+ /** Backing uuid value. */
+ private final UUID uuid;
+
+ /**
+ * Constructor.
+ *
+ * @param uuid UUID.
+ */
+ public UuidIgniteRowId(UUID uuid) {
+ this.uuid = uuid;
+ }
+
+ /**
+ * Returns {@link UuidIgniteRowId} instance based on {@link
UUID#randomUUID()}.
+ */
+ public static IgniteRowId randomRowId() {
+ return new UuidIgniteRowId(UUID.randomUUID());
+ }
+
+ /** {@inheritDoc} */
+ @Override
+ public void writeTo(ByteBuffer buf, boolean signedBytesCompare) {
+ assert buf.order() == ByteOrder.BIG_ENDIAN;
+
+ long mask = longBytesSignsMask(signedBytesCompare);
+
+ buf.putLong(mask ^ uuid.getMostSignificantBits());
+ buf.putLong(mask ^ uuid.getLeastSignificantBits());
+ }
+
+ /** {@inheritDoc} */
+ @Override
+ public int compare(ByteBuffer buf, boolean signedBytesCompare) {
+ assert buf.order() == ByteOrder.BIG_ENDIAN;
+
+ long mask = longBytesSignsMask(signedBytesCompare);
+
+ int cmp = Long.compare(uuid.getMostSignificantBits(), mask ^
buf.getLong());
+
+ if (cmp != 0) {
+ return cmp;
+ }
+
+ return Long.compare(uuid.getLeastSignificantBits(), mask ^
buf.getLong());
+ }
+
+ /** {@inheritDoc} */
+ @Override
+ public int compareTo(@NotNull IgniteRowId o) {
+ if (!(o instanceof UuidIgniteRowId)) {
+ throw new IllegalArgumentException(o.getClass().getName());
Review Comment:
I suggest making the error message more specific, like "I can only be
compared to ..." instead of stating the class name. This would be a bit more
friendly to the one who trips this exception.
##########
modules/storage-api/src/main/java/org/apache/ignite/internal/storage/UuidIgniteRowId.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.storage;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.util.UUID;
+import org.jetbrains.annotations.NotNull;
+
+/**
+ * UUID-based ignite row id implementation.
+ */
+public class UuidIgniteRowId implements IgniteRowId {
+ /** Backing uuid value. */
+ private final UUID uuid;
+
+ /**
+ * Constructor.
+ *
+ * @param uuid UUID.
+ */
+ public UuidIgniteRowId(UUID uuid) {
+ this.uuid = uuid;
+ }
+
+ /**
+ * Returns {@link UuidIgniteRowId} instance based on {@link
UUID#randomUUID()}.
+ */
+ public static IgniteRowId randomRowId() {
+ return new UuidIgniteRowId(UUID.randomUUID());
+ }
+
+ /** {@inheritDoc} */
+ @Override
+ public void writeTo(ByteBuffer buf, boolean signedBytesCompare) {
+ assert buf.order() == ByteOrder.LITTLE_ENDIAN;
+
+ long mask = signedBytesCompare ? 0x0080808080808080L :
0x8000000000000000L;
Review Comment:
Why such masks?
##########
modules/storage-api/src/main/java/org/apache/ignite/internal/storage/IgniteRowId.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.storage;
+
+import java.nio.ByteBuffer;
+
+/**
+ * Interface that represents row id in primary index of the table.
+ *
+ * @see MvPartitionStorage
+ */
+public interface IgniteRowId extends Comparable<IgniteRowId> {
+ /**
+ * Maximum possible row id size in bytes. If PK columns exceed this size,
then UUID-based row id should be used.
+ */
+ final int MAX_ROW_ID_SIZE = 16;
+
+ /**
+ * Writes row id into a byte buffer. Binary row representation should
match natural order defined by {@link #compareTo(Object)} when
+ * comparing lexicographically.
+ *
+ * @param buf Output byte buffer with {@link
java.nio.ByteOrder#BIG_ENDIAN} byte order.
+ * @param signedBytesCompare Defines properties of a target binary
comparator. {@code true} if bytes are compared as signed values,
+ * {@code false} if unsigned.
+ */
+ void writeTo(ByteBuffer buf, boolean signedBytesCompare);
+
+ /**
+ * Compares row id with a byte buffer, previously ritten by a {@link
#writeTo(ByteBuffer, boolean)} method.
+ *
+ * @param buf Input byte buffer with {@link java.nio.ByteOrder#BIG_ENDIAN}
byte order.
+ * @param signedBytesCompare Defines properties of a binary comparator.
{@code true} if bytes are compared as signed values,
+ * {@code false} if unsigned.
+ */
+ int compare(ByteBuffer buf, boolean signedBytesCompare);
Review Comment:
How about renaming the method to `compareTo()` so that it becomes clear
'what' is compared to 'what'?
##########
modules/storage-api/src/main/java/org/apache/ignite/internal/storage/MvPartitionStorage.java:
##########
@@ -48,31 +47,30 @@ public interface MvPartitionStorage {
* @throws TxIdMismatchException If there's another pending update
associated with different transaction id.
* @throws StorageException If failed to write data to the storage.
*/
- void addWrite(BinaryRow row, UUID txId) throws TxIdMismatchException,
StorageException;
+ void addWrite(IgniteRowId rowId, @Nullable BinaryRow row, UUID txId)
throws TxIdMismatchException, StorageException;
Review Comment:
Does `row` represent just a value now, or it's still the full key+value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
