sashapolo commented on a change in pull request #566: URL: https://github.com/apache/ignite-3/pull/566#discussion_r789681775
########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/FullPageId.java ########## @@ -0,0 +1,174 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory; + +import org.apache.ignite.internal.pagememory.util.PageIdUtils; +import org.apache.ignite.internal.util.IgniteUtils; +import org.apache.ignite.lang.IgniteStringBuilder; + +/** + * Compound object used to address a page in the global page space. + * <h3>Page ID structure</h3> + * + * <p>Generally, a full page ID consists of a group ID and a page ID. A page ID consists of a page index (32 bits), a partition ID (16 bits) + * and flags. Group ID is an integer inentifier of a logical pages group, like a specific SQL table or metadata storage, for example. + * Page index is the unique page identifier inside of a specific partition of a specific group. Set of indexes in the partition represents Review comment: ```suggestion * Page index is a unique page identifier inside of a specific partition of a page group. Set of indexes in the partition represents ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/FullPageId.java ########## @@ -0,0 +1,174 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory; + +import org.apache.ignite.internal.pagememory.util.PageIdUtils; +import org.apache.ignite.internal.util.IgniteUtils; +import org.apache.ignite.lang.IgniteStringBuilder; + +/** + * Compound object used to address a page in the global page space. + * <h3>Page ID structure</h3> + * + * <p>Generally, a full page ID consists of a group ID and a page ID. A page ID consists of a page index (32 bits), a partition ID (16 bits) + * and flags. Group ID is an integer inentifier of a logical pages group, like a specific SQL table or metadata storage, for example. Review comment: ```suggestion * and flags (8 bits). Group ID is an integer identifier of a logical pages group, like a SQL table or metadata storage, for example. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIo.java ########## @@ -0,0 +1,501 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.nio.ByteBuffer; +import org.apache.ignite.internal.pagememory.PageMemory; +import org.apache.ignite.internal.pagememory.metric.IoStatisticsHolder; +import org.apache.ignite.internal.pagememory.util.PageHandler; +import org.apache.ignite.internal.pagememory.util.PageIdUtils; +import org.apache.ignite.internal.pagememory.util.PageLockListener; +import org.apache.ignite.internal.pagememory.util.PageUtils; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Base format for all the page types. + * + * <p>Checklist for page IO implementations and usage (The Rules): + * + * <p>1. IO should not have any `public static` methods. We have versioned IOs and any static method will mean that it have to always work + * in backward compatible way between all the IO versions. The base class {@link PageIo} has static methods (like {@link #getPageId(long)}) + * intentionally: this base format can not be changed between versions. + * + * <p>2. IO must correctly override {@link #initNewPage(long, long, int)} method and call super. We have logic that relies on this behavior. + * + * <p>3. Always keep in mind that IOs are versioned and their format can change from version to version. In this respect it is a good + * practice to avoid exposing details of IO internal format on it's API. The API should be minimalistic and abstract, so that internal + * format in future IO version can be completely changed without any changes to the API of this page IO. + * + * <p>4. Page IO API should not have any version dependent semantics and should not change API semantics in newer versions. + * + * <p>5. It is almost always preferable to read or write (especially write) page contents using static methods on {@link PageHandler}. To + * just initialize new page use {@link PageHandler#initPage(PageMemory, int, long, PageIo, PageLockListener, IoStatisticsHolder)} method + * with needed IO instance. + */ +public abstract class PageIo { + /** + * Maximum value for page type. + */ + public static final int MAX_IO_TYPE = 65535 - 1; + + /** + * Offset for "short" page type. + */ + public static final int TYPE_OFF = 0; + + /** + * Offset for "short" page version. + */ + public static final int VER_OFF = TYPE_OFF + Short.BYTES; + + /** + * Offset for "int" CRC. + */ + public static final int CRC_OFF = VER_OFF + Short.BYTES; + + /** + * Offset for "long" page ID. + */ + public static final int PAGE_ID_OFF = CRC_OFF + Integer.BYTES; + + /** + * Offset for "byte" rotated ID. + */ + public static final int ROTATED_ID_PART_OFF = PAGE_ID_OFF + Long.BYTES; + + /** + * Offset for "byte" compression type. + */ + private static final int COMPRESSION_TYPE_OFF = ROTATED_ID_PART_OFF + Byte.BYTES; + + /** + * Offset for "short" compressed size. + */ + private static final int COMPRESSED_SIZE_OFF = COMPRESSION_TYPE_OFF + Byte.BYTES; + + /** + * Offset for "short" compracted size. Review comment: ```suggestion * Offset for "short" compacted size. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIoRegistry.java ########## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.nio.ByteBuffer; +import java.util.ServiceLoader; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Page IO Registry. This component registers and provides all available {@link PageIo} types. + */ +public class PageIoRegistry { + /** + * Arrays of {@link IoVersions} for fast access. Element 0 is reserved. + */ + private final IoVersions<?>[] ioVersions = new IoVersions[PageIo.MAX_IO_TYPE + 1]; + + /** + * Loads all {@link IoVersions} from corresponding {@link PageIoModule} using a {@link ServiceLoader} mechanism. + */ + public void loadFromServiceLoader() { + ServiceLoader<PageIoModule> serviceLoader = ServiceLoader.load(PageIoModule.class); + + for (PageIoModule pageIoModule : serviceLoader) { + for (IoVersions<?> ios : pageIoModule.ioVersions()) { + assert ios.getType() != 0 : "Type 0 is reserved and can't be used: " + ios; Review comment: I'm not sure whether this check should be an assertion and not a proper exception ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIo.java ########## @@ -0,0 +1,501 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.nio.ByteBuffer; +import org.apache.ignite.internal.pagememory.PageMemory; +import org.apache.ignite.internal.pagememory.metric.IoStatisticsHolder; +import org.apache.ignite.internal.pagememory.util.PageHandler; +import org.apache.ignite.internal.pagememory.util.PageIdUtils; +import org.apache.ignite.internal.pagememory.util.PageLockListener; +import org.apache.ignite.internal.pagememory.util.PageUtils; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Base format for all the page types. + * + * <p>Checklist for page IO implementations and usage (The Rules): + * + * <p>1. IO should not have any `public static` methods. We have versioned IOs and any static method will mean that it have to always work + * in backward compatible way between all the IO versions. The base class {@link PageIo} has static methods (like {@link #getPageId(long)}) + * intentionally: this base format can not be changed between versions. + * + * <p>2. IO must correctly override {@link #initNewPage(long, long, int)} method and call super. We have logic that relies on this behavior. + * + * <p>3. Always keep in mind that IOs are versioned and their format can change from version to version. In this respect it is a good + * practice to avoid exposing details of IO internal format on it's API. The API should be minimalistic and abstract, so that internal + * format in future IO version can be completely changed without any changes to the API of this page IO. + * + * <p>4. Page IO API should not have any version dependent semantics and should not change API semantics in newer versions. + * + * <p>5. It is almost always preferable to read or write (especially write) page contents using static methods on {@link PageHandler}. To + * just initialize new page use {@link PageHandler#initPage(PageMemory, int, long, PageIo, PageLockListener, IoStatisticsHolder)} method + * with needed IO instance. + */ +public abstract class PageIo { + /** + * Maximum value for page type. + */ + public static final int MAX_IO_TYPE = 65535 - 1; + + /** + * Offset for "short" page type. + */ + public static final int TYPE_OFF = 0; + + /** + * Offset for "short" page version. + */ + public static final int VER_OFF = TYPE_OFF + Short.BYTES; + + /** + * Offset for "int" CRC. + */ + public static final int CRC_OFF = VER_OFF + Short.BYTES; + + /** + * Offset for "long" page ID. + */ + public static final int PAGE_ID_OFF = CRC_OFF + Integer.BYTES; + + /** + * Offset for "byte" rotated ID. + */ + public static final int ROTATED_ID_PART_OFF = PAGE_ID_OFF + Long.BYTES; + + /** + * Offset for "byte" compression type. + */ + private static final int COMPRESSION_TYPE_OFF = ROTATED_ID_PART_OFF + Byte.BYTES; + + /** + * Offset for "short" compressed size. + */ + private static final int COMPRESSED_SIZE_OFF = COMPRESSION_TYPE_OFF + Byte.BYTES; + + /** + * Offset for "short" compracted size. + */ + private static final int COMPACTED_SIZE_OFF = COMPRESSED_SIZE_OFF + Short.BYTES; + + /** + * Offset for reserved "short" value. + */ + private static final int RESERVED_SHORT_OFF = COMPACTED_SIZE_OFF + Short.BYTES; + + /** + * Offset for reserved "long" value. + */ + private static final int RESERVED_2_OFF = RESERVED_SHORT_OFF + Short.BYTES; + + /** + * Offset for reserved "long" value. + */ + private static final int RESERVED_3_OFF = RESERVED_2_OFF + Long.BYTES; + + /** + * Total size of common header, including reserved bytes. + */ + public static final int COMMON_HEADER_END = RESERVED_3_OFF + Long.BYTES; + + /** + * IO version. + */ + private final int ver; + + /** + * IO type. + */ + private final int type; + + /** + * Constructor. + * + * @param type Page type. + * @param ver Page format version. + */ + protected PageIo(int type, int ver) { + assert ver > 0 && ver <= MAX_IO_TYPE : ver; + assert type > 0 && type <= MAX_IO_TYPE : type; + + this.type = type; + this.ver = ver; + } + + /** + * Returns a type. + */ + public final int getType() { + return type; + } + + /** + * Returns a page type. + * + * @param buf Buffer. + * @return Page type. + */ + public static int getType(ByteBuffer buf) { + return buf.getShort(TYPE_OFF) & 0xFFFF; + } + + /** + * Returns a page type. + * + * @param pageAddr Page address. + * @return Page type. + */ + public static int getType(long pageAddr) { + return PageUtils.getShort(pageAddr, TYPE_OFF) & 0xFFFF; + } + + /** + * Sets the type to the page. + * + * @param pageAddr Page address. + * @param type Type. + */ + public static void setType(long pageAddr, int type) { + PageUtils.putShort(pageAddr, TYPE_OFF, (short) type); + + assert getType(pageAddr) == type : getType(pageAddr); + } + + /** + * Returns a version. + */ + public final int getVersion() { + return ver; + } + + /** + * Returns a page version. + * + * @param buf Buffer. + * @return Version. + */ + public static int getVersion(ByteBuffer buf) { + return buf.getShort(VER_OFF) & 0xFFFF; + } + + /** + * Returns a page version. + * + * @param pageAddr Page address. + * @return Version. + */ + public static int getVersion(long pageAddr) { + return PageUtils.getShort(pageAddr, VER_OFF) & 0xFFFF; + } + + /** + * Sets the version to the page. + * + * @param pageAddr Page address. + * @param ver Version. + */ + protected static void setVersion(long pageAddr, int ver) { + PageUtils.putShort(pageAddr, VER_OFF, (short) ver); + + assert getVersion(pageAddr) == ver; + } + + /** + * Returns a page ID. + * + * @param buf Buffer. + * @return Page ID. + */ + public static long getPageId(ByteBuffer buf) { + return buf.getLong(PAGE_ID_OFF); + } + + /** + * Returns a page ID. + * + * @param pageAddr Page address. + * @return Page ID. + */ + public static long getPageId(long pageAddr) { + return PageUtils.getLong(pageAddr, PAGE_ID_OFF); + } + + /** + * Sets the page ID to the page. + * + * @param pageAddr Page address. + * @param pageId Page ID. + */ + public static void setPageId(long pageAddr, long pageId) { + PageUtils.putLong(pageAddr, PAGE_ID_OFF, pageId); + + assert getPageId(pageAddr) == pageId; + } + + /** + * Returns a rotated ID. + * + * @param pageAddr Page address. + * @return Rotated page ID part. + */ + public static int getRotatedIdPart(long pageAddr) { + return PageUtils.getUnsignedByte(pageAddr, ROTATED_ID_PART_OFF); + } + + /** + * Sets the rotated ID to the page. + * + * @param pageAddr Page address. + * @param rotatedIdPart Rotated page ID part. + */ + public static void setRotatedIdPart(long pageAddr, int rotatedIdPart) { + PageUtils.putUnsignedByte(pageAddr, ROTATED_ID_PART_OFF, rotatedIdPart); + + assert getRotatedIdPart(pageAddr) == rotatedIdPart; + } + + /** + * Sets the compression type to the page. + * + * @param page Page buffer. + * @param compressType Compression type. + */ + public static void setCompressionType(ByteBuffer page, byte compressType) { + page.put(COMPRESSION_TYPE_OFF, compressType); + } + + /** + * Returns a compression type. + * + * @param page Page buffer. + * @return Compression type. + */ + public static byte getCompressionType(ByteBuffer page) { + return page.get(COMPRESSION_TYPE_OFF); + } + + /** + * Returns a compression type. + * + * @param pageAddr Page address. + * @return Compression type. + */ + public static byte getCompressionType(long pageAddr) { + return PageUtils.getByte(pageAddr, COMPRESSION_TYPE_OFF); + } + + /** + * Sets the compressed size to the page. + * + * @param page Page buffer. + * @param compressedSize Compressed size. + */ + public static void setCompressedSize(ByteBuffer page, short compressedSize) { + page.putShort(COMPRESSED_SIZE_OFF, compressedSize); + } + + /** + * Returns a compressed size. + * + * @param page Page buffer. + * @return Compressed size. + */ + public static short getCompressedSize(ByteBuffer page) { + return page.getShort(COMPRESSED_SIZE_OFF); + } + + /** + * Returns a compressed size. + * + * @param pageAddr Page address. + * @return Compressed size. + */ + public static short getCompressedSize(long pageAddr) { + return PageUtils.getShort(pageAddr, COMPRESSED_SIZE_OFF); + } + + /** + * Sets the comacted size to the page. Review comment: ```suggestion * Sets the compacted size to the page. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIoModule.java ########## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.util.Collection; + +/** + * Extension point for modules to provide their own {@link PageIo} implementations by implementing current interface and exporting + * the implementation via corresponding {@code META-INF/services} resource. Review comment: ```suggestion * it via the {@code META-INF/services} resource. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/PageSupport.java ########## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory; + +import org.apache.ignite.internal.pagememory.metric.IoStatisticsHolder; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Class responsible for acquiring/releasing and locking/unlocking pages. + */ +public interface PageSupport { + /** + * Returns an absolute pointer to a page, associated with the given page ID. Each pointer obtained with this method must be released by + * calling {@link #releasePage(int, long, long)}. This method will allocate a page with the given ID if it doesn't exist. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @return Page pointer. + * @throws IgniteInternalCheckedException If failed. + */ + long acquirePage(int groupId, long pageId) throws IgniteInternalCheckedException; + + /** + * Returns an absolute pointer to a page, associated with the given page ID. Each page obtained with this method must be released by + * calling {@link #releasePage(int, long, long)}. This method will allocate a page with the given ID if it doesn't exist. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param statHolder Statistics holder to track IO operations. + * @return Page pointer. + * @throws IgniteInternalCheckedException If failed. + */ + long acquirePage(int groupId, long pageId, IoStatisticsHolder statHolder) throws IgniteInternalCheckedException; + + /** + * Releases pages acquired by any of the {@code acquirePage} methods. + * + * @param groupId Group ID. + * @param pageId Page ID to release. + * @param page Page pointer returned by the corresponding {@code acquirePage} call. + */ + void releasePage(int groupId, long pageId, long page); + + /** + * Acquires a read lock associated with the given page. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + * @return Pointer for reading the page or {@code 0} if page has been reused. + */ + long readLock(int groupId, long pageId, long page); + + /** + * Acquires a read lock, associated with a given page, without checking the page tag. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + * @return Pointer for reading the page. + */ + long readLockForce(int groupId, long pageId, long page); + + /** + * Releases a read lock, associated with a given page. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + */ + void readUnlock(int groupId, long pageId, long page); + + /** + * Acquired a write lock on the page. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + * @return Address of a buffer with contents of the given page or {@code 0L} if attempt to take the write lock failed. + */ + long writeLock(int groupId, long pageId, long page); + + /** + * Tries to acquire a write lock on the page. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + * @return Address of a buffer with contents of the given page or {@code 0L} if attempt to take the write lock failed. + */ + long tryWriteLock(int groupId, long pageId, long page); + + /** + * Releases locked page. + * + * @param groupId Group ID. + * @param pageId Page ID. + * @param page Page pointer. + * @param dirtyFlag Determines whether the page was modified since the last checkpoint. + */ + void writeUnlock(int groupId, long pageId, long page, boolean dirtyFlag); + + /** + * Checks whether the page is dirty or not. Review comment: ```suggestion * Checks whether the page is dirty. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIo.java ########## @@ -0,0 +1,501 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.nio.ByteBuffer; +import org.apache.ignite.internal.pagememory.PageMemory; +import org.apache.ignite.internal.pagememory.metric.IoStatisticsHolder; +import org.apache.ignite.internal.pagememory.util.PageHandler; +import org.apache.ignite.internal.pagememory.util.PageIdUtils; +import org.apache.ignite.internal.pagememory.util.PageLockListener; +import org.apache.ignite.internal.pagememory.util.PageUtils; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Base format for all the page types. + * + * <p>Checklist for page IO implementations and usage (The Rules): + * + * <p>1. IO should not have any `public static` methods. We have versioned IOs and any static method will mean that it have to always work + * in backward compatible way between all the IO versions. The base class {@link PageIo} has static methods (like {@link #getPageId(long)}) + * intentionally: this base format can not be changed between versions. + * + * <p>2. IO must correctly override {@link #initNewPage(long, long, int)} method and call super. We have logic that relies on this behavior. + * + * <p>3. Always keep in mind that IOs are versioned and their format can change from version to version. In this respect it is a good + * practice to avoid exposing details of IO internal format on it's API. The API should be minimalistic and abstract, so that internal + * format in future IO version can be completely changed without any changes to the API of this page IO. + * + * <p>4. Page IO API should not have any version dependent semantics and should not change API semantics in newer versions. + * + * <p>5. It is almost always preferable to read or write (especially write) page contents using static methods on {@link PageHandler}. To + * just initialize new page use {@link PageHandler#initPage(PageMemory, int, long, PageIo, PageLockListener, IoStatisticsHolder)} method + * with needed IO instance. + */ Review comment: ```suggestion /** * Base format for all page types. * * <p>Checklist for {@code PageIo} implementations and usage (The Rules): * * <ol> * <li> * IO should not have any {@code public static} methods. * * IO implementations are versioned and static methods make it difficult to implement correctly in a backward-compatible way. * The base {@link PageIo} class has some static methods (like {@link #getPageId(long)}) intentionally: * this base format can not be changed between versions; * </li> * <li> * IO must correctly override {@link #initNewPage(long, long, int)} method and call the super method; * </li> * <li> * Always keep in mind that IOs are versioned and their format can change from version to version. In this respect it is a good * practice to avoid exposing details of the internal format through the API. The API should be minimalistic and abstract, * so the internal format in future IO versions can be changed without any changes to the API of this page IO; * </li> * <li> * Page IO API should not have any version dependent semantics and should not change API semantics in newer versions; * </li> * <li> * It is almost always preferable to read or write (especially write) page contents using static methods declared in * {@link PageHandler}. To initialize a new page use {@link PageHandler#initPage} method with a corresponding IO instance. * </li> * </ol> */ ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/IoVersions.java ########## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import org.apache.ignite.internal.tostring.S; + +/** + * Registry for IO versions of the same type. + */ +public final class IoVersions<V extends PageIo> { + /** + * Sorted array of IO objects. + */ + private final V[] vers; + + /** + * Page type. + */ + private final int type; + + /** + * Last element of {@link #vers} for faster access. + */ + private final V latest; + + /** + * Constructor. + * + * @param vers Array of IOs. All {@link PageIo#getType()} must match and all {@link PageIo#getVersion()} must continuously increase, + * starting with {@code 1}. + */ + @SafeVarargs + public IoVersions(V... vers) { + assert vers != null; + assert vers.length > 0; + + this.vers = vers; + this.type = vers[0].getType(); + + latest = vers[vers.length - 1]; + + assert checkVersions(); + } + + /** + * Returns type of {@link PageIo}s. + */ + public int getType() { + return type; + } + + /** + * Checks versions array invariants. + * + * @return {@code true} If versions are correct. + */ + private boolean checkVersions() { + for (int i = 0; i < vers.length; i++) { + V v = vers[i]; + + if (v.getType() != type || v.getVersion() != i + 1) { + return false; + } + } + + return true; + } + + /** + * Returns latest IO version. + */ + public V latest() { + return latest; + } + + /** + * Returns IO for the given version. + * + * @param ver Version. + * @return IO. + * @throws IllegalStateException If requested version is not found. + */ + public V forVersion(int ver) { + if (ver == 0) { + throw new IllegalStateException("Failed to get page IO instance (page content is corrupted)"); + } + + return vers[ver - 1]; + } + + /** + * Returns IO for the given version. Review comment: ```suggestion * Returns IO for the given page. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIoModule.java ########## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.util.Collection; + +/** + * Extension point for modules to provide their own {@link PageIo} implementations by implementing current interface and exporting + * the implementation via corresponding {@code META-INF/services} resource. + */ +public interface PageIoModule { + /** + * Returns a collection of {@link IoVersions} instances for all new page IO types in the module. Review comment: ```suggestion * Returns a collection of {@link IoVersions} instances for all page IO types declared in a module. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIoRegistry.java ########## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.nio.ByteBuffer; +import java.util.ServiceLoader; +import org.apache.ignite.lang.IgniteInternalCheckedException; + +/** + * Page IO Registry. This component registers and provides all available {@link PageIo} types. + */ +public class PageIoRegistry { + /** + * Arrays of {@link IoVersions} for fast access. Element 0 is reserved. + */ + private final IoVersions<?>[] ioVersions = new IoVersions[PageIo.MAX_IO_TYPE + 1]; + + /** + * Loads all {@link IoVersions} from corresponding {@link PageIoModule} using a {@link ServiceLoader} mechanism. Review comment: ```suggestion * Loads all {@link IoVersions} from a {@link PageIoModule} using the {@link ServiceLoader} mechanism. ``` ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/util/PageUtils.java ########## @@ -0,0 +1,255 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.util; + +import org.apache.ignite.internal.util.GridUnsafe; + +/** + * Page utils. + */ +public class PageUtils { + /** + * Reads a byte from the memory. + * + * @param addr Start address. + * @param off Offset. + * @return Byte value from given address. + */ + public static byte getByte(long addr, int off) { + assert addr > 0 : addr; + assert off >= 0; + + return GridUnsafe.getByte(addr + off); + } + + /** + * Reads an unsigned byte from the memory. + * + * @param addr Start address. + * @param off Offset. + * @return Byte value from given address. + */ + public static int getUnsignedByte(long addr, int off) { + assert addr > 0 : addr; + assert off >= 0; + + return GridUnsafe.getByte(addr + off) & 0xFF; + } + + /** + * Reads a byte array from the memory. + * + * @param addr Start address. + * @param off Offset. + * @param len Bytes length. + * @return Bytes from given address. + */ + public static byte[] getBytes(long addr, int off, int len) { + assert addr > 0 : addr; + assert off >= 0; + assert len >= 0; + + byte[] bytes = new byte[len]; + + GridUnsafe.copyMemory(null, addr + off, bytes, GridUnsafe.BYTE_ARR_OFF, len); + + return bytes; + } + + /** + * Reads a byte array from the memory. + * + * @param srcAddr Source address. + * @param srcOff Source offset. + * @param dst Destination array. + * @param dstOff Destination offset. + * @param len Length. + */ + public static void getBytes(long srcAddr, int srcOff, byte[] dst, int dstOff, int len) { + assert srcAddr > 0; + assert srcOff > 0; + assert dst != null; + assert dstOff >= 0; + assert len >= 0; + + GridUnsafe.copyMemory(null, srcAddr + srcOff, dst, GridUnsafe.BYTE_ARR_OFF + dstOff, len); + } + + /** + * Reads a short value from the memory. + * + * @param addr Address. + * @param off Offset. + * @return Value. + */ + public static short getShort(long addr, int off) { + assert addr > 0 : addr; + assert off >= 0; + + return GridUnsafe.getShort(addr + off); + } + + /** + * Reads an int value from the memory. + * + * @param addr Address. + * @param off Offset. + * @return Value. + */ + public static int getInt(long addr, int off) { + assert addr > 0 : addr; + assert off >= 0; + + return GridUnsafe.getInt(addr + off); + } + + /** + * Reads a long value from the memory. + * + * @param addr Address. + * @param off Offset. + * @return Value. + */ + public static long getLong(long addr, int off) { + assert addr > 0 : addr; + assert off >= 0; + + return GridUnsafe.getLong(addr + off); + } + + /** + * Writes a byte array into the memory. + * + * @param addr Address. + * @param off Offset. + * @param bytes Bytes. + */ + public static void putBytes(long addr, int off, byte[] bytes) { + assert addr > 0 : addr; + assert off >= 0; + assert bytes != null; + + GridUnsafe.copyMemory(bytes, GridUnsafe.BYTE_ARR_OFF, null, addr + off, bytes.length); + } + + /** + * Writes a byte array into the memory. + * + * @param addr Address. + * @param off Offset. + * @param bytes Bytes array. + * @param bytesOff Bytes array offset. + */ + public static void putBytes(long addr, int off, byte[] bytes, int bytesOff) { + assert addr > 0 : addr; + assert off >= 0; + assert bytes != null; + assert bytesOff >= 0 && (bytesOff < bytes.length || bytes.length == 0) : bytesOff; + + GridUnsafe.copyMemory(bytes, GridUnsafe.BYTE_ARR_OFF + bytesOff, null, addr + off, bytes.length - bytesOff); Review comment: probably that you can use an overloaded `putBytes` version below instead ########## File path: modules/page-memory/src/main/java/org/apache/ignite/internal/pagememory/io/PageIoModule.java ########## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.pagememory.io; + +import java.util.Collection; + +/** + * Extension point for modules to provide their own {@link PageIo} implementations by implementing current interface and exporting Review comment: ```suggestion * Extension point for modules to provide their own {@link PageIo} implementations by implementing this interface and exporting ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
