sergey-chugunov-1985 commented on a change in pull request #9198:
URL: https://github.com/apache/ignite/pull/9198#discussion_r664527318
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/aware/SegmentArchiveSizeStorage.java
##########
@@ -17,65 +17,132 @@
package org.apache.ignite.internal.processors.cache.persistence.wal.aware;
+import java.util.Map;
+import java.util.TreeMap;
+import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
+import org.jetbrains.annotations.Nullable;
+
+import static
org.apache.ignite.configuration.DataStorageConfiguration.UNLIMITED_WAL_ARCHIVE;
/**
* Storage WAL archive size.
* Allows to track the exceeding of the maximum archive size.
*/
class SegmentArchiveSizeStorage {
- /** Current WAL archive size in bytes. */
+ /** Current WAL archive size in bytes. Guarded by {@code this}. */
private long curr;
- /** Reserved WAL archive size in bytes. */
- private long reserved;
+ /** Flag of interrupt waiting on this object. Guarded by {@code this}. */
+ private boolean interrupted;
+
+ /** Minimum size of the WAL archive in bytes. */
+ private final long minWalArchiveSize;
+
+ /** Maximum size of the WAL archive in bytes. */
+ private final long maxWalArchiveSize;
+
+ /**
+ * Segment sizes. Mapping: segment idx -> size in bytes. Guarded by {@code
this}.
+ * {@code null} if {@link #maxWalArchiveSize} == {@link
DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ */
+ @Nullable private final Map<Long, Long> segmentSize;
- /** Flag of interrupt waiting on this object. */
- private volatile boolean interrupted;
+ /**
+ * Segment reservations storage.
+ * {@code null} if {@link #maxWalArchiveSize} == {@link
DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ */
+ @Nullable private final SegmentReservationStorage reservationStorage;
/**
- * Adding current WAL archive size in bytes.
+ * Constructor.
*
- * @param size Size in bytes.
+ * @param minWalArchiveSize Minimum size of the WAL archive in bytes.
+ * @param maxWalArchiveSize Maximum size of the WAL archive in bytes
+ * or {@link DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ * @param reservationStorage Segment reservations storage.
*/
- synchronized void addCurrentSize(long size) {
- curr += size;
+ public SegmentArchiveSizeStorage(
+ long minWalArchiveSize,
+ long maxWalArchiveSize,
+ SegmentReservationStorage reservationStorage
+ ) {
+ this.minWalArchiveSize = minWalArchiveSize;
+ this.maxWalArchiveSize = maxWalArchiveSize;
- if (size > 0)
- notifyAll();
+ if (maxWalArchiveSize != UNLIMITED_WAL_ARCHIVE) {
+ segmentSize = new TreeMap<>();
+ this.reservationStorage = reservationStorage;
+ }
+ else {
+ segmentSize = null;
+ this.reservationStorage = null;
+ }
}
/**
- * Adding reserved WAL archive size in bytes.
- * Defines a hint to determine if the maximum size is exceeded before a
new segment is archived.
+ * Adding the WAL segment size in the archive.
*
- * @param size Size in bytes.
+ * @param idx Absolut segment index.
+ * @param curr Current WAL archive size in bytes.
Review comment:
This javadoc is wrong and name of parameter is bad. Semantics of the
parameter is about change of WAL archive size. Lets call it that way:
`sizeChange`.
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/aware/SegmentArchiveSizeStorage.java
##########
@@ -17,65 +17,132 @@
package org.apache.ignite.internal.processors.cache.persistence.wal.aware;
+import java.util.Map;
+import java.util.TreeMap;
+import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
+import org.jetbrains.annotations.Nullable;
+
+import static
org.apache.ignite.configuration.DataStorageConfiguration.UNLIMITED_WAL_ARCHIVE;
/**
* Storage WAL archive size.
* Allows to track the exceeding of the maximum archive size.
*/
class SegmentArchiveSizeStorage {
- /** Current WAL archive size in bytes. */
+ /** Current WAL archive size in bytes. Guarded by {@code this}. */
private long curr;
Review comment:
Name of the field doesn't relate to its semantics. I suggest to rename
it to something like `archiveSize` or `walArchiveSize`.
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/aware/SegmentAware.java
##########
@@ -318,22 +329,13 @@ public boolean minLockIndex(long absIdx) {
}
/**
- * Adding current WAL archive size in bytes.
- *
- * @param size Size in bytes.
- */
- public void addCurrentWalArchiveSize(long size) {
- archiveSizeStorage.addCurrentSize(size);
- }
-
- /**
- * Adding reserved WAL archive size in bytes.
- * Defines a hint to determine if the maximum size is exceeded before a
new segment is archived.
+ * Adding the WAL segment size in the archive.
*
- * @param size Size in bytes.
+ * @param idx Absolut segment index.
+ * @param curr Current WAL archive size in bytes.
*/
- public void addReservedWalArchiveSize(long size) {
- archiveSizeStorage.addReservedSize(size);
+ public void addSize(long idx, long curr) {
Review comment:
The same about semantics of the second parameter is here.
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/aware/SegmentArchiveSizeStorage.java
##########
@@ -17,65 +17,132 @@
package org.apache.ignite.internal.processors.cache.persistence.wal.aware;
+import java.util.Map;
+import java.util.TreeMap;
+import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
+import org.jetbrains.annotations.Nullable;
+
+import static
org.apache.ignite.configuration.DataStorageConfiguration.UNLIMITED_WAL_ARCHIVE;
/**
* Storage WAL archive size.
* Allows to track the exceeding of the maximum archive size.
*/
class SegmentArchiveSizeStorage {
- /** Current WAL archive size in bytes. */
+ /** Current WAL archive size in bytes. Guarded by {@code this}. */
private long curr;
- /** Reserved WAL archive size in bytes. */
- private long reserved;
+ /** Flag of interrupt waiting on this object. Guarded by {@code this}. */
+ private boolean interrupted;
+
+ /** Minimum size of the WAL archive in bytes. */
+ private final long minWalArchiveSize;
+
+ /** Maximum size of the WAL archive in bytes. */
+ private final long maxWalArchiveSize;
+
+ /**
+ * Segment sizes. Mapping: segment idx -> size in bytes. Guarded by {@code
this}.
+ * {@code null} if {@link #maxWalArchiveSize} == {@link
DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ */
+ @Nullable private final Map<Long, Long> segmentSize;
- /** Flag of interrupt waiting on this object. */
- private volatile boolean interrupted;
+ /**
+ * Segment reservations storage.
+ * {@code null} if {@link #maxWalArchiveSize} == {@link
DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ */
+ @Nullable private final SegmentReservationStorage reservationStorage;
/**
- * Adding current WAL archive size in bytes.
+ * Constructor.
*
- * @param size Size in bytes.
+ * @param minWalArchiveSize Minimum size of the WAL archive in bytes.
+ * @param maxWalArchiveSize Maximum size of the WAL archive in bytes
+ * or {@link DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ * @param reservationStorage Segment reservations storage.
*/
- synchronized void addCurrentSize(long size) {
- curr += size;
+ public SegmentArchiveSizeStorage(
+ long minWalArchiveSize,
+ long maxWalArchiveSize,
+ SegmentReservationStorage reservationStorage
+ ) {
+ this.minWalArchiveSize = minWalArchiveSize;
+ this.maxWalArchiveSize = maxWalArchiveSize;
- if (size > 0)
- notifyAll();
+ if (maxWalArchiveSize != UNLIMITED_WAL_ARCHIVE) {
+ segmentSize = new TreeMap<>();
+ this.reservationStorage = reservationStorage;
+ }
+ else {
+ segmentSize = null;
+ this.reservationStorage = null;
+ }
}
/**
- * Adding reserved WAL archive size in bytes.
- * Defines a hint to determine if the maximum size is exceeded before a
new segment is archived.
+ * Adding the WAL segment size in the archive.
*
- * @param size Size in bytes.
+ * @param idx Absolut segment index.
+ * @param curr Current WAL archive size in bytes.
*/
- synchronized void addReservedSize(long size) {
- reserved += size;
+ void addSize(long idx, long curr) {
+ long releaseIdx = -1;
+
+ synchronized (this) {
+ this.curr += curr;
+
+ if (segmentSize != null) {
+ segmentSize.compute(idx, (i, size) -> {
+ long res = (size == null ? 0 : size) + curr;
+
+ return res == 0 ? null : res;
+ });
+ }
+
+ if (curr > 0) {
+ if (segmentSize != null && this.curr >= maxWalArchiveSize) {
+ long size = 0;
+
+ for (Map.Entry<Long, Long> e : segmentSize.entrySet()) {
+ releaseIdx = e.getKey();
+
+ if (this.curr - (size += e.getValue()) <
minWalArchiveSize)
+ break;
+ }
+ }
- if (size > 0)
- notifyAll();
+ notifyAll();
+ }
+ }
+
+ if (releaseIdx != -1) {
+ assert reservationStorage != null;
Review comment:
Assert doesn't provide information on which `releaseIdx` it has failed.
Please add a message to the assert clause.
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/aware/SegmentArchiveSizeStorage.java
##########
@@ -17,65 +17,132 @@
package org.apache.ignite.internal.processors.cache.persistence.wal.aware;
+import java.util.Map;
+import java.util.TreeMap;
+import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
+import org.jetbrains.annotations.Nullable;
+
+import static
org.apache.ignite.configuration.DataStorageConfiguration.UNLIMITED_WAL_ARCHIVE;
/**
* Storage WAL archive size.
* Allows to track the exceeding of the maximum archive size.
*/
class SegmentArchiveSizeStorage {
- /** Current WAL archive size in bytes. */
+ /** Current WAL archive size in bytes. Guarded by {@code this}. */
private long curr;
- /** Reserved WAL archive size in bytes. */
- private long reserved;
+ /** Flag of interrupt waiting on this object. Guarded by {@code this}. */
+ private boolean interrupted;
+
+ /** Minimum size of the WAL archive in bytes. */
+ private final long minWalArchiveSize;
+
+ /** Maximum size of the WAL archive in bytes. */
+ private final long maxWalArchiveSize;
+
+ /**
+ * Segment sizes. Mapping: segment idx -> size in bytes. Guarded by {@code
this}.
+ * {@code null} if {@link #maxWalArchiveSize} == {@link
DataStorageConfiguration#UNLIMITED_WAL_ARCHIVE}.
+ */
+ @Nullable private final Map<Long, Long> segmentSize;
Review comment:
Field name is a bit misleading: it is a size of a segment, but sizes of
all segments in archive dir.
How about renaming it to `segmentsSizes`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
