sashapolo commented on code in PR #2228:
URL: https://github.com/apache/ignite-3/pull/2228#discussion_r1238194079
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -130,24 +132,32 @@ public class CatalogServiceImpl extends
Producer<CatalogEvent, CatalogEventParam
private final PendingComparableValuesTracker<Integer, Void> versionTracker
= new PendingComparableValuesTracker<>(0);
- private final HybridClock clock;
+ private final ClockWaiter clockWaiter;
- private final long delayDurationMs;
+ private final Lazy<Long> delayDurationMs;
/**
* Constructor.
*/
- public CatalogServiceImpl(UpdateLog updateLog, HybridClock clock) {
- this(updateLog, clock, DEFAULT_DELAY_DURATION);
+ public CatalogServiceImpl(UpdateLog updateLog, ClockWaiter clockWaiter) {
+ this(updateLog, clockWaiter, DEFAULT_DELAY_DURATION);
}
/**
* Constructor.
*/
- public CatalogServiceImpl(UpdateLog updateLog, HybridClock clock, long
delayDurationMs) {
+ CatalogServiceImpl(UpdateLog updateLog, ClockWaiter clockWaiter, long
delayDurationMs) {
+ this(updateLog, clockWaiter, () -> delayDurationMs);
+ }
+
+ /**
+ * Constructor.
+ */
+ public CatalogServiceImpl(UpdateLog updateLog, ClockWaiter clockWaiter,
LongSupplier delayDurationMsSupplier) {
Review Comment:
Actually, using `Supplier<Long>` makes more sense here, less boxing/unboxing
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -773,11 +783,21 @@ private void registerCatalog(Catalog newCatalog) {
catalogByTs.put(newCatalog.time(), newCatalog);
}
- private CompletableFuture<Void> saveUpdate(UpdateProducer updateProducer) {
- return saveUpdate(updateProducer, 0);
+ private CompletableFuture<Void>
saveUpdateAndWaitForActivation(UpdateProducer updateProducer) {
+ return saveUpdate(updateProducer, 0)
+ .thenCompose(newVersion -> versionTracker.waitFor(newVersion)
+ .thenCompose(unused -> {
+ Catalog catalog = catalogByVer.get(newVersion);
+ HybridTimestamp activationTs =
HybridTimestamp.hybridTimestamp(catalog.time());
+ HybridTimestamp clusterWideEnsuredActivationTs =
activationTs.addPhysicalTime(
+ HybridTimestamp.maxClockSkew()
Review Comment:
Why do we use only the Clock Skew here? IEP also talks about propagation
times...
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/VersionedUpdate.java:
##########
@@ -56,9 +56,9 @@ public int version() {
return version;
}
- /** Returns activation timestamp. */
- public long activationTimestamp() {
- return activationTimestamp;
+ /** Returns Delay Duration for this update (in milliseconds). */
+ public long delayDuration() {
Review Comment:
Can we keep the `Ms` suffix as a part of this method's name?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java:
##########
@@ -209,13 +214,14 @@ public CompletableFuture<Void> onUpdate(WatchEvent event)
{
VersionedUpdate update = fromBytes(payload);
- onUpdateHandler.handle(update);
+ HybridTimestamp entryTimestamp =
metastoreRevisionToTs.apply(eventEntry.newEntry().revision());
Review Comment:
Can't we include the update timestamp as a part of the `WatchEvent`? This
looks like an unnecessary lookup...
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -773,11 +783,21 @@ private void registerCatalog(Catalog newCatalog) {
catalogByTs.put(newCatalog.time(), newCatalog);
}
- private CompletableFuture<Void> saveUpdate(UpdateProducer updateProducer) {
- return saveUpdate(updateProducer, 0);
+ private CompletableFuture<Void>
saveUpdateAndWaitForActivation(UpdateProducer updateProducer) {
+ return saveUpdate(updateProducer, 0)
+ .thenCompose(newVersion -> versionTracker.waitFor(newVersion)
Review Comment:
Why do we need to call `versionTracker.waitFor` if we do the same thing in
`saveUpdate`?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java:
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.catalog;
+
+import static java.util.concurrent.CompletableFuture.failedFuture;
+
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.hlc.ClockUpdateListener;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.logger.IgniteLogger;
+import org.apache.ignite.internal.logger.Loggers;
+import org.apache.ignite.internal.manager.IgniteComponent;
+import org.apache.ignite.internal.thread.NamedThreadFactory;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.PendingComparableValuesTracker;
+import org.apache.ignite.internal.util.TrackerClosedException;
+import org.apache.ignite.lang.NodeStoppingException;
+
+/**
+ * Allows to wait for the supplied clock to reach a required timesdtamp. It
only uses the clock itself,
+ * no SafeTime mechanisms are involved.
+ */
+public class ClockWaiter implements IgniteComponent {
+ private static final IgniteLogger LOG =
Loggers.forClass(ClockWaiter.class);
+
+ private final String nodeName;
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean stopGuard = new AtomicBoolean(false);
+
+ private final PendingComparableValuesTracker<HybridTimestamp, Void>
nowTracker = new PendingComparableValuesTracker<>(
+ new HybridTimestamp(1, 0)
+ );
+
+ private final ClockUpdateListener updateListener = this::onUpdate;
+
+ private volatile ScheduledExecutorService scheduler;
+
+ public ClockWaiter(String nodeName, HybridClock clock) {
+ this.nodeName = nodeName;
+ this.clock = clock;
+ }
+
+ @Override
+ public void start() {
+ clock.addUpdateListener(updateListener);
+
+ scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter", LOG));
+ }
+
+ @Override
+ public void stop() throws Exception {
+ if (!stopGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+
+ clock.removeUpdateListener(updateListener);
+
+ nowTracker.close();
+
+ scheduler.shutdownNow();
+ scheduler.awaitTermination(10, TimeUnit.SECONDS);
+ }
+
+ private void onUpdate(HybridTimestamp newTs) {
+ nowTracker.update(newTs, null);
+ }
+
+ /**
+ * Wait for the clock to reach the given timestamp.
+ *
+ * @param targetTimestamp Timestamp to wait for.
+ * @return A future that completes when the timestamp is reached by the
clock's time.
+ */
+ public CompletableFuture<Void> waitFor(HybridTimestamp targetTimestamp) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new NodeStoppingException());
+ }
+
+ try {
+ return doWaitFor(targetTimestamp);
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ private CompletableFuture<Void> doWaitFor(HybridTimestamp targetTimestamp)
{
+ CompletableFuture<Void> future = nowTracker.waitFor(targetTimestamp);
+
+ AtomicReference<ScheduledFuture<?>> scheduledFutureRef = new
AtomicReference<>();
Review Comment:
Why do you need an atomic reference here?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/VersionedUpdate.java:
##########
@@ -40,12 +40,12 @@ public class VersionedUpdate implements Serializable {
* Constructs the object.
*
* @param version A version the changes relate to.
- * @param activationTimestamp Timestamp given changes become active at.
+ * @param delayDurationMs Delay duration that, when added to the entry
save timestamp, will produce activation timestamp (milliseconds).
Review Comment:
```suggestion
* @param delayDurationMs Delay duration that, when added to the entry
save timestamp, will produce the activation timestamp (milliseconds).
```
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/VersionedUpdate.java:
##########
@@ -40,12 +40,12 @@ public class VersionedUpdate implements Serializable {
* Constructs the object.
*
* @param version A version the changes relate to.
- * @param activationTimestamp Timestamp given changes become active at.
+ * @param delayDurationMs Delay duration that, when added to the entry
save timestamp, will produce activation timestamp (milliseconds).
Review Comment:
`entry save timestamp` doesn't sound right, can you please rephrase it?
##########
modules/core/src/main/java/org/apache/ignite/internal/hlc/HybridClockImpl.java:
##########
@@ -67,11 +71,18 @@ public long nowLong() {
long newLatestTime = max(oldLatestTime + 1, now);
if (LATEST_TIME.compareAndSet(this, oldLatestTime, newLatestTime))
{
+ HybridTimestamp newTs = hybridTimestamp(newLatestTime);
Review Comment:
Should we optimize the `now` method? We will be creating excessive
`HybridTimestamp` objects otherwise
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -773,11 +783,21 @@ private void registerCatalog(Catalog newCatalog) {
catalogByTs.put(newCatalog.time(), newCatalog);
}
- private CompletableFuture<Void> saveUpdate(UpdateProducer updateProducer) {
- return saveUpdate(updateProducer, 0);
+ private CompletableFuture<Void>
saveUpdateAndWaitForActivation(UpdateProducer updateProducer) {
+ return saveUpdate(updateProducer, 0)
+ .thenCompose(newVersion -> versionTracker.waitFor(newVersion)
+ .thenCompose(unused -> {
+ Catalog catalog = catalogByVer.get(newVersion);
+ HybridTimestamp activationTs =
HybridTimestamp.hybridTimestamp(catalog.time());
+ HybridTimestamp clusterWideEnsuredActivationTs =
activationTs.addPhysicalTime(
+ HybridTimestamp.maxClockSkew()
+ );
+
+ return
clockWaiter.waitFor(clusterWideEnsuredActivationTs);
+ }));
}
- private CompletableFuture<Void> saveUpdate(UpdateProducer updateProducer,
int attemptNo) {
+ private CompletableFuture<Integer> saveUpdate(UpdateProducer
updateProducer, int attemptNo) {
Review Comment:
I would suggest to add a javadoc to this method
##########
modules/core/src/testFixtures/java/org/apache/ignite/internal/TestHybridClock.java:
##########
@@ -38,6 +41,8 @@ public class TestHybridClock implements HybridClock {
/** Latest time. */
private volatile long latestTime;
+ private final List<ClockUpdateListener> updateListeners = new
CopyOnWriteArrayList<>();
Review Comment:
Is it ok that these listeners do not get notified about updates?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -130,24 +132,32 @@ public class CatalogServiceImpl extends
Producer<CatalogEvent, CatalogEventParam
private final PendingComparableValuesTracker<Integer, Void> versionTracker
= new PendingComparableValuesTracker<>(0);
- private final HybridClock clock;
+ private final ClockWaiter clockWaiter;
- private final long delayDurationMs;
+ private final Lazy<Long> delayDurationMs;
Review Comment:
Why do we use `Lazy` and not simply cache the supplier? Shouldn't we be able
to change the configuration on-the-fly?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -812,9 +830,9 @@ private CompletableFuture<Void> saveUpdate(UpdateProducer
updateProducer, int at
class OnUpdateHandlerImpl implements OnUpdateHandler {
@Override
- public void handle(VersionedUpdate update) {
+ public void handle(VersionedUpdate update, HybridTimestamp
metastoreTimestamp) {
Review Comment:
`metastoreTimestamp` is a bit misleading name, maybe
`metastoreUpdateTimestamp` or `updateTimestamp` is better?
##########
modules/core/src/testFixtures/java/org/apache/ignite/internal/testframework/matchers/CompletableFutureMatcher.java:
##########
@@ -145,7 +146,7 @@ public static <T> CompletableFutureMatcher<T>
willBe(Matcher<T> matcher) {
* @param <T> value type
* @return matcher
*/
- public static <T> CompletableFutureMatcher<T> willBe(T value) {
+ public static <T> CompletableFutureMatcher<T> willBe(@Nullable T value) {
Review Comment:
Why is this change needed? I would expect nulls to be matched as
`willBe(nullValue())`
##########
modules/core/src/main/java/org/apache/ignite/internal/hlc/HybridTimestamp.java:
##########
@@ -243,4 +243,11 @@ public HybridTimestamp addPhysicalTime(long mills) {
return new HybridTimestamp(time + (mills << LOGICAL_TIME_BITS_SIZE));
}
+
+ /**
+ * Returns max clock skew for the cluster (in millis).
+ */
+ public static long maxClockSkew() {
+ return CLOCK_SKEW;
Review Comment:
Why isn't this a cluster-wide property?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java:
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.catalog;
+
+import static java.util.concurrent.CompletableFuture.failedFuture;
+
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.hlc.ClockUpdateListener;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.logger.IgniteLogger;
+import org.apache.ignite.internal.logger.Loggers;
+import org.apache.ignite.internal.manager.IgniteComponent;
+import org.apache.ignite.internal.thread.NamedThreadFactory;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.PendingComparableValuesTracker;
+import org.apache.ignite.internal.util.TrackerClosedException;
+import org.apache.ignite.lang.NodeStoppingException;
+
+/**
+ * Allows to wait for the supplied clock to reach a required timesdtamp. It
only uses the clock itself,
+ * no SafeTime mechanisms are involved.
+ */
+public class ClockWaiter implements IgniteComponent {
+ private static final IgniteLogger LOG =
Loggers.forClass(ClockWaiter.class);
+
+ private final String nodeName;
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean stopGuard = new AtomicBoolean(false);
+
+ private final PendingComparableValuesTracker<HybridTimestamp, Void>
nowTracker = new PendingComparableValuesTracker<>(
+ new HybridTimestamp(1, 0)
+ );
+
+ private final ClockUpdateListener updateListener = this::onUpdate;
+
+ private volatile ScheduledExecutorService scheduler;
+
+ public ClockWaiter(String nodeName, HybridClock clock) {
+ this.nodeName = nodeName;
+ this.clock = clock;
+ }
+
+ @Override
+ public void start() {
+ clock.addUpdateListener(updateListener);
+
+ scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter", LOG));
+ }
+
+ @Override
+ public void stop() throws Exception {
+ if (!stopGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+
+ clock.removeUpdateListener(updateListener);
+
+ nowTracker.close();
+
+ scheduler.shutdownNow();
+ scheduler.awaitTermination(10, TimeUnit.SECONDS);
+ }
+
+ private void onUpdate(HybridTimestamp newTs) {
+ nowTracker.update(newTs, null);
+ }
+
+ /**
+ * Wait for the clock to reach the given timestamp.
+ *
+ * @param targetTimestamp Timestamp to wait for.
+ * @return A future that completes when the timestamp is reached by the
clock's time.
+ */
+ public CompletableFuture<Void> waitFor(HybridTimestamp targetTimestamp) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new NodeStoppingException());
+ }
+
+ try {
+ return doWaitFor(targetTimestamp);
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ private CompletableFuture<Void> doWaitFor(HybridTimestamp targetTimestamp)
{
+ CompletableFuture<Void> future = nowTracker.waitFor(targetTimestamp);
+
+ AtomicReference<ScheduledFuture<?>> scheduledFutureRef = new
AtomicReference<>();
+
+ if (!future.isDone()) {
+ // This triggers a clock update.
+ HybridTimestamp now = clock.now();
+
+ if (targetTimestamp.compareTo(now) <= 0) {
+ assert future.isDone();
+ } else {
+ // Adding 1 to account for a possible non-null logical part of
the targetTimestamp.
+ long millisToWait = targetTimestamp.getPhysical() -
now.getPhysical() + 1;
+
+ ScheduledFuture<?> scheduledFuture =
scheduler.schedule(this::triggerClockUpdate, millisToWait,
TimeUnit.MILLISECONDS);
+ scheduledFutureRef.set(scheduledFuture);
+ }
+ }
+
+ return future.handle((res, ex) -> {
+ ScheduledFuture<?> scheduledFuture = scheduledFutureRef.get();
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(true);
+ }
+
+ if (ex != null) {
Review Comment:
Why do you need this exception handling?
##########
modules/catalog/src/test/java/org/apache/ignite/internal/catalog/CatalogServiceSelfTest.java:
##########
@@ -1571,6 +1585,55 @@ public void testColumnEvents() {
verifyNoMoreInteractions(eventListener);
}
+
+ @Test
+ public void userFutureCompletesAfterClusterWideActivationHappens() throws
Exception {
+ final long delayDuration = TimeUnit.DAYS.toMillis(365);
+
+ HybridTimestamp startTs = clock.now();
+
+ InMemoryVaultService vaultService = new InMemoryVaultService();
Review Comment:
This code looks weird, why do we have set up code in the test?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java:
##########
@@ -165,9 +173,13 @@ private void restoreStateFromVault(OnUpdateHandler
handler) {
break;
}
+ if (appliedRevTimestamp == null) {
+ appliedRevTimestamp =
metastoreRevisionToTs.apply(metastore.appliedRevision());
Review Comment:
Can we add an `appliedRevisionTimestamp` method to the Meta Storage
interface? Looks useful
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java:
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.catalog;
+
+import static java.util.concurrent.CompletableFuture.failedFuture;
+
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.hlc.ClockUpdateListener;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.logger.IgniteLogger;
+import org.apache.ignite.internal.logger.Loggers;
+import org.apache.ignite.internal.manager.IgniteComponent;
+import org.apache.ignite.internal.thread.NamedThreadFactory;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.PendingComparableValuesTracker;
+import org.apache.ignite.internal.util.TrackerClosedException;
+import org.apache.ignite.lang.NodeStoppingException;
+
+/**
+ * Allows to wait for the supplied clock to reach a required timesdtamp. It
only uses the clock itself,
+ * no SafeTime mechanisms are involved.
+ */
+public class ClockWaiter implements IgniteComponent {
+ private static final IgniteLogger LOG =
Loggers.forClass(ClockWaiter.class);
+
+ private final String nodeName;
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean stopGuard = new AtomicBoolean(false);
+
+ private final PendingComparableValuesTracker<HybridTimestamp, Void>
nowTracker = new PendingComparableValuesTracker<>(
+ new HybridTimestamp(1, 0)
+ );
+
+ private final ClockUpdateListener updateListener = this::onUpdate;
+
+ private volatile ScheduledExecutorService scheduler;
+
+ public ClockWaiter(String nodeName, HybridClock clock) {
+ this.nodeName = nodeName;
+ this.clock = clock;
+ }
+
+ @Override
+ public void start() {
+ clock.addUpdateListener(updateListener);
+
+ scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter", LOG));
+ }
+
+ @Override
+ public void stop() throws Exception {
+ if (!stopGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+
+ clock.removeUpdateListener(updateListener);
+
+ nowTracker.close();
+
+ scheduler.shutdownNow();
Review Comment:
Can we use the method from `IgniteUtils`?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java:
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.catalog;
+
+import static java.util.concurrent.CompletableFuture.failedFuture;
+
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.hlc.ClockUpdateListener;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.logger.IgniteLogger;
+import org.apache.ignite.internal.logger.Loggers;
+import org.apache.ignite.internal.manager.IgniteComponent;
+import org.apache.ignite.internal.thread.NamedThreadFactory;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.PendingComparableValuesTracker;
+import org.apache.ignite.internal.util.TrackerClosedException;
+import org.apache.ignite.lang.NodeStoppingException;
+
+/**
+ * Allows to wait for the supplied clock to reach a required timesdtamp. It
only uses the clock itself,
+ * no SafeTime mechanisms are involved.
+ */
+public class ClockWaiter implements IgniteComponent {
+ private static final IgniteLogger LOG =
Loggers.forClass(ClockWaiter.class);
+
+ private final String nodeName;
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean stopGuard = new AtomicBoolean(false);
+
+ private final PendingComparableValuesTracker<HybridTimestamp, Void>
nowTracker = new PendingComparableValuesTracker<>(
+ new HybridTimestamp(1, 0)
+ );
+
+ private final ClockUpdateListener updateListener = this::onUpdate;
+
+ private volatile ScheduledExecutorService scheduler;
+
+ public ClockWaiter(String nodeName, HybridClock clock) {
+ this.nodeName = nodeName;
+ this.clock = clock;
+ }
+
+ @Override
+ public void start() {
+ clock.addUpdateListener(updateListener);
+
+ scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter", LOG));
+ }
+
+ @Override
+ public void stop() throws Exception {
+ if (!stopGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+
+ clock.removeUpdateListener(updateListener);
+
+ nowTracker.close();
+
+ scheduler.shutdownNow();
+ scheduler.awaitTermination(10, TimeUnit.SECONDS);
+ }
+
+ private void onUpdate(HybridTimestamp newTs) {
+ nowTracker.update(newTs, null);
Review Comment:
Should it be wrapped in the busy lock?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java:
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.catalog;
+
+import static java.util.concurrent.CompletableFuture.failedFuture;
+
+import java.util.concurrent.CancellationException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.hlc.ClockUpdateListener;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.logger.IgniteLogger;
+import org.apache.ignite.internal.logger.Loggers;
+import org.apache.ignite.internal.manager.IgniteComponent;
+import org.apache.ignite.internal.thread.NamedThreadFactory;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.PendingComparableValuesTracker;
+import org.apache.ignite.internal.util.TrackerClosedException;
+import org.apache.ignite.lang.NodeStoppingException;
+
+/**
+ * Allows to wait for the supplied clock to reach a required timesdtamp. It
only uses the clock itself,
+ * no SafeTime mechanisms are involved.
+ */
+public class ClockWaiter implements IgniteComponent {
+ private static final IgniteLogger LOG =
Loggers.forClass(ClockWaiter.class);
+
+ private final String nodeName;
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean stopGuard = new AtomicBoolean(false);
+
+ private final PendingComparableValuesTracker<HybridTimestamp, Void>
nowTracker = new PendingComparableValuesTracker<>(
+ new HybridTimestamp(1, 0)
Review Comment:
Why not `HybridTimestamp.MIN_VALUE`?
##########
modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java:
##########
@@ -773,11 +783,21 @@ private void registerCatalog(Catalog newCatalog) {
catalogByTs.put(newCatalog.time(), newCatalog);
}
- private CompletableFuture<Void> saveUpdate(UpdateProducer updateProducer) {
- return saveUpdate(updateProducer, 0);
+ private CompletableFuture<Void>
saveUpdateAndWaitForActivation(UpdateProducer updateProducer) {
+ return saveUpdate(updateProducer, 0)
+ .thenCompose(newVersion -> versionTracker.waitFor(newVersion)
Review Comment:
If this part is still needed, I would propose to reorganize the code a
little bit:
```
return saveUpdate(updateProducer, 0)
.thenCompose(newVersion ->
versionTracker.waitFor(newVersion).thenApply(v -> catalogByVer.get(newVersion)))
.thenCompose(catalog -> {
HybridTimestamp activationTs =
HybridTimestamp.hybridTimestamp(catalog.time());
HybridTimestamp clusterWideEnsuredActivationTs =
activationTs.addPhysicalTime(HybridTimestamp.maxClockSkew());
return clockWaiter.waitFor(clusterWideEnsuredActivationTs);
});
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
