stefan-egli commented on a change in pull request #4: URL: https://github.com/apache/sling-org-apache-sling-discovery-oak/pull/4#discussion_r708330075
########## File path: src/main/java/org/apache/sling/discovery/oak/cluster/PartialStartupDetector.java ########## @@ -0,0 +1,189 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.sling.discovery.oak.cluster; + +import java.util.Collection; +import java.util.HashSet; +import java.util.Set; + +import org.apache.sling.api.resource.Resource; +import org.apache.sling.api.resource.ResourceResolver; +import org.apache.sling.api.resource.ValueMap; +import org.apache.sling.discovery.commons.providers.util.LogSilencer; +import org.apache.sling.discovery.oak.Config; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Discovery.oak requires that both Oak and Sling are operating normally in + * order to declare victory and announce a new topology. + * <p/> + * The startup phase is especially tricky in this regard, since there are + * multiple elements that need to get updated (some are in the Oak layer, some + * in Sling): + * <ul> + * <li>lease & clusterNodeId : this is maintained by Oak</li> + * <li>idMap : this is maintained by IdMapService</li> + * <li>leaderElectionId : this is maintained by OakViewChecker</li> + * <li>syncToken : this is maintained by SyncTokenService</li> + * </ul> + * A successful join of a cluster instance to the topology requires all 4 + * elements to be set (and maintained, in case of lease and syncToken) + * correctly. + * <p/> + * This PartialStartupDetector is in charge of ensuring that a newly joined + * instance has all these elements set. Otherwise it is considered a "partially + * started instance" (PSI) and suppressed. + * <p/> + * The suppression ensures that existing instances aren't blocked by a rogue, + * partially starting instance. However, there's also a timeout after which the + * suppression is no longer applied - at which point such a rogue instance will + * block existing instances. Infrastructure must ensure that a rogue instance is + * detected and restarted/fixed in a reasonable amount of time. + */ +public class PartialStartupDetector { + + private final Logger logger = LoggerFactory.getLogger(this.getClass()); + + private final ResourceResolver resourceResolver; + private final Config config; + private final int me; + private final long currentSeqNum; + + private final boolean syncTokenEnabled; + + private final boolean suppressingApplicable; + private final Set<Integer> partiallyStartedClusterNodeIds = new HashSet<>(); + + private final LogSilencer logSilencer; + + /** + * @param lowestLocalSeqNum the lowest sequence number which + * the local OakClusterViewService has handled as part of asClusterView + * @param me the clusterNodeId (provided by oak) of the local instance (==me) + * @param localSlingIds slingId previously seen by this cluster instance (those will not be suppressed) + * @param timeoutMillis -1 or 0 disables the timeout, otherwise the suppression + * is only done for the provided maximum number of milliseconds. + * @param logSilencer + */ + PartialStartupDetector(ResourceResolver resourceResolver, Config config, + long lowestLocalSeqNum, int me, String mySlingId, long currentSeqNum, long timeoutMillis, + LogSilencer logSilencer) { + this.resourceResolver = resourceResolver; + this.config = config; + this.me = me; + this.currentSeqNum = currentSeqNum; + + this.syncTokenEnabled = config != null && config.getSyncTokenEnabled(); + + // suppressing is enabled + // * when so configured + // * we haven't hit the timeout yet + // * when the local instance ever showed to peers that it has fully started. + // and one way to verify for that is to demand that it ever wrote a synctoken. + // and to check that we keep note of the first ever successful seq num returned + // here + // and require the current syncToken to be at least that. + final long now = System.currentTimeMillis(); + final long mySyncToken = readSyncToken(resourceResolver, mySlingId); + final boolean suppressionConfigured = config != null && config.getSuppressPartiallyStartedInstances(); + suppressingApplicable = suppressionConfigured + && ((timeoutMillis <= 0) || (now < timeoutMillis)) Review comment: there should be 4 different cases: 1. suppression timeout is disabled (as is by default), in that case `timeoutMillis` is always 0 here, but suppressing should be applicable 2. there was never/no recent case of suppressing, `timeoutMillis` starts at 0 then, and suppressing should be applicable 3. with a suppression timeout configured and once the suppression starts, the `timeoutMillis` has a valid timestamp, in the future. Until then the suppressing is applicable 4. with a suppression timeout configured and a `timeoutMillis` of not 0, but a timestamp in the past the suppression no longer is applicable (the timeout feature). I'll look into coding this a bit nicer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
