dcapwell commented on code in PR #3656:
URL: https://github.com/apache/cassandra/pull/3656#discussion_r1829799080
##########
src/java/org/apache/cassandra/service/accord/AccordService.java:
##########
@@ -568,6 +570,46 @@ public static List<ClusterMetadata> tcmLoadRange(long min,
long max)
return afterLoad;
}
+ /**
+ * This method exists due to the fact that we define a retry policy for
TCM to follow, and then TCM ignores it and does no retries...
+ */
+ private static List<ClusterMetadata> reconstruct(long min, long max)
+ {
+ Epoch start = Epoch.create(min);
+ Epoch end = Epoch.create(max);
+ Retry.Deadline retryPolicyThatGetsIgnored =
Retry.Deadline.retryIndefinitely(DatabaseDescriptor.getCmsAwaitTimeout().to(NANOSECONDS),
+
TCMMetrics.instance.fetchLogRetries);
+ Throwable lastError = null;
+ Backoff backoff = new Backoff.ExponentialBackoff(42, 200,
SECONDS.toMillis(1), ThreadLocalRandom.current()::nextDouble);
+ long startNanos = Clock.Global.nanoTime();
+ for (int i = 0; backoff.mayRetry(i); i++)
+ {
+ try
+ {
+ Processor processor =
ClusterMetadataService.instance().processor();
+ // When starting up paxos based processor has shown to be
flakey (this node is still starting up), so attempt to leverage the remote
process to overload this work
+ if (processor instanceof
ClusterMetadataService.SwitchableProcessor)
+ processor = ((ClusterMetadataService.SwitchableProcessor)
processor).remoteProcessor();
Review Comment:
yes, this is 100% a hack, but its needed for now to be stable... Sam and I
are still looking into if this is a real issue or a JVM-Dtest issue... On
startup FD still needs to learn what is alive and it looks very flakey while we
are starting up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
