Copilot commented on code in PR #10748:
URL: https://github.com/apache/ozone/pull/10748#discussion_r3575529787
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis_snapshot/OmRatisSnapshotProvider.java:
##########
@@ -86,6 +97,103 @@ public class OmRatisSnapshotProvider extends
RDBSnapshotProvider {
private final boolean spnegoEnabled;
private final URLConnectionFactory connectionFactory;
private final boolean useV2CheckpointApi;
+ /** Minimum usable bytes on snapshot volume before download; 0 = disabled. */
+ private final long bootstrapMinSpaceBytes;
+ /** Applied to leader-reported SST byte estimate to reserve tar/unpack
headroom. */
+ private final double bootstrapCheckpointHeadroomRatio;
+
+ private static final class BootstrapSpaceRequirement {
+ private final long requiredBytes;
+ private final boolean usedLeaderEstimateHeader;
+
+ private BootstrapSpaceRequirement(long requiredBytes, boolean
usedLeaderEstimateHeader) {
+ this.requiredBytes = requiredBytes;
+ this.usedLeaderEstimateHeader = usedLeaderEstimateHeader;
+ }
+ }
+
+ /**
+ * Whether this {@link IOException} (or its causes) typically means the
+ * local filesystem ran out of space or hit a quota while writing.
+ */
+ public static boolean isDiskFullOrQuotaIOException(IOException ioe) {
+ for (Throwable t = ioe; t != null; t = t.getCause()) {
+ if (t instanceof DiskOutOfSpaceException) {
+ return true;
+ }
+ if (matchesDiskFullOrQuotaMessage(t)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ /**
+ * Best-effort supplement for JDK {@link FileSystemException} ENOSPC and
+ * quota wording on Linux OM deployments (typically English from libc/JVM).
+ * {@link DiskOutOfSpaceException} in the cause chain is handled by type
+ * in {@link #isDiskFullOrQuotaIOException(IOException)} and does not depend
+ * on message text. Localized OS messages without matching substrings are not
+ * detected here.
+ */
+ private static boolean matchesDiskFullOrQuotaMessage(Throwable throwable) {
+ if (throwable instanceof FileSystemException) {
+ String reason = ((FileSystemException) throwable).getReason();
+ if (reason != null && containsDiskFullOrQuotaText(reason)) {
+ return true;
+ }
+ }
+ String msg = throwable.getMessage();
+ return msg != null && containsDiskFullOrQuotaText(msg);
+ }
+
+ private static boolean containsDiskFullOrQuotaText(String text) {
+ String m = text.toLowerCase(Locale.ROOT);
+ return m.contains("no space left on device")
+ || m.contains("no space")
+ || m.contains("space left")
+ || m.contains("enospc")
+ || m.contains("disk quota exceeded")
+ || m.contains("quota exceeded")
+ || m.contains("quota");
+ }
Review Comment:
containsDiskFullOrQuotaText() matches very generic substrings (eg "no
space", "space left", and plain "quota"), which can misclassify unrelated
IOExceptions as disk-full/quota and lead to misleading ERROR logs during
bootstrap. Narrow the match set to specific ENOSPC/quota phrases (and rely on
DiskOutOfSpaceException in the cause chain for type-based detection).
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java:
##########
@@ -291,6 +291,18 @@ public final class OMConfigKeys {
OZONE_OM_SNAPSHOT_PROVIDER_REQUEST_TIMEOUT_DEFAULT =
TimeDuration.valueOf(300000, TimeUnit.MILLISECONDS);
+ public static final String OZONE_OM_BOOTSTRAP_MIN_SPACE_KEY =
+ "ozone.om.bootstrap.min.space";
+ public static final String OZONE_OM_BOOTSTRAP_MIN_SPACE_DEFAULT = "5GB";
+
Review Comment:
This PR is titled/described as a flaky follower-read test fix, but it also
introduces new OM bootstrap disk-space configuration keys and a new checkpoint
response header used in snapshot download logic. Please update the PR
description/title to reflect this broader scope, or split the disk-space/header
change into a separate PR to keep review/rollback boundaries clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
