[GitHub] [hadoop] hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load
Snapshot info when OM Ratis server starts.
URL: https://github.com/apache/hadoop/pull/1130#discussion_r315976786
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OMRatisSnapshotInfo.java
##
@@ -0,0 +1,133 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ratis;
+
+import org.apache.ratis.server.protocol.TermIndex;
+import org.apache.ratis.server.storage.FileInfo;
+import org.apache.ratis.statemachine.SnapshotInfo;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.OzoneConsts.OM_RATIS_SNAPSHOT_INDEX;
+
+/**
+ * This class captures the {@link OzoneManager#snapshotIndex} and term of the
+ * snapshot.
+ * Ratis server loads the snapshotInfo during startup and updates the
+ * lastApplied index to this snapshotIndex. OM SnaphsotInfo does not contain
+ * any files. It is used only to store/ update the last applied index and term.
+ */
+public class OMRatisSnapshotInfo implements SnapshotInfo {
+
+ static final Logger LOG = LoggerFactory.getLogger(OMRatisSnapshotInfo.class);
+
+ private volatile long term = 0;
+ private volatile long snapshotIndex = -1;
+
+ private final File ratisSnapshotFile;
+
+ private final static String TERM_PREFIX = "term";
Review comment:
Updated to Yaml.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load
Snapshot info when OM Ratis server starts.
URL: https://github.com/apache/hadoop/pull/1130#discussion_r315976747
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OzoneManagerStateMachine.java
##
@@ -103,6 +111,16 @@ public void initialize(RaftServer server, RaftGroupId id,
});
}
+ @Override
+ public SnapshotInfo getLatestSnapshot() {
+return snapshotInfo;
+ }
+
+ @Override
+ public void notifyIndexUpdate(long term, long index) {
Review comment:
The supplied index here is the latest log index. We do not want the
snapshotIndex to be updated to the last log index. It should be the last
applied index which is persisted to the DB. So we don't update that here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load
Snapshot info when OM Ratis server starts.
URL: https://github.com/apache/hadoop/pull/1130#discussion_r315976601
##
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestOMRatisSnapshotInfo.java
##
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.snapshot;
+
+import org.apache.hadoop.ozone.om.ratis.OMRatisSnapshotInfo;
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import java.io.File;
+
+/**
+ * Tests {@link org.apache.hadoop.ozone.om.ratis.OMRatisSnapshotInfo}
+ */
+public class TestOMRatisSnapshotInfo {
+
+ @Rule
+ public TemporaryFolder folder = new TemporaryFolder();
+
+ @Test
+ public void testSaveAndLoadSnapshotInfo() throws Exception {
+File rootDir = folder.newFolder();
+OMRatisSnapshotInfo omRatisSnapshotInfo = new OMRatisSnapshotInfo(rootDir);
+
+// Initially term and index should be 0
+Assert.assertEquals(0, omRatisSnapshotInfo.getTerm());
+Assert.assertEquals(-1, omRatisSnapshotInfo.getIndex());
+
+int snapshotIndex = 10;
+int termIndex = 2;
+
+// Save snapshotInfo to disk
+omRatisSnapshotInfo.updateTerm(termIndex);
+omRatisSnapshotInfo.saveRatisSnapshotToDisk(snapshotIndex);
+
+Assert.assertEquals(2, omRatisSnapshotInfo.getTerm());
+Assert.assertEquals(10, omRatisSnapshotInfo.getIndex());
Review comment:
These were random values I set for the term and index. I updated to use
Random.nextInt() instead of constant value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru commented on a change in pull request #1130: HDDS-1827. Load
Snapshot info when OM Ratis server starts.
URL: https://github.com/apache/hadoop/pull/1130#discussion_r314449239
##
File path:
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/ratis/TestOzoneManagerRatisServer.java
##
@@ -126,6 +130,24 @@ public void testStartOMRatisServer() throws Exception {
LifeCycle.State.RUNNING, omRatisServer.getServerState());
}
+ @Test
+ public void testLoadSnapshotInfoOnStart() throws Exception {
+// Stop the Ratis server and manually update the snapshotInfo.
+long oldSnaphsotIndex = ozoneManager.saveRatisSnapshot();
+ozoneManager.getSnapshotInfo().saveRatisSnapshotToDisk(oldSnaphsotIndex);
+omRatisServer.stop();
+long newSnapshotIndex = oldSnaphsotIndex + 100;
+ozoneManager.getSnapshotInfo().saveRatisSnapshotToDisk(newSnapshotIndex);
+
+// Start new Ratis server. It should pick up and load the new SnapshotInfo
+omRatisServer = OzoneManagerRatisServer.newOMRatisServer(conf,
ozoneManager,
+omNodeDetails, Collections.emptyList());
+omRatisServer.start();
Review comment:
omRatisServer is stopped when shutdown() is executed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
