bharatviswa504 commented on a change in pull request #982:
URL: https://github.com/apache/hadoop-ozone/pull/982#discussion_r431993761
##########
File path:
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBStore.java
##########
@@ -94,6 +97,14 @@ public RDBStore(File dbFile, DBOptions options,
this.writeOptions = writeOptions;
try {
+ List<TableConfig> columnFamiliesInDb = getColumnFamiliesInExistingDb();
+ List<TableConfig> extraCf = columnFamiliesInDb.stream().filter(
+ cf -> !families.contains(cf)).collect(Collectors.toList());
+ for (TableConfig family : extraCf) {
+ LOG.info("Adding column family {}", family.getName());
Review comment:
Can this comment be more clear saying ColumnFamily in DB, but not in the
current version something like that?
##########
File path:
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBStore.java
##########
@@ -94,6 +97,14 @@ public RDBStore(File dbFile, DBOptions options,
this.writeOptions = writeOptions;
try {
+ List<TableConfig> columnFamiliesInDb = getColumnFamiliesInExistingDb();
+ List<TableConfig> extraCf = columnFamiliesInDb.stream().filter(
+ cf -> !families.contains(cf)).collect(Collectors.toList());
+ for (TableConfig family : extraCf) {
+ LOG.info("Adding column family {}", family.getName());
Review comment:
One question, can these extra column families can be dropped, if they
are not required after upgrade/downgrade.
Because now we removed S3Table in #940. So, once after upgrade, technically
we don't require the column family any more.
##########
File path:
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/TestRDBStore.java
##########
@@ -348,5 +348,40 @@ public void testGetDBUpdatesSince() throws Exception {
}
}
+ @Test
+ public void testDowngrade() throws Exception {
+
+ // Write data to current DB which has 6 column families at the time of
+ // writing this test.
+ for (String family : families) {
+ try (Table table = rdbStore.getTable(family)) {
+ byte[] key = family.getBytes(StandardCharsets.UTF_8);
+ byte[] value =
+ RandomStringUtils.random(10).getBytes(StandardCharsets.UTF_8);
+ table.put(key, value);
+ }
+ }
+ // Close current DB.
+ rdbStore.close();
+
+ // Reopen DB with the last column family removed.
+ options = new DBOptions();
+ options.setCreateIfMissing(true);
+ options.setCreateMissingColumnFamilies(true);
+ configSet = new HashSet<>();
+ List<String> familiesMinusOne = families.subList(0, families.size() - 1);
+ for(String name : familiesMinusOne) {
+ TableConfig newConfig = new TableConfig(name, new ColumnFamilyOptions());
+ configSet.add(newConfig);
+ }
+ rdbStore = new RDBStore(rdbStore.getDbLocation(), options, configSet);
+ for (String family : familiesMinusOne) {
Review comment:
Question: If column family is missing also, but DBStore will open that
column family right currently? So, do we need to check the columnFamily is
opened or not also?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
