errose28 commented on a change in pull request #3018:
URL: https://github.com/apache/ozone/pull/3018#discussion_r802029974
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/multitenant/TestMultiTenantVolume.java
##########
@@ -43,22 +55,108 @@
private static MiniOzoneCluster cluster;
private static String s3VolumeName;
+ private static final String TENANT_NAME = "tenant";
+ private static final String USER_PRINCIPAL = "username";
+ private static final String BUCKET_NAME = "bucket";
+ private static final String ACCESS_ID = UUID.randomUUID().toString();
+
@BeforeClass
public static void initClusterProvider() throws Exception {
OzoneConfiguration conf = new OzoneConfiguration();
conf.setBoolean(
OMMultiTenantManagerImpl.OZONE_OM_TENANT_DEV_SKIP_RANGER, true);
MiniOzoneCluster.Builder builder = MiniOzoneCluster.newBuilder(conf)
- .withoutDatanodes();
+ .withoutDatanodes()
+ .setOmLayoutVersion(OMLayoutFeature.INITIAL_VERSION.layoutVersion());
cluster = builder.build();
s3VolumeName = HddsClientUtils.getDefaultS3VolumeName(conf);
+
+ preFinalizationChecks(getStoreForAccessID(ACCESS_ID));
+ finalizeOMUpgrade();
}
@AfterClass
public static void shutdownClusterProvider() {
cluster.shutdown();
}
+ private static void expectFailurePreFinalization(VoidCallable eval)
+ throws Exception {
+ LambdaTestUtils.intercept(OMException.class,
+ "cannot be invoked before finalization", eval);
+ }
+
+ /**
+ * Perform sanity checks before triggering upgrade finalization.
+ */
+ private static void preFinalizationChecks(ObjectStore store)
+ throws Exception {
+
+ // None of the tenant APIs is usable before the upgrade finalization step
+ expectFailurePreFinalization(
+ store::listTenant);
+ expectFailurePreFinalization(() ->
+ store.listUsersInTenant(TENANT_NAME, ""));
+ expectFailurePreFinalization(() ->
+ store.tenantGetUserInfo(USER_PRINCIPAL));
+ expectFailurePreFinalization(() ->
+ store.createTenant(TENANT_NAME));
+ expectFailurePreFinalization(() ->
+ store.tenantAssignUserAccessId(USER_PRINCIPAL, TENANT_NAME,
ACCESS_ID));
+ expectFailurePreFinalization(() ->
+ store.tenantAssignAdmin(USER_PRINCIPAL, TENANT_NAME, true));
+ expectFailurePreFinalization(() ->
+ store.tenantRevokeAdmin(ACCESS_ID, TENANT_NAME));
+ expectFailurePreFinalization(() ->
+ store.tenantRevokeUserAccessId(ACCESS_ID));
+ expectFailurePreFinalization(() ->
+ store.deleteTenant(TENANT_NAME));
+
+ // S3 get/set/revoke secret APIs still work before finalization
+ final String accessId = "testUser1accessId1";
+ S3SecretValue s3SecretValue = store.getS3Secret(accessId);
+ Assert.assertEquals(accessId, s3SecretValue.getAwsAccessKey());
+ final String setSecret = "testsecret";
+ s3SecretValue = store.setS3Secret(accessId, setSecret);
+ Assert.assertEquals(accessId, s3SecretValue.getAwsAccessKey());
+ Assert.assertEquals(setSecret, s3SecretValue.getAwsSecret());
+ store.revokeS3Secret(accessId);
+ }
+
+ /**
+ * Trigger OM upgrade finalization from the client and block until completion
+ * (status FINALIZATION_DONE).
+ */
+ private static void finalizeOMUpgrade()
+ throws IOException, InterruptedException, TimeoutException {
+
+ // Trigger OM upgrade finalization. Ref: FinalizeUpgradeSubCommand#call
+ final OzoneManagerProtocol client = cluster.getRpcClient().getObjectStore()
+ .getClientProxy().getOzoneManagerClient();
+ final String upgradeClientID = "Test-Upgrade-Client-" + UUID.randomUUID();
+ UpgradeFinalizer.StatusAndMessages finalizationResponse =
+ client.finalizeUpgrade(upgradeClientID);
+
+ // Not sure if the status transitions as soon as client call returns.
+ // Can remove if this causes issue.
+ Assert.assertTrue(isStarting(finalizationResponse.status()));
Review comment:
isStarting is the expected return value here, so it shouldn't hurt to
leave it in. If something else is returned it would be an error since the
cluster should not already be finalized. The finalization required status
should only returned if the query upgrade progress command was given here
instead of a finalize command.
##########
File path: hadoop-ozone/dist/src/main/smoketest/upgrade/validate.robot
##########
@@ -28,3 +29,25 @@ Read data from previously created key
${random} = Generate Random String 5 [NUMBERS]
${output} = Execute ozone sh key get
/${PREFIX}-volume/${PREFIX}-bucket/${PREFIX}-key /tmp/key-${random}
Should not contain ${output} Failed
+
+Setup credentials for S3
+# TODO: Run "Setup secure v4 headers" instead when security is enabled
+ Run Keyword Setup dummy credentials for S3
+
+Read data from previously created key using S3 API
+ ${result} = Execute AWSS3APICli and checkrc get-object --bucket
${PREFIX}-bucket --key key1 /tmp/key1.result 0
+ # Note: "Compare files" doesn't work on NOTICE.txt as it is updated in new
Ozone versions.
Review comment:
How about the license file? Should be more stable. We could update the
generate/validate for normal keys to use this as well.
##########
File path: hadoop-ozone/dist/src/main/smoketest/upgrade/validate.robot
##########
@@ -28,3 +29,25 @@ Read data from previously created key
${random} = Generate Random String 5 [NUMBERS]
${output} = Execute ozone sh key get
/${PREFIX}-volume/${PREFIX}-bucket/${PREFIX}-key /tmp/key-${random}
Should not contain ${output} Failed
+
+Setup credentials for S3
+# TODO: Run "Setup secure v4 headers" instead when security is enabled
+ Run Keyword Setup dummy credentials for S3
+
+Read data from previously created key using S3 API
+ ${result} = Execute AWSS3APICli and checkrc get-object --bucket
${PREFIX}-bucket --key key1 /tmp/key1.result 0
+ # Note: "Compare files" doesn't work on NOTICE.txt as it is updated in new
Ozone versions.
+# Compare files /opt/hadoop/NOTICE.txt
/tmp/key1.result
+ Execute and checkrc rm /tmp/key1.result 0
+
+Create bucket using S3 API if the bucket doesn't exist
Review comment:
Can we move the aws client writes to generate.robot and leave the aws
client reads in validate.robot? It seems a little confusing now having
validate.robot do generate and validate for S3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
