adoroszlai commented on code in PR #10966: URL: https://github.com/apache/ozone/pull/10966#discussion_r4371876613
########## hadoop-ozone/ozone-testcontainers/src/main/java/org/apache/ozone/testcontainers/OzoneContainer.java: ########## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ozone.testcontainers; + +import java.time.Duration; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.containers.wait.strategy.Wait; +import org.testcontainers.containers.wait.strategy.WaitAllStrategy; +import org.testcontainers.utility.DockerImageName; + +/** + * Testcontainers wrapper for a single-node Ozone with S3 Gateway, so JVM + * integration tests can start Ozone the way they use the MinIO or LocalStack + * modules. Wraps the all-in-one quickstart image (HDDS-14452 / ozone-docker#49), + * which runs SCM, OM, datanode, S3 Gateway, Recon and HttpFS in one container + * and exposes S3G on 9878. + * + * <p>Credentials: in non-secure mode S3G does not verify request signatures, so any secret / region is + * accepted. The access key is still passed to OM as the Ozone user name, so it decides the owner of the + * buckets and keys a test creates. {@link #getAccessKey()} / {@link #getSecretKey()} / {@link #getRegion()} + * provide a consistent set for the caller to sign requests with. + * + * <p>Readiness: the container waits for the S3 Gateway port, then retries {@code ozone admin safemode wait} + * until SCM leaves safe mode. The startup timeout defaults to four minutes and can be changed with + * {@link #withStartupTimeout(Duration)}. + * + * <p><b>Incubating:</b> this module is new and its API may change in a future + * release. Review Comment: We have an annotation to emphasize unstable interface. ```java import org.apache.hadoop.hdds.annotation.InterfaceStability; @InterfaceStability.Evolving ``` ########## hadoop-ozone/ozone-testcontainers/pom.xml: ########## @@ -0,0 +1,79 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> +<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd";> + <modelVersion>4.0.0</modelVersion> + <parent> + <groupId>org.apache.ozone</groupId> + <artifactId>ozone</artifactId> + <version>2.3.0-SNAPSHOT</version> + </parent> + <artifactId>ozone-testcontainers</artifactId> + <version>2.3.0-SNAPSHOT</version> + <packaging>jar</packaging> + <name>Apache Ozone Testcontainers</name> + <description>Testcontainers module to run a single-node Ozone with S3 Gateway in integration tests</description> + + <properties> + <aws-java-sdk2.version>2.55.6</aws-java-sdk2.version> + <maven.compiler.release>${ozone.java.version.client}</maven.compiler.release> + <testcontainers.version>2.0.5</testcontainers.version> + </properties> + + <dependencyManagement> + <dependencies> + <dependency> + <groupId>software.amazon.awssdk</groupId> + <artifactId>bom</artifactId> + <version>${aws-java-sdk2.version}</version> + <type>pom</type> + <scope>import</scope> + </dependency> + </dependencies> + </dependencyManagement> Review Comment: This is no longer needed after 1acf315d5467a18f125f1e516e42741556cfcdc5. Please also remove `aws-java-sdk2.version`. ########## hadoop-ozone/ozone-testcontainers/src/main/java/org/apache/ozone/testcontainers/OzoneContainer.java: ########## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ozone.testcontainers; + +import java.time.Duration; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.containers.wait.strategy.Wait; +import org.testcontainers.containers.wait.strategy.WaitAllStrategy; +import org.testcontainers.utility.DockerImageName; + +/** + * Testcontainers wrapper for a single-node Ozone with S3 Gateway, so JVM + * integration tests can start Ozone the way they use the MinIO or LocalStack + * modules. Wraps the all-in-one quickstart image (HDDS-14452 / ozone-docker#49), + * which runs SCM, OM, datanode, S3 Gateway, Recon and HttpFS in one container + * and exposes S3G on 9878. + * + * <p>Credentials: in non-secure mode S3G does not verify request signatures, so any secret / region is + * accepted. The access key is still passed to OM as the Ozone user name, so it decides the owner of the + * buckets and keys a test creates. {@link #getAccessKey()} / {@link #getSecretKey()} / {@link #getRegion()} + * provide a consistent set for the caller to sign requests with. + * + * <p>Readiness: the container waits for the S3 Gateway port, then retries {@code ozone admin safemode wait} + * until SCM leaves safe mode. The startup timeout defaults to four minutes and can be changed with + * {@link #withStartupTimeout(Duration)}. + * + * <p><b>Incubating:</b> this module is new and its API may change in a future + * release. + */ +public class OzoneContainer extends GenericContainer<OzoneContainer> { + + public static final int S3G_PORT = 9878; + + static final DockerImageName DEFAULT_IMAGE = + DockerImageName.parse("apache/ozone:2.2.1-all-in-one"); Review Comment: The new `ozone-testcontainers` module is completely independent of the rest of the Ozone codebase. If we change Ozone core, testcontainer does not need to be rebuilt / tested, and vice versa. The default image is not the same as the current version (`2.3.0-SNAPSHOT` now), since there is no such image. It cannot match the version even at release time (2.3.0), since the release must precede Docker image creation (chicken-egg problem). Would it then make sense to create it as a completely distinct project with its own release cadence? Similar to Ozone Helm Charts. Or is it common to have testcontainers module embedded in the main project? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
