Reamer commented on a change in pull request #4266: URL: https://github.com/apache/zeppelin/pull/4266#discussion_r757291318
########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> <profile> <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> Review comment: I don't see any use for this property. Where have we used it? ########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> <profile> <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> Review comment: I think we can remove this dependency because `hadoop-client-api` is a transitive dependency of `hadoop-client-runtime`. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> Review comment: The same as in `zeppelin-server`. I don't see any use for this property. ########## File path: zeppelin-server/pom.xml ########## @@ -293,30 +293,9 @@ </dependency> <dependency> - <groupId>org.apache.hadoop</groupId> - <artifactId>hadoop-client</artifactId> - <exclusions> - <!-- Provided by javax-websocket-server-impl --> - <exclusion> - <groupId>org.ow2.asm</groupId> - <artifactId>asm</artifactId> - </exclusion> - </exclusions> - </dependency> - - <!--test libraries--> - <dependency> - <groupId>org.apache.hadoop</groupId> - <artifactId>hadoop-common</artifactId> - <classifier>tests</classifier> - <scope>test</scope> - <exclusions> - <!-- Provided by javax-websocket-server-impl --> - <exclusion> - <groupId>org.ow2.asm</groupId> - <artifactId>asm</artifactId> - </exclusion> - </exclusions> + <groupId>org.apache.directory.server</groupId> + <artifactId>apacheds-kerberos-codec</artifactId> + <version>2.0.0-M15</version> Review comment: This version should be moved to a new property. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> Review comment: I do not see any use of `curator.version`, this property should be removed. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-api.artifact}</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-runtime.artifact}</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-minicluster.artifact}</artifactId> Review comment: scope `test` and version `hadoop.version` should be added ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> Review comment: `hadoop-yarn-common` is a transitive dependency of `hadoop-yarn-client`, it does not need to be defined twice. ########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> Review comment: I do not see any use of `commons-io.version`, this property should be removed. ########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> Review comment: Where do we use YARN components in zeppelin-server? In my opinion, these dependencies can be removed. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> Review comment: The same as in `zeppelin-server`. I don't see any use for this property. ########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> Review comment: I do not see any use of `curator.version`, this property should be removed. ########## File path: zeppelin-server/pom.xml ########## @@ -498,17 +477,95 @@ </zeppelin.daemon.package.base> </properties> </profile> + + <profile> + <id>hadoop2</id> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + <version>${hadoop.version}</version> + <scope>${hadoop.deps.scope}</scope> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> <profile> <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> Review comment: This version is hard coded in line 298. We should remove this at this point. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> Review comment: `hadoop-yarn-client` is declared twice. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-api.artifact}</artifactId> Review comment: `hadoop-client-api` is a transitive dependency of `hadoop-client-runtime`, it does not need to be defined twice. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> Review comment: `hadoop-yarn-common` is a transitive dependency of `hadoop-yarn-client`, it does not need to be defined twice. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> Review comment: `hadoop-yarn-api` is a transitive dependency of `hadoop-yarn-client`, it does not need to be defined twice. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> Review comment: I do not see any use of `commons-io.version`, this property should be removed. ########## File path: pom.xml ########## @@ -398,12 +401,27 @@ <version>${jettison.version}</version> </dependency> +<!-- <dependency>--> +<!-- <groupId>org.apache.hadoop</groupId>--> +<!-- <artifactId>${hadoop-client-api.artifact}</artifactId>--> Review comment: Remove this block. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> Review comment: I see no use for this property. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> Review comment: I do not see any use of `curator.version`, this property should be removed. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-api.artifact}</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-runtime.artifact}</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-minicluster.artifact}</artifactId> Review comment: scope `test` and version `hadoop.version` should be added ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> Review comment: Duplicate declaration. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> Review comment: `hadoop-yarn-api` is a transitive dependency of `hadoop-yarn-client`, it does not need to be defined twice. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> Review comment: I do not see any use of `kerberos-client.version`, this property should be removed. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-api.artifact}</artifactId> Review comment: I think we can remove this dependency because `hadoop-client-api` is a transitive dependency of `hadoop-client-runtime`. ########## File path: zeppelin-zengine/pom.xml ########## @@ -317,4 +312,86 @@ </plugin> </plugins> </build> + + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> Review comment: I see no use for this property. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> Review comment: Hadoop libraries are only needed for the YarnUtils class. If I understand YARN correctly, the Hadoop libraries are provided by the system. Therefore, I think we can set the dependency scope to `provided` and a small comment. ########## File path: zeppelin-interpreter/pom.xml ########## @@ -233,4 +226,86 @@ </resources> </build> + <profiles> + <profile> + <id>hadoop2</id> + + <activation> + <activeByDefault>true</activeByDefault> + </activation> + <properties> + <hadoop.version>${hadoop2.7.version}</hadoop.version> + <curator.version>2.7.1</curator.version> + <commons-io.version>2.11</commons-io.version> + <!-- + the declaration site above of these variables explains why we need to re-assign them here + --> + <hadoop-client-api.artifact>hadoop-client</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-yarn-api</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client</hadoop-client-minicluster.artifact> + </properties> + + <dependencies> + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-api</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-common</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>hadoop-yarn-client</artifactId> + </dependency> + </dependencies> + + </profile> + <profile> + <id>hadoop3</id> + + <properties> + <hadoop.version>${hadoop3.2.version}</hadoop.version> + <curator.version>2.13.0</curator.version> + <kerberos-client.version>2.0.0-M15</kerberos-client.version> + + <hadoop-client-api.artifact>hadoop-client-api</hadoop-client-api.artifact> + <hadoop-client-runtime.artifact>hadoop-client-runtime</hadoop-client-runtime.artifact> + <hadoop-client-minicluster.artifact>hadoop-client-minicluster</hadoop-client-minicluster.artifact> + + </properties> + + <dependencies> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-api.artifact}</artifactId> + </dependency> + + <dependency> + <groupId>org.apache.hadoop</groupId> + <artifactId>${hadoop-client-runtime.artifact}</artifactId> Review comment: Hadoop libraries are only needed for the `YarnUtils` class. If I understand YARN correctly, the Hadoop libraries are provided by the system. Therefore, I think we can set the dependency scope to `provided` and a small comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@zeppelin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org