yihua commented on code in PR #12964:
URL: https://github.com/apache/hudi/pull/12964#discussion_r2105687907
##########
hudi-spark-datasource/hudi-spark3-common/pom.xml:
##########
@@ -158,7 +158,16 @@
</build>
<dependencies>
-
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-mapreduce-client-core</artifactId>
+ <scope>compile</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-common</artifactId>
+ <scope>compile</scope>
+ </dependency>
Review Comment:
Should these be compile or provided scope?
##########
pom.xml:
##########
@@ -495,22 +476,6 @@
<pattern>org.apache.http.</pattern>
<shadedPattern>org.apache.hudi.org.apache.http.</shadedPattern>
</relocation>
- <!-- hbase -->
- <relocation>
- <pattern>org.apache.hadoop.hbase.</pattern>
-
<shadedPattern>org.apache.hudi.org.apache.hadoop.hbase.</shadedPattern>
- <excludes>
-
<exclude>org.apache.hadoop.hbase.KeyValue$KeyComparator</exclude>
- </excludes>
- </relocation>
- <relocation>
- <pattern>org.apache.hbase.</pattern>
- <shadedPattern>org.apache.hudi.org.apache.hbase.</shadedPattern>
- </relocation>
- <relocation>
- <pattern>org.apache.htrace.</pattern>
- <shadedPattern>org.apache.hudi.org.apache.htrace.</shadedPattern>
- </relocation>
Review Comment:
There are more relocation rules below for HBase. Those should be removed
too.
##########
hudi-sync/hudi-datahub-sync/pom.xml:
##########
@@ -93,6 +93,13 @@
<scope>compile</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-common</artifactId>
+ <version>${hadoop.version}</version>
+ <scope>compile</scope>
+ </dependency>
Review Comment:
Should `hudi-hadoop-common` module be added here?
##########
pom.xml:
##########
@@ -870,6 +835,11 @@
<version>${slf4j.version}</version>
<scope>provided</scope>
</dependency>
+ <dependency>
+ <groupId>javax.xml.bind</groupId>
+ <artifactId>jaxb-api</artifactId>
+ <version>2.3.1</version>
+ </dependency>
Review Comment:
What is this used for?
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/hudi/benchmark/HoodieBenchmarks.scala:
##########
@@ -16,14 +16,14 @@
*/
package org.apache.spark.hudi.benchmark
-import org.apache.hbase.thirdparty.com.google.common.reflect.ClassPath
+import org.apache.hadoop.shaded.com.google.common.reflect.ClassPath
import java.io.File
import java.lang.reflect.Modifier
import java.nio.file.{FileSystems, Paths}
import java.util.Locale
-import scala.collection.JavaConverters._
+import scala.jdk.CollectionConverters._
Review Comment:
Is this change necessary?
##########
pom.xml:
##########
@@ -1875,6 +1845,11 @@
<artifactId>mbknor-jackson-jsonschema_${scala.binary.version}</artifactId>
<version>1.0.39</version>
</dependency>
+ <dependency>
+ <groupId>javax.validation</groupId>
+ <artifactId>validation-api</artifactId>
+ <version>2.0.1.Final</version>
+ </dependency>
Review Comment:
What is this used for?
##########
hudi-sync/hudi-datahub-sync/pom.xml:
##########
@@ -93,6 +93,13 @@
<scope>compile</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.hadoop</groupId>
+ <artifactId>hadoop-common</artifactId>
+ <version>${hadoop.version}</version>
+ <scope>compile</scope>
+ </dependency>
Review Comment:
Wondering what are the failures without this dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
