SendDreams commented on issue #20485:
URL: https://github.com/apache/hudi/issues/20485#issuecomment-6594371567
> 3\. Could you try building the Flink bundle with the Hive 3 profile?
> mvn clean package -DskipTests -Drat.skip=true \
> -Pflink2.0,flink-bundle-shade-hive3
>
>
> This bundles Hive 3.1.2, which still has `METASTOREURIS`. Please also
check that the JobManager loads `HiveConf` from this bundle rather than a Hive
4 jar.
Sorry! Our project use hive 4.0.1, so I must build with hive 4!
I download the source codes。My idea is update the Module hudi-sync pom to
hive 4。for example:
<!-- Hive -->
<dependency>
<groupId>${hive.groupid}</groupId>
<artifactId>hive-service</artifactId>
<version>4.0.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${hive.groupid}</groupId>
<artifactId>hive-jdbc</artifactId>
<version>4.0.1</version>
</dependency>
<dependency>
<groupId>${hive.groupid}</groupId>
<artifactId>hive-metastore</artifactId>
<version>4.0.1</version>
</dependency>
<dependency>
<groupId>${hive.groupid}</groupId>
<artifactId>hive-common</artifactId>
<version>4.0.1</version>
</dependency>
<dependency>
<groupId>${hive.groupid}</groupId>
<artifactId>hive-exec</artifactId>
<version>4.0.1</version>
</dependency>
And update Module hudi-flink-bundle to hive 4. for example:
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<parent>
<artifactId>hudi</artifactId>
<groupId>org.apache.hudi</groupId>
<version>1.2.0</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>hudi-flink${flink.bundle.version}-bundle</artifactId>
<packaging>jar</packaging>
<properties>
<checkstyle.skip>true</checkstyle.skip>
<main.basedir>${project.parent.basedir}</main.basedir>
<skipTests>true</skipTests>
<flink.bundle.hive.scope>provided</flink.bundle.hive.scope>
<flink.bundle.shade.prefix>org.apache.hudi.</flink.bundle.shade.prefix>
<javax.servlet.version>3.1.0</javax.servlet.version>
<!-- override to be same with flink 1.15.x -->
<parquet.version>${flink.format.parquet.version}</parquet.version>
<hive.version>4.0.1</hive.version>
<thrift.version>0.9.3</thrift.version>
</properties>
And then rebuild the project
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
