zhang-yue1 opened a new issue, #14254:
URL: https://github.com/apache/hudi/issues/14254
### Describe the problem you faced
**Summary**
When running a Flink job with `hudi-flink1.14-bundle-1.0.2.jar` I get a
runtime error during HFile initialization:
Caused by: java.lang.RuntimeException: Could not create interface
org.apache.hudi.org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceFactory
Is the hadoop compatibility jar on the classpath?
at
org.apache.hudi.org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:74)
~[hudi-flink1.14-bundle-1.0.2.jar:1.0.2]
at
org.apache.hudi.org.apache.hadoop.hbase.io.MetricsIO.<init>(MetricsIO.java:31)
~[hudi-flink1.14-bundle-1.0.2.jar:1.0.2]
at
org.apache.hudi.org.apache.hadoop.hbase.io.hfile.HFile.<clinit>(HFile.java:172)
~[hudi-flink1.14-bundle-1.0.2.jar:1.0.2]
**Environment**
- Hudi version: 1.0.2 (using prebuilt `hudi-flink1.14-bundle-1.0.2.jar`)
- Flink version: 1.14.5
- JDK: 1.8.0_251
- Deployment: Flink on YARN (application / per-job mode) — job uses the hudi
flink bundle jar
- Notes: The hudi bundle appears to relocate HBase classes under
`org.apache.hudi.org.apache.hadoop.hbase.*` (shaded/relocated).
**What I observed**
- The bundle fails while initializing
`org.apache.hudi.org.apache.hadoop.hbase.io.hfile.HFile` with the message
saying the "hadoop compatibility jar" may be missing.
- If I put `hbase-hadoop-compat` (and related HBase jars) into
`$FLINK_HOME/lib`, the "Is the hadoop compatibility jar on the classpath?"
error disappears, but then I hit class initialization failures (`Could not
initialize class org.apache.hudi.org.apache.hadoop.hbase.io.hfile.HFile`) —
indicating a shaded-vs-cluster jar conflict.
- If I remove HBase jars from `$FLINK_HOME/lib` and rely on the bundle
shading, the initial error about missing compatibility jar occurs.
### To Reproduce
1.
2.
3.1
4.
### Expected behavior
The Flink job should start and run successfully, writing data to the Hudi
table (COW or MOR) without throwing MetricsRegionServerSourceFactory or HFile
initialization exceptions.
It should not require placing any HBase-related jars in $FLINK_HOME/lib; the
job should run correctly using only the classes shaded inside
hudi-flink1.14-bundle-1.0.2.jar.
HFile writes should complete successfully, and the job should be able to
commit/compact without class conflicts or compatibility issues.
### Environment Description
* Hudi version:1.0.2
* Spark version:3.3.3
* Flink version:1.14.5
* Hive version:
* Hadoop version:3.1.0
* Storage (HDFS/S3/GCS..):
* Running on Docker? (yes/no):
### Additional context
_No response_
### Stacktrace
```shell
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
