lkemmerer commented on issue #433:
URL:
https://github.com/apache/incubator-xtable/issues/433#issuecomment-2125143847
I no longer see hadoop v2.10 in the install logs or dependency tree (yay!)
but I'm getting the (I think) same error. :-/
I ran (for completeness's sake, including git commands)
```shell
$ git pull
$ git checkout 433-hadoop-dependency
$ mvn clean install -DskipTests
$ java -jar utilities/target/utilities-0.1.0-SNAPSHOT-bundled.jar
--datasetConfig ../conversion.yaml
2024-05-22 08:19:07 INFO org.apache.xtable.utilities.RunSync:147 - Running
sync for basePath s3://s3-bucket-XXXX/x_table_prefix/ for following table
formats [DELTA]
2024-05-22 08:19:07 INFO
org.apache.hudi.common.table.HoodieTableMetaClient:133 - Loading
HoodieTableMetaClient from s3://s3-bucket-XXXX/x_table_prefix
2024-05-22 08:19:07 WARN org.apache.hadoop.util.NativeCodeLoader:60 -
Unable to load native-hadoop library for your platform... using builtin-java
classes where applicable
2024-05-22 08:19:07 WARN org.apache.hadoop.metrics2.impl.MetricsConfig:136
- Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2024-05-22 08:19:08 WARN org.apache.hadoop.fs.s3a.SDKV2Upgrade:39 -
Directly referencing AWS SDK V1 credential provider
com.amazonaws.auth.DefaultAWSCredentialsProviderChain. AWS SDK V1 credential
providers will be removed once S3A is upgraded to SDK V2
2024-05-22 08:19:08 INFO org.apache.hudi.common.table.HoodieTableConfig:276
- Loading table properties from
s3://s3-bucket-XXXX/x_table_prefix/.hoodie/hoodie.properties
Exception in thread "main" java.lang.NoSuchMethodError: 'java.lang.Object
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(org.apache.hadoop.fs.statistics.DurationTracker,
org.apache.hadoop.util.functional.CallableRaisingIOE)'
at
org.apache.hadoop.fs.s3a.Invoker.onceTrackingDuration(Invoker.java:147)
at
org.apache.hadoop.fs.s3a.S3AInputStream.reopen(S3AInputStream.java:282)
at
org.apache.hadoop.fs.s3a.S3AInputStream.lambda$lazySeek$1(S3AInputStream.java:435)
at
org.apache.hadoop.fs.s3a.Invoker.lambda$maybeRetry$3(Invoker.java:284)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
at
org.apache.hadoop.fs.s3a.Invoker.lambda$maybeRetry$5(Invoker.java:408)
at
org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:404)
at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:282)
at org.apache.hadoop.fs.s3a.Invoker.maybeRetry(Invoker.java:326)
at
org.apache.hadoop.fs.s3a.S3AInputStream.lazySeek(S3AInputStream.java:427)
at
org.apache.hadoop.fs.s3a.S3AInputStream.read(S3AInputStream.java:545)
at java.base/java.io.DataInputStream.read(DataInputStream.java:149)
at java.base/java.io.DataInputStream.read(DataInputStream.java:100)
at
java.base/java.util.Properties$LineReader.readLine(Properties.java:502)
at java.base/java.util.Properties.load0(Properties.java:418)
at java.base/java.util.Properties.load(Properties.java:407)
at
org.apache.hudi.common.table.HoodieTableConfig.fetchConfigs(HoodieTableConfig.java:352)
at
org.apache.hudi.common.table.HoodieTableConfig.<init>(HoodieTableConfig.java:278)
at
org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:141)
at
org.apache.hudi.common.table.HoodieTableMetaClient.newMetaClient(HoodieTableMetaClient.java:692)
at
org.apache.hudi.common.table.HoodieTableMetaClient.access$000(HoodieTableMetaClient.java:85)
at
org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:774)
at
org.apache.xtable.hudi.HudiSourceClientProvider.getSourceClientInstance(HudiSourceClientProvider.java:42)
at
org.apache.xtable.hudi.HudiSourceClientProvider.getSourceClientInstance(HudiSourceClientProvider.java:31)
at
org.apache.xtable.client.OneTableClient.sync(OneTableClient.java:90)
at org.apache.xtable.utilities.RunSync.main(RunSync.java:168)
```
[dependencies.txt](https://github.com/apache/incubator-xtable/files/15405833/dependencies.txt)
[install.txt](https://github.com/apache/incubator-xtable/files/15405834/install.txt)
The only thing I see that might be suspicious is the inclusion of
`org.apache.hbase:hbase-hadoop-compat:jar:2.4.9` as part of hudi common...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
