[
https://issues.apache.org/jira/browse/HIVE-28229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dominik Diedrich updated HIVE-28229:
------------------------------------
Description:
I've been using _apache.hive_ with _apache.iceberg_ in order to communicate
with a Hive MetaStore. Since the 4.0.0-beta-1 changes, underscores for all
environment variables have been introduced. Instead of using the
_iceberg-hive-metastore_ dependency with including HiveCatalog,
HiveTableOperations I had to switch to _hive-iceberg-catalog_ from
_apache.hive_ dependency because of the now missing underscores in the
_apache.iceberg_ HiveCatalog class version.
Normally, this shouldn't be a problem but I figured that the
HiveTableOperations class of _apache.hive_ is using the relocated _fasterxml_
dependency and then calls the JsonUtil class when using _setSnapshotSummary,_
where _fasterxml_ is being imported by the non-relocated _fasterxml_ dependency
path.
Now the Hive Metastore Client breaks because I get the error message that the
ObjectMapper method can't be found:
{code:java}
DEBUG [log]
io.grpc.netty.shaded.io.netty.util.internal.logging.AbstractInternalLogger
(AbstractInternalLogger.java:214) : [id: 0xc34d5ce5, L:/127.0.0.1:1238 -
R:/127.0.0.1:37678] OUTBOUND HEADERS: streamId=3
headers=GrpcHttp2OutboundHeaders[:status: 200, content-type: application/grpc,
grpc-status: 2, grpc-message: Application error processing RPC] padding=0
endStream=true java.lang.NoSuchMethodError:
'org.apache.hive.iceberg.com.fasterxml.jackson.databind.ObjectMapper
org.apache.iceberg.util.JsonUtil.mapper()' at
org.apache.iceberg.hive.HiveTableOperations.setSnapshotSummary(HiveTableOperations.java:435)
at
org.apache.iceberg.hive.HiveTableOperations.setSnapshotStats(HiveTableOperations.java:426)
at
org.apache.iceberg.hive.HiveTableOperations.setHmsTableParameters(HiveTableOperations.java:408)
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:222)
at
org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135)
at
org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:417)
at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) at
org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:413)
at
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:308)
at
....
{code}
The only workound I could find is to manually add the Shadings of package
_hive-iceberg-shading_ from apache.hive to my own pom. But now I need to always
build an uber jar.
Did I find a bug here or is this expected behavior? If so, how can this be
fixed?
was:
I've been using _apache.hive_ with _apache.iceberg_ in order to communicate
with a Hive MetaStore. Since the 4.0.0-beta-1 changes, underscores for all
environment variables have been introduced. Instead of using the
_iceberg-hive-metastore_ dependency with including HiveCatalog,
HiveTableOperations I had to switch to _hive-iceberg-catalog_ from
_apache.hive_ dependency because of the now missing underscores in the
_apache.iceberg_ HiveCatalog class version.
Normally, this shouldn't be a problem but I figured that the
HiveTableOperations class of _apache.hive_ is using the relocated _fasterxml_
dependency and then calls the JsonUtil class when using _setSnapshotSummary,_
where _fasterxml_ is being imported by the non-relocated _fasterxml_ dependency
path.
Now the Hive Metastore Client breaks because I get the error message that the
ObjectMapper method can't be found:
{code:java}
DEBUG [log]
io.grpc.netty.shaded.io.netty.util.internal.logging.AbstractInternalLogger
(AbstractInternalLogger.java:214) : [id: 0xc34d5ce5, L:/127.0.0.1:1238 -
R:/127.0.0.1:37678] OUTBOUND HEADERS: streamId=3
headers=GrpcHttp2OutboundHeaders[:status: 200, content-type: application/grpc,
grpc-status: 2, grpc-message: Application error processing RPC] padding=0
endStream=true java.lang.NoSuchMethodError:
'org.apache.hive.iceberg.com.fasterxml.jackson.databind.ObjectMapper
org.apache.iceberg.util.JsonUtil.mapper()' at
org.apache.iceberg.hive.HiveTableOperations.setSnapshotSummary(HiveTableOperations.java:435)
at
org.apache.iceberg.hive.HiveTableOperations.setSnapshotStats(HiveTableOperations.java:426)
at
org.apache.iceberg.hive.HiveTableOperations.setHmsTableParameters(HiveTableOperations.java:408)
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:222)
at
org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135)
at
org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:417)
at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at
org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) at
org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:413)
at
org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:308)
at iceberg_cli.IcebergConnector.tableTransaction(IcebergConnector.java:1336) at
iceberg_cli.grpc.icebergmanager.IcebergManagerServer$IcebergManagerService.dataModifier(IcebergManagerServer.java:333)
at
iceberg_cli.grpc.icebergmanager.IcebergManagerGrpc$MethodHandlers.invoke(IcebergManagerGrpc.java:544)
at
io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at
io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:351)
at
io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:861)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at
io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
{code}
The only workound I could find is to manually add the Shadings of package
_hive-iceberg-shading_ from apache.hive to my own pom. But now I need to always
build an uber jar.
Did I find a bug here or is this expected behavior? If so, how can this be
fixed?
> Fasterxml Shading Problem (hive-iceberg-shading)
> ------------------------------------------------
>
> Key: HIVE-28229
> URL: https://issues.apache.org/jira/browse/HIVE-28229
> Project: Hive
> Issue Type: Bug
> Components: HCatalog, Hive
> Affects Versions: 4.0.0, 4.0.0-beta-1
> Environment: openjdk 18.0.2-ea 2022-07-19
> OpenJDK Runtime Environment (build 18.0.2-ea+9-Ubuntu-222.04)
> OpenJDK 64-Bit Server VM (build 18.0.2-ea+9-Ubuntu-222.04, mixed mode,
> sharing)
>
>
> Reporter: Dominik Diedrich
> Priority: Minor
>
> I've been using _apache.hive_ with _apache.iceberg_ in order to communicate
> with a Hive MetaStore. Since the 4.0.0-beta-1 changes, underscores for all
> environment variables have been introduced. Instead of using the
> _iceberg-hive-metastore_ dependency with including HiveCatalog,
> HiveTableOperations I had to switch to _hive-iceberg-catalog_ from
> _apache.hive_ dependency because of the now missing underscores in the
> _apache.iceberg_ HiveCatalog class version.
> Normally, this shouldn't be a problem but I figured that the
> HiveTableOperations class of _apache.hive_ is using the relocated _fasterxml_
> dependency and then calls the JsonUtil class when using _setSnapshotSummary,_
> where _fasterxml_ is being imported by the non-relocated _fasterxml_
> dependency path.
> Now the Hive Metastore Client breaks because I get the error message that the
> ObjectMapper method can't be found:
> {code:java}
> DEBUG [log]
> io.grpc.netty.shaded.io.netty.util.internal.logging.AbstractInternalLogger
> (AbstractInternalLogger.java:214) : [id: 0xc34d5ce5, L:/127.0.0.1:1238 -
> R:/127.0.0.1:37678] OUTBOUND HEADERS: streamId=3
> headers=GrpcHttp2OutboundHeaders[:status: 200, content-type:
> application/grpc, grpc-status: 2, grpc-message: Application error processing
> RPC] padding=0 endStream=true java.lang.NoSuchMethodError:
> 'org.apache.hive.iceberg.com.fasterxml.jackson.databind.ObjectMapper
> org.apache.iceberg.util.JsonUtil.mapper()' at
> org.apache.iceberg.hive.HiveTableOperations.setSnapshotSummary(HiveTableOperations.java:435)
> at
> org.apache.iceberg.hive.HiveTableOperations.setSnapshotStats(HiveTableOperations.java:426)
> at
> org.apache.iceberg.hive.HiveTableOperations.setHmsTableParameters(HiveTableOperations.java:408)
> at
> org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:222)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135)
> at
> org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:417)
> at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at
> org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at
> org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at
> org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) at
> org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:413)
> at
> org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:308)
> at
> ....
> {code}
> The only workound I could find is to manually add the Shadings of package
> _hive-iceberg-shading_ from apache.hive to my own pom. But now I need to
> always build an uber jar.
> Did I find a bug here or is this expected behavior? If so, how can this be
> fixed?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
