tooptoop4 edited a comment on issue #836:
URL: https://github.com/apache/incubator-hudi/issues/836#issuecomment-625480332
just posting this in case anyone else hits this.
i had hive (v 2.3.4) tables using 'com.uber.hoodie.hadoop.HoodieInputFormat'
and COW hoodie files (written by spark 2.3.x using hoodie v 0.4.6) working with
prestodb 220, java8, there was a custom hoodie jar in the presto plugin folder.
i then tried to upgrade to prestosql 333 (which requires java11) but did not
change hive inputformat definitions/version or hoodie jar in presto or how
spark writing to the hoodie files. Simply trying presto (and JRE it runs on)
change...
First hit this error on 333:
```
Inconsistent constant pool data in classfile for class
com/uber/hoodie/common/table/HoodieTimeline. Method \u0027boolean
lambda$static$0(java.lang.String, java.lang.String)\u0027 at index 231 is
CONSTANT_MethodRef and should be CONSTANT_InterfaceMethodRef
Could not initialize class
com.uber.hoodie.common.table.timeline.HoodieActiveTimeline [DB
Errorcode=16777221]
"failureInfo" : {
"type" : "io.prestosql.spi.PrestoException",
"message" : "Could not initialize class
com.uber.hoodie.common.table.timeline.HoodieActiveTimeline",
"cause" : {
"type" : "java.lang.NoClassDefFoundError",
"message" : "Could not initialize class
com.uber.hoodie.common.table.timeline.HoodieActiveTimeline",
"suppressed" : [ ],
"stack" : [
"com.uber.hoodie.common.table.HoodieTableMetaClient.getActiveTimeline(HoodieTableMetaClient.java:192)",
"com.uber.hoodie.hadoop.HoodieInputFormat.listStatus(HoodieInputFormat.java:86)",
"org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:325)",
"io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:407)",
"io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:287)",
"io.prestosql.plugin.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:107)",
"io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:216)",
"io.prestosql.plugin.hive.util.ResumableTasks$1.run(ResumableTasks.java:38)",
"io.prestosql.$gen.Presto_333____20200507_180703_2.run(Unknown Source)",
"io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)",
"java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)",
"java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)",
"java.base/java.lang.Thread.run(Unknown Source)" ],
"errorCode" : {
"code" : 65536,
"name" : "GENERIC_INTERNAL_ERROR",
"type" : "INTERNAL_ERROR"
}
},
"suppressed" : [ ],
"stack" : [
"io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:223)",
"io.prestosql.plugin.hive.util.ResumableTasks$1.run(ResumableTasks.java:38)",
"io.prestosql.$gen.Presto_333____20200507_180703_2.run(Unknown Source)",
"io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)",
"java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)",
"java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)",
"java.base/java.lang.Thread.run(Unknown Source)" ],
"errorCode" : {
"code" : 16777221,
"name" : "HIVE_UNKNOWN_ERROR",
"type" : "EXTERNAL"
}
},
```
then tried to replace the custom hoodie jar in presto plugin folder with
https://repo1.maven.org/maven2/com/uber/hoodie/hoodie-presto-bundle/0.4.7/hoodie-presto-bundle-0.4.7.jar
hit this error:
```
2020-05-07T20:01:19.405Z ERROR main
io.prestosql.server.PrestoServer
'com.fasterxml.jackson.databind.ObjectMapper
com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(com.fasterxml.jackson.annotation.JsonInclude$Value)'
java.lang.NoSuchMethodError: 'com.fasterxml.jackson.databind.ObjectMapper
com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(com.fasterxml.jackson.annotation.JsonInclude$Value)'
at
io.airlift.json.ObjectMapperProvider.get(ObjectMapperProvider.java:128)
at
io.airlift.json.ObjectMapperProvider.get(ObjectMapperProvider.java:47)
at
com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:85)
at
com.google.inject.internal.BoundProviderFactory.provision(BoundProviderFactory.java:77)
at
com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:66)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:112)
at
io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:120)
at
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
at
com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)
at
com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:61)
at
com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1094)
at
io.airlift.json.JsonCodecFactory.createObjectMapper(JsonCodecFactory.java:130)
at
io.airlift.json.JsonCodecFactory.jsonCodec(JsonCodecFactory.java:70)
at io.airlift.json.JsonCodecProvider.get(JsonCodecProvider.java:43)
at io.airlift.json.JsonCodecProvider.get(JsonCodecProvider.java:23)
at
com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:85)
at
com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:57)
at
com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:66)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:112)
at
io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:120)
at
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
at
com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)
at
com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:47)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at
com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at
com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at
com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
at
com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
at
com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32)
at
com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:98)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:112)
at
io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54)
at
com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:120)
at
com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93)
at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at
com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at
com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at
com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:213)
at
com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
at
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:111)
at com.google.inject.Guice.createInjector(Guice.java:87)
at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:262)
at
io.prestosql.plugin.hive.InternalHiveConnectorFactory.createConnector(InternalHiveConnectorFactory.java:118)
at
io.prestosql.plugin.hive.InternalHiveConnectorFactory.createConnector(InternalHiveConnectorFactory.java:78)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at
io.prestosql.plugin.hive.HiveConnectorFactory.create(HiveConnectorFactory.java:63)
at
io.prestosql.connector.ConnectorManager.createConnector(ConnectorManager.java:349)
at
io.prestosql.connector.ConnectorManager.createCatalog(ConnectorManager.java:208)
at
io.prestosql.connector.ConnectorManager.createCatalog(ConnectorManager.java:200)
at
io.prestosql.connector.ConnectorManager.createCatalog(ConnectorManager.java:186)
at
io.prestosql.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
at
io.prestosql.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
at io.prestosql.server.PrestoServer.run(PrestoServer.java:118)
at io.prestosql.$gen.Presto_333____20200507_200106_1.run(Unknown
Source)
at io.prestosql.server.PrestoServer.main(PrestoServer.java:72)
```
then instead put
https://mvnrepository.com/artifact/org.apache.hudi/hudi-presto-bundle/0.5.2-incubating
in presto plugin folder and seems to work. So i guess new hudi bundle shades
jackson , caters for java11 reflection and can still read the uber inputformat
from hive definitions? @bhasudha
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
