pvary commented on pull request #1505:
URL: https://github.com/apache/iceberg/pull/1505#issuecomment-698443907
> That was my fear as well, but did not have time to test them out yet. It
might even cause issues for users who are not using the Hive tables, just want
to use HiveCatalog to store the table versions.
Your comment give me a better idea how to test. Changed the serde to an
arbitrary string, and got this Exception:
```
FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException:
Error in loading storage
handler.org.apache.iceberg.mr.hive.HiveIcebergStorageHandler3
[a3db973d-5b1f-4fde-985b-ad5aa4d09018 Thread-4] ERROR
org.apache.hadoop.hive.ql.Driver - FAILED: RuntimeException
org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage
handler.org.apache.iceberg.mr.hive.HiveIcebergStorageHandler3
java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage
handler.org.apache.iceberg.mr.hive.HiveIcebergStorageHandler3
at
org.apache.hadoop.hive.ql.metadata.Table.getStorageHandler(Table.java:297)
at
org.apache.hadoop.hive.ql.metadata.Table.getInputFormatClass(Table.java:307)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2041)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1934)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11080)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11133)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10807)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1295)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:204)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:500)
at
org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:265)
at
com.klarna.hiverunner.HiveServerContainer.executeStatement(HiveServerContainer.java:120)
at
com.klarna.hiverunner.builder.HiveShellBase.executeStatementsWithCommandShellEmulation(HiveShellBase.java:115)
at
com.klarna.hiverunner.builder.HiveShellBase.executeStatementWithCommandShellEmulation(HiveShellBase.java:109)
at
com.klarna.hiverunner.builder.HiveShellBase.executeStatement(HiveShellBase.java:99)
at
org.apache.iceberg.mr.hive.HiveIcebergStorageHandlerBaseTest.testScanEmptyTable(HiveIcebergStorageHandlerBaseTest.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at
com.klarna.hiverunner.StandaloneHiveRunner.evaluateStatement(StandaloneHiveRunner.java:165)
at
com.klarna.hiverunner.HiveRunnerRule$HiveRunnerRuleStatement.evaluate(HiveRunnerRule.java:82)
at com.klarna.hiverunner.ThrowOnTimeout$1.run(ThrowOnTimeout.java:60)
at java.lang.Thread.run(Thread.java:748)
```
Is it ok to require "iceberg-mr" classes on the classpath to use HiveCatalog?
I would be afraid of this requirement until we do not have a Catalog
implementation which could serve as a replacement catalog in case we do not
have HDFS at hand.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
