[
https://issues.apache.org/jira/browse/HIVE-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277208#comment-17277208
]
Attila Magyar edited comment on HIVE-24584 at 2/2/21, 3:39 PM:
---------------------------------------------------------------
Hi Syed, thanks for trying it out.
I think it should have worked that way. Anyway I created a test that reproduces
the issue, depending on what value you set for IS_METASTORE_REMOTE.
{code:java}
$ cd itests/hive-unit
$ mvn test -Dtest=TestMsck
$ grep "Failed to deserialize the expression" target/tmp/log/hive.log {code}
The test doesn't fail in any case (that's a testing issue) but you can see the
Kryo related error in the log when you run it with IS_METASTORE_REMOTE=true.
[^msckrepro.patch]
was (Author: amagyar):
Hi Syed, thanks for trying it out.
I think it should have worked that way. Anyway I created a test that reproduces
the issue, depending on what value you set for IS_METASTORE_REMOTE.
{code:java}
$ cd itests/hive-unit
$ mvn test -Dtest=TestMsck
$ grep "Failed to deserialize the expression" target/tmp/log/hive.log {code}
The test doesn't fail in any case but you can see (that's a testing issue) but
you can see the Kry related error when you run it with IS_METASTORE_REMOTE=true.
[^msckrepro.patch]
> IndexOutOfBoundsException from Kryo when running msck repair
> ------------------------------------------------------------
>
> Key: HIVE-24584
> URL: https://issues.apache.org/jira/browse/HIVE-24584
> Project: Hive
> Issue Type: Bug
> Reporter: Attila Magyar
> Assignee: Attila Magyar
> Priority: Major
> Labels: pull-request-available
> Attachments: msckrepro.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The following exception is coming when running "msck repair table t1 sync
> partitions".
> {code:java}
> java.lang.IndexOutOfBoundsException: Index: 97, Size: 0
> at java.util.ArrayList.rangeCheck(ArrayList.java:657) ~[?:1.8.0_232]
> at java.util.ArrayList.get(ArrayList.java:433) ~[?:1.8.0_232]
> at
> org.apache.hive.com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:60)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:834)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:684)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:211)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeObjectFromKryo(SerializationUtilities.java:814)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.SerializationUtilities.deserializeExpressionFromKryo(SerializationUtilities.java:775)
> ~[hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionExpressionForMetastore.deserializeExpr(PartitionExpressionForMetastore.java:116)
> [hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionExpressionForMetastore.filterPartitionsByExpr(PartitionExpressionForMetastore.java:88)
> [hive-exec-3.1.3000.7.2.7.0-144.jar:3.1.3000.7.2.7.0-SNAPSHOT] {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
