Quanlong Huang created IMPALA-12257:
---------------------------------------
Summary: createInsertEvents failed by NullPointerException:
Invalid partition name
Key: IMPALA-12257
URL: https://issues.apache.org/jira/browse/IMPALA-12257
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Assignee: Quanlong Huang
INSERT on partitioned table could fail in createInsertEvents() if some updated
partitions are missing in catalogd but actually exists in HMS. The failure is a
NullPointerException, e.g.
{code:java}
I0630 14:44:29.779798 30287 jni-util.cc:288] java.lang.NullPointerException:
Invalid partition name: p=0
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:907)
at
org.apache.impala.catalog.HdfsTable.getPartitionsForNames(HdfsTable.java:1758)
at
org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:6935)
at
org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:6830)
at
org.apache.impala.service.JniCatalog.lambda$updateCatalog$16(JniCatalog.java:471)
at
org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:230)
at
org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:470) {code}
We've seen this in an issue caused by IMPALA-12256 in which the stale
DROP_PARTITION event incorrectly drops the partition, causing inconsistency
between catalogd and HMS on the partition list.
To steadily reproduce the issue, we can disable HMS event-processing and
manually make the partition list differs between catalogd and HMS.
{code}
impala> create table my_part (id int) partitioned by (p int) stored as textfile;
# Add one partition in Hive
hive> alter table my_part add partition(p=0);
{code}
Then run INSERT on the same partition in Impala:
{code}
impala> insert into my_part partition(p=0) values (0);
ERROR: NullPointerException: Invalid partition name: p=0
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
