[
https://issues.apache.org/jira/browse/IMPALA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118589#comment-18118589
]
ASF subversion and git services commented on IMPALA-15346:
----------------------------------------------------------
Commit 8e7626ce231edca0807e7cf423923bed484fe4d4 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8e7626ce2 ]
IMPALA-15346: Fix IllegalStateException in createInsertEvents
This fixes an IllegalStateException in createInsertEvents when inserting
empty results into an unpartitioned table.
The exception is thrown in CatalogOpExecutor.createInsertEvents() when
asserting the number of updated partitions to be 1. The map of updated
partitions is created in CatalogOpExecutor.getFileMetadata() which loads
file metadata like checksums. It unintentionally skips adding partitions
that have an empty new file list by putting the put() statement in the
for-loop that iterates the file list. This patch fixes it by moving the
put() statement outside that for-loop. Also simplifies the case of empty
new file list.
The Preconditions check is updated to log the actual number of updated
partitions.
Testing
- Added custom-cluster test
Change-Id: I0eb366f55f4e95b229a57297e42d14760b31be21
Reviewed-on: http://gerrit.cloudera.org:8080/24835
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> IllegalStateException in createInsertEvents when inserting empty results into
> an unpartitioned table
> ----------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15346
> URL: https://issues.apache.org/jira/browse/IMPALA-15346
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> Saw an IllegalStateException in catalogd logs of
> query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_select_with_empty_resultset.
> It's pretty easy to reproduce the issue using an unpartitioned table:
> {code:sql}
> create table empty_tbl (i int);
> insert into empty_tbl select 1 from functional.alltypestiny limit 0;{code}
> In catalogd logs:
> {noformat}
> I20260914 09:24:50.147665 69505 JniUtil.java:181]
> 03450364b05b0caf:38045f1900000000] updateCatalog request: Update catalog for
> default.empty_tbl
> E20260914 09:24:50.148128 69505 CatalogOpExecutor.java:7904]
> 03450364b05b0caf:38045f1900000000] Failed to fire insert events for table
> default.empty_tbl
> Java exception follows:
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:8183)
> at
> org.apache.impala.service.CatalogOpExecutor.updateCatalogImpl(CatalogOpExecutor.java:7901)
> at
> org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:7753)
> at
> org.apache.impala.service.JniCatalog.lambda$updateCatalog$15(JniCatalog.java:558)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
> at
> org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:249)
> at
> org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:263)
> at
> org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:557)
> I20260914 09:24:50.151366 69505 HdfsTable.java:1276]
> 03450364b05b0caf:38045f1900000000] Reloading metadata for all partition(s) of
> default.empty_tbl (INSERT)
> I20260914 09:24:50.151432 69505 HdfsTable.java:3035]
> 03450364b05b0caf:38045f1900000000] Load Valid Write Id List Done. Time taken:
> 2.079us
> I20260914 09:24:50.152450 69505 HdfsTable.java:799]
> 03450364b05b0caf:38045f1900000000] Loaded file and block metadata for
> default.empty_tbl. Time taken: 812.141us.
> I20260914 09:24:50.152523 69505 HdfsTable.java:1325]
> 03450364b05b0caf:38045f1900000000] Incrementally loaded table metadata for:
> default.empty_tbl
> I20260914 09:24:50.152606 69505 HdfsTable.java:3131]
> 03450364b05b0caf:38045f1900000000] Setting the catalog version of
> HdfsTable@2c66125f default.empty_tbl to 2399
> I20260914 09:24:50.152704 69505 JniUtil.java:192]
> 03450364b05b0caf:38045f1900000000] Finished updateCatalog request: Update
> catalog for default.empty_tbl. Time spent: 5ms{noformat}
> This won't fail the INSERT since all exceptions thrown from
> createInsertEvents() are swallowed. But it's confusing and not firing an
> INSERT event as expected. Though the event is probably redundent, that's
> another issue. Hive also fires an empty INSERT event when the insert result
> set is empty.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
