[
https://issues.apache.org/jira/browse/IMPALA-15051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-15051.
-------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> INVALIDATE METADATA fails with java.lang.IllegalStateException in
> TableEventExecutor.clear with case-sensitive rename
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15051
> URL: https://issues.apache.org/jira/browse/IMPALA-15051
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Venugopal Reddy K
> Assignee: Venugopal Reddy K
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> *[Description]*
> The Impala Catalog service crashes during an Invalidate metadata command due
> to the use of case-variant db name(e.g., 'Mydb', 'myDB', 'MYDB' etc are
> case-variants of 'mydb') in rename table operation. It is treated as a new
> instance even though the mapping of the table name to TableEventExecutor
> already exists. Upon running invalidate metadata, since the mapping to the
> old instance still exists, TableEventExecutor.clear() triggers a Google Guava
> state check failure ({{{}Preconditions.checkState{}}}). This occurs after a
> table is renamed to a case-variant database name (e.g., changing from
> lowercase to uppercase or vice versa).
> Exception is thrown at:
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java#L440]
> *[Reproduction steps]*
> {code:java}
> create database mydb;
> use mydb;
> create table test1(i int);
> insert into test1 values(1),(2),(3);
> alter table mydb.test1 rename to mydb.test2;
> alter table mydb.test2 rename to Mydb.Test1;
> invalidate metadata{code}
> *[Crash Logs]*
> {code:java}
> I20260529 18:31:07.856305 4116682 JniUtil.java:181]
> 16431b7fb1c71a99:2c87910c00000000] resetMetadata request: INVALIDATE ALL
> issued by kvenureddy
> I20260529 18:31:07.856406 4116682 CatalogServiceCatalog.java:2450]
> 16431b7fb1c71a99:2c87910c00000000] Invalidating all metadata. Version: 4319,
> IsCatalogServerRequest: false
> I20260529 18:31:07.857302 4116682 MetastoreEventsProcessor.java:1050]
> 16431b7fb1c71a99:2c87910c00000000] Event processing is paused. Greatest
> synced event id is 3609368
> I20260529 18:31:07.857375 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-0
> I20260529 18:31:07.857390 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-1
> I20260529 18:31:07.857402 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-2
> I20260529 18:31:07.857414 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-3
> I20260529 18:31:07.857425 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-4
> I20260529 18:31:07.857436 4116682 DbEventExecutor.java:622]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor: DbEventExecutor-2
> I20260529 18:31:07.857458 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-4.TableEventExecutor-0
> I20260529 18:31:07.857473 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-4.TableEventExecutor-1
> I20260529 18:31:07.857487 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-4.TableEventExecutor-2
> I20260529 18:31:07.857527 4114201 DbEventExecutor.java:760] Unassigned
> executor: DbEventExecutor-0.TableEventExecutor-1 for table: mydb.test2
> I20260529 18:31:07.857579 4114201 DbEventExecutor.java:760] Unassigned
> executor: DbEventExecutor-0.TableEventExecutor-2 for table: Mydb.test1
> I20260529 18:31:07.857611 4114204 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-0
> I20260529 18:31:07.857647 4114204 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-1
> I20260529 18:31:07.857669 4114204 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-2
> I20260529 18:31:07.857702 4114201 DbEventExecutor.java:804] Unassigned
> executor: DbEventExecutor-0 for db: mydb
> I20260529 18:31:07.857738 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-4.TableEventExecutor-3
> I20260529 18:31:07.857751 4116682 TableEventExecutor.java:443]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor:
> DbEventExecutor-4.TableEventExecutor-4
> I20260529 18:31:07.857764 4116682 DbEventExecutor.java:622]
> 16431b7fb1c71a99:2c87910c00000000] Cleared executor: DbEventExecutor-4
> I20260529 18:31:07.857791 4114205 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-0
> I20260529 18:31:07.857813 4114205 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-1
> I20260529 18:31:07.857822 4114205 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-2
> I20260529 18:31:07.857831 4114205 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-3
> I20260529 18:31:07.857838 4114205 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-4
> I20260529 18:31:07.857846 4114205 DbEventExecutor.java:622] Cleared executor:
> DbEventExecutor-3
> I20260529 18:31:07.857874 4114204 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-3
> I20260529 18:31:07.857908 4114204 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-4
> I20260529 18:31:07.857950 4114204 DbEventExecutor.java:622] Cleared executor:
> DbEventExecutor-1
> E20260529 18:31:07.858183 4116682 JniUtil.java:198]
> 16431b7fb1c71a99:2c87910c00000000] Error in INVALIDATE ALL issued by
> kvenureddy. Time spent: 2ms
> I20260529 18:31:07.858538 4116682 jni-util.cc:335]
> 16431b7fb1c71a99:2c87910c00000000] java.lang.IllegalStateException:
> java.lang.IllegalStateException
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
> at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
> at
> java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
> at
> java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
> at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
> at
> java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
> at
> java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
> at
> org.apache.impala.catalog.events.EventExecutorService.clear(EventExecutorService.java:195)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.clear(MetastoreEventsProcessor.java:1210)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:2475)
> at
> org.apache.impala.service.CatalogOpExecutor.execResetMetadataImpl(CatalogOpExecutor.java:7591)
> at
> org.apache.impala.service.CatalogOpExecutor.execResetMetadata(CatalogOpExecutor.java:7407)
> at
> org.apache.impala.service.JniCatalog.lambda$resetMetadata$4(JniCatalog.java:369)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
> at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:253)
> at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:267)
> at org.apache.impala.service.JniCatalog.resetMetadata(JniCatalog.java:368)
> Caused by: java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.catalog.events.TableEventExecutor.clear(TableEventExecutor.java:440)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at
> org.apache.impala.catalog.events.DbEventExecutor.clearInternal(DbEventExecutor.java:612)
> at
> org.apache.impala.catalog.events.DbEventExecutor.clear(DbEventExecutor.java:621)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
> at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
> at
> java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
> at
> java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
> at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
> at
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
> at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
> at
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
> at
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
> I20260529 18:31:07.858549 4116682 status.cc:129]
> 16431b7fb1c71a99:2c87910c00000000] IllegalStateException:
> java.lang.IllegalStateException
> CAUSED BY: IllegalStateException: null
> @ 0x1367653 impala::Status::Status()
> @ 0x1ce0b89 impala::JniUtil::GetJniExceptionMsg()
> @ 0x12e323f impala::JniCall::Call<>()
> @ 0x12b0da0 impala::Catalog::ResetMetadata()
> @ 0x1329b9a impala::CatalogServiceThriftIf::ResetMetadata()
> @ 0x1271451
> impala::CatalogServiceProcessorT<>::process_ResetMetadata()
> @ 0x129e163 impala::CatalogServiceProcessorT<>::dispatchCall()
> @ 0x11e66c5 apache::thrift::TDispatchProcessor::process()
> @ 0x16a7785
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x1693a35 impala::ThriftThread::RunRunnable()
> @ 0x1694a6c
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x1dc3b1a boost::function0<>::operator()()
> @ 0x1dc29f6 impala::Thread::SuperviseThread()
> @ 0x1dc3457 boost::detail::thread_data<>::run()
> @ 0x2921467 thread_proxy
> @ 0x7f4577985609 start_thread
> @ 0x7f457756a133 clone
> @ (nil) (unknown)
> E20260529 18:31:07.874303 4116682 catalog-server.cc:431]
> IllegalStateException: java.lang.IllegalStateException
> CAUSED BY: IllegalStateException: null
> W20260529 18:31:08.317953 4114257 MetastoreEventsProcessor.java:1348] Event
> processing is skipped since status is PAUSED. Last synced event id is 3609368
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
