[
https://issues.apache.org/jira/browse/IMPALA-15051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venugopal Reddy K reassigned IMPALA-15051:
------------------------------------------
Assignee: Venugopal Reddy K
> INVALIDATE METADATA fails with java.lang.IllegalStateException in
> TableEventExecutor.clear with case-sensitive rename
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15051
> URL: https://issues.apache.org/jira/browse/IMPALA-15051
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Venugopal Reddy K
> Assignee: Venugopal Reddy K
> Priority: Major
>
>
> *[Description]*
> The Impala Catalog service crashes during an Invalidate metadata command due
> to the use of case-sensitive db name in rename table operation. It is treated
> as a new instance even though the mapping of the table name to
> TableEventExecutor already exists. Upon running invalidate metadata, since
> the mapping to the old instance still exists, TableEventExecutor.clear()
> triggers a Google Guava state check failure
> ({{{}Preconditions.checkState{}}}). This occurs after a table is renamed to a
> case-variant database name (e.g., changing from lowercase to uppercase or
> vice versa).
> Exception is thrown at:
> [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java#L440]
> *[Reproduction steps]*
> Execute the python test sequence below:
> {code:java}
> @CustomClusterTestSuite.with_args(
> catalogd_args="--hms_event_polling_interval_s=0.2"
> " --enable_hierarchical_event_processing=true")
> def test_case_sensitive_rename(self, unique_database):
> t1 = "t1"
> t2 = "t2"
> self.execute_query("create table {}.{} (i int)".format(unique_database,
> t1))
> self.execute_query("alter table {}.{} rename to {}.{}"
> .format(unique_database, t1,unique_database, t2))
> self.execute_query("alter table {}.{} rename to {}.{}"
> .format(unique_database, t2, unique_database.upper(),
> t1.upper()))
> EventProcessorUtils.wait_for_event_processing(self)
> self.execute_query("invalidate metadata")
> assert EventProcessorUtils.get_event_processor_status() == "ACTIVE"{code}
> *[Crash Logs]*
> {code:java}
> I20260529 08:01:44.534435 3797781 JniUtil.java:181]
> e64db92811da7b34:e680dd3700000000] resetMetadata request: INVALIDATE ALL
> issued by kvenureddy
> I20260529 08:01:44.536720 3797781 CatalogServiceCatalog.java:2450]
> e64db92811da7b34:e680dd3700000000] Invalidating all metadata. Version: 2174,
> IsCatalogServerRequest: false
> I20260529 08:01:44.537449 3797781 MetastoreEventsProcessor.java:1050]
> e64db92811da7b34:e680dd3700000000] Event processing is paused. Greatest
> synced event id is 3609262
> I20260529 08:01:44.537541 3797781 TableEventExecutor.java:443]
> e64db92811da7b34:e680dd3700000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-0
> I20260529 08:01:44.537559 3797781 TableEventExecutor.java:443]
> e64db92811da7b34:e680dd3700000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-1
> I20260529 08:01:44.537571 3797781 TableEventExecutor.java:443]
> e64db92811da7b34:e680dd3700000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-2
> I20260529 08:01:44.537583 3797781 TableEventExecutor.java:443]
> e64db92811da7b34:e680dd3700000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-3
> I20260529 08:01:44.537596 3797781 TableEventExecutor.java:443]
> e64db92811da7b34:e680dd3700000000] Cleared executor:
> DbEventExecutor-2.TableEventExecutor-4
> I20260529 08:01:44.537608 3797781 DbEventExecutor.java:622]
> e64db92811da7b34:e680dd3700000000] Cleared executor: DbEventExecutor-2
> I20260529 08:01:44.537642 3797653 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-0
> I20260529 08:01:44.537665 3797653 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-1
> I20260529 08:01:44.537675 3797653 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-2
> I20260529 08:01:44.537684 3797653 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-3
> I20260529 08:01:44.537693 3797653 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-1.TableEventExecutor-4
> I20260529 08:01:44.537720 3797653 DbEventExecutor.java:622] Cleared executor:
> DbEventExecutor-1
> I20260529 08:01:44.537763 3797655 DbEventExecutor.java:760] Unassigned
> executor: DbEventExecutor-0.TableEventExecutor-2 for table:
> TEST_CASE_SENSITIVE_RENAME_89F3C8C4.t1
> I20260529 08:01:44.537801 3797652 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-0
> I20260529 08:01:44.537840 3797654 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-4.TableEventExecutor-0
> I20260529 08:01:44.537863 3797652 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-1
> I20260529 08:01:44.537873 3797652 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-2
> I20260529 08:01:44.537881 3797652 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-3
> I20260529 08:01:44.537889 3797652 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-3.TableEventExecutor-4
> I20260529 08:01:44.537899 3797652 DbEventExecutor.java:622] Cleared executor:
> DbEventExecutor-3
> I20260529 08:01:44.537923 3797655 DbEventExecutor.java:760] Unassigned
> executor: DbEventExecutor-0.TableEventExecutor-1 for table:
> test_case_sensitive_rename_89f3c8c4.t2
> I20260529 08:01:44.537954 3797654 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-4.TableEventExecutor-1
> I20260529 08:01:44.537988 3797655 DbEventExecutor.java:804] Unassigned
> executor: DbEventExecutor-0 for db: test_case_sensitive_rename_89f3c8c4
> I20260529 08:01:44.538018 3797654 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-4.TableEventExecutor-2
> I20260529 08:01:44.538043 3797654 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-4.TableEventExecutor-3
> I20260529 08:01:44.538066 3797654 TableEventExecutor.java:443] Cleared
> executor: DbEventExecutor-4.TableEventExecutor-4
> I20260529 08:01:44.538087 3797654 DbEventExecutor.java:622] Cleared executor:
> DbEventExecutor-4
> E20260529 08:01:44.538468 3797781 JniUtil.java:198]
> e64db92811da7b34:e680dd3700000000] Error in INVALIDATE ALL issued by
> kvenureddy. Time spent: 4ms
> I20260529 08:01:44.538808 3797781 jni-util.cc:335]
> e64db92811da7b34:e680dd3700000000] java.lang.IllegalStateException:
> java.lang.IllegalStateException
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
> at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
> at
> java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
> at
> java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
> at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
> at
> java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
> at
> java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
> at
> org.apache.impala.catalog.events.EventExecutorService.clear(EventExecutorService.java:195)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.clear(MetastoreEventsProcessor.java:1210)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:2475)
> at
> org.apache.impala.service.CatalogOpExecutor.execResetMetadataImpl(CatalogOpExecutor.java:7591)
> at
> org.apache.impala.service.CatalogOpExecutor.execResetMetadata(CatalogOpExecutor.java:7407)
> at
> org.apache.impala.service.JniCatalog.lambda$resetMetadata$4(JniCatalog.java:369)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
> at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:253)
> at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:267)
> at org.apache.impala.service.JniCatalog.resetMetadata(JniCatalog.java:368)
> Caused by: java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.catalog.events.TableEventExecutor.clear(TableEventExecutor.java:440)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
> at
> org.apache.impala.catalog.events.DbEventExecutor.clearInternal(DbEventExecutor.java:612)
> at
> org.apache.impala.catalog.events.DbEventExecutor.clear(DbEventExecutor.java:621)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
> at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
> at
> java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
> at
> java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
> at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
> at
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
> at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
> at
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
> at
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
> I20260529 08:01:44.538830 3797781 status.cc:129]
> e64db92811da7b34:e680dd3700000000] IllegalStateException:
> java.lang.IllegalStateException
> CAUSED BY: IllegalStateException: null
> @ 0x1367053 impala::Status::Status()
> @ 0x1cdff17 impala::JniUtil::GetJniExceptionMsg()
> @ 0x12e2c3f impala::JniCall::Call<>()
> @ 0x12b07a0 impala::Catalog::ResetMetadata()
> @ 0x132959a impala::CatalogServiceThriftIf::ResetMetadata()
> @ 0x1270e51
> impala::CatalogServiceProcessorT<>::process_ResetMetadata()
> @ 0x129db63 impala::CatalogServiceProcessorT<>::dispatchCall()
> @ 0x11e60c5 apache::thrift::TDispatchProcessor::process()
> @ 0x16a6df5
> apache::thrift::server::TAcceptQueueServer::Task::run()
> @ 0x16930a5 impala::ThriftThread::RunRunnable()
> @ 0x16940dc
> boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x1dc2ea8 boost::function0<>::operator()()
> @ 0x1dc1d84 impala::Thread::SuperviseThread()
> @ 0x1dc27e5 boost::detail::thread_data<>::run()
> @ 0x291f967 thread_proxy
> @ 0x7f689ac9d609 start_thread
> @ 0x7f689a882133 clone
> @ (nil) (unknown)
> E20260529 08:01:44.555306 3797781 catalog-server.cc:431]
> IllegalStateException: java.lang.IllegalStateException
> CAUSED BY: IllegalStateException: null
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
