Venugopal Reddy K created IMPALA-15051:
------------------------------------------
Summary: INVALIDATE METADATA fails with
java.lang.IllegalStateException in TableEventExecutor.clear with case-sensitive
rename
Key: IMPALA-15051
URL: https://issues.apache.org/jira/browse/IMPALA-15051
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Venugopal Reddy K
*[Description]*
The Impala Catalog service crashes during an Invalidate metadata command due to
the use of case-sensitive db name in rename table operation. It is treated as a
new instance even though the mapping of the table name to TableEventExecutor
already exists. Upon running invalidate metadata, since the mapping to the old
instance still exists, TableEventExecutor.clear() triggers a Google Guava state
check failure ({{{}Preconditions.checkState{}}}). This occurs after a table is
renamed to a case-variant database name (e.g., changing from lowercase to
uppercase or vice versa).
Exception is thrown at:
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/TableEventExecutor.java#L440]
*[Reproduction steps]*
Execute the python test sequence below:
{code:java}
@CustomClusterTestSuite.with_args(
catalogd_args="--hms_event_polling_interval_s=0.2"
" --enable_hierarchical_event_processing=true")
def test_case_sensitive_rename(self, unique_database):
t1 = "t1"
t2 = "t2"
self.execute_query("create table {}.{} (i int)".format(unique_database, t1))
self.execute_query("alter table {}.{} rename to {}.{}"
.format(unique_database, t1,unique_database, t2))
self.execute_query("alter table {}.{} rename to {}.{}"
.format(unique_database, t2, unique_database.upper(),
t1.upper()))
EventProcessorUtils.wait_for_event_processing(self)
self.execute_query("invalidate metadata")
assert EventProcessorUtils.get_event_processor_status() == "ACTIVE"{code}
*[Crash Logs]*
{code:java}
I20260529 08:01:44.534435 3797781 JniUtil.java:181]
e64db92811da7b34:e680dd3700000000] resetMetadata request: INVALIDATE ALL issued
by kvenureddy
I20260529 08:01:44.536720 3797781 CatalogServiceCatalog.java:2450]
e64db92811da7b34:e680dd3700000000] Invalidating all metadata. Version: 2174,
IsCatalogServerRequest: false
I20260529 08:01:44.537449 3797781 MetastoreEventsProcessor.java:1050]
e64db92811da7b34:e680dd3700000000] Event processing is paused. Greatest synced
event id is 3609262
I20260529 08:01:44.537541 3797781 TableEventExecutor.java:443]
e64db92811da7b34:e680dd3700000000] Cleared executor:
DbEventExecutor-2.TableEventExecutor-0
I20260529 08:01:44.537559 3797781 TableEventExecutor.java:443]
e64db92811da7b34:e680dd3700000000] Cleared executor:
DbEventExecutor-2.TableEventExecutor-1
I20260529 08:01:44.537571 3797781 TableEventExecutor.java:443]
e64db92811da7b34:e680dd3700000000] Cleared executor:
DbEventExecutor-2.TableEventExecutor-2
I20260529 08:01:44.537583 3797781 TableEventExecutor.java:443]
e64db92811da7b34:e680dd3700000000] Cleared executor:
DbEventExecutor-2.TableEventExecutor-3
I20260529 08:01:44.537596 3797781 TableEventExecutor.java:443]
e64db92811da7b34:e680dd3700000000] Cleared executor:
DbEventExecutor-2.TableEventExecutor-4
I20260529 08:01:44.537608 3797781 DbEventExecutor.java:622]
e64db92811da7b34:e680dd3700000000] Cleared executor: DbEventExecutor-2
I20260529 08:01:44.537642 3797653 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-1.TableEventExecutor-0
I20260529 08:01:44.537665 3797653 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-1.TableEventExecutor-1
I20260529 08:01:44.537675 3797653 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-1.TableEventExecutor-2
I20260529 08:01:44.537684 3797653 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-1.TableEventExecutor-3
I20260529 08:01:44.537693 3797653 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-1.TableEventExecutor-4
I20260529 08:01:44.537720 3797653 DbEventExecutor.java:622] Cleared executor:
DbEventExecutor-1
I20260529 08:01:44.537763 3797655 DbEventExecutor.java:760] Unassigned
executor: DbEventExecutor-0.TableEventExecutor-2 for table:
TEST_CASE_SENSITIVE_RENAME_89F3C8C4.t1
I20260529 08:01:44.537801 3797652 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-3.TableEventExecutor-0
I20260529 08:01:44.537840 3797654 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-4.TableEventExecutor-0
I20260529 08:01:44.537863 3797652 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-3.TableEventExecutor-1
I20260529 08:01:44.537873 3797652 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-3.TableEventExecutor-2
I20260529 08:01:44.537881 3797652 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-3.TableEventExecutor-3
I20260529 08:01:44.537889 3797652 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-3.TableEventExecutor-4
I20260529 08:01:44.537899 3797652 DbEventExecutor.java:622] Cleared executor:
DbEventExecutor-3
I20260529 08:01:44.537923 3797655 DbEventExecutor.java:760] Unassigned
executor: DbEventExecutor-0.TableEventExecutor-1 for table:
test_case_sensitive_rename_89f3c8c4.t2
I20260529 08:01:44.537954 3797654 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-4.TableEventExecutor-1
I20260529 08:01:44.537988 3797655 DbEventExecutor.java:804] Unassigned
executor: DbEventExecutor-0 for db: test_case_sensitive_rename_89f3c8c4
I20260529 08:01:44.538018 3797654 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-4.TableEventExecutor-2
I20260529 08:01:44.538043 3797654 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-4.TableEventExecutor-3
I20260529 08:01:44.538066 3797654 TableEventExecutor.java:443] Cleared
executor: DbEventExecutor-4.TableEventExecutor-4
I20260529 08:01:44.538087 3797654 DbEventExecutor.java:622] Cleared executor:
DbEventExecutor-4
E20260529 08:01:44.538468 3797781 JniUtil.java:198]
e64db92811da7b34:e680dd3700000000] Error in INVALIDATE ALL issued by
kvenureddy. Time spent: 4ms
I20260529 08:01:44.538808 3797781 jni-util.cc:335]
e64db92811da7b34:e680dd3700000000] java.lang.IllegalStateException:
java.lang.IllegalStateException
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
at
java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:562)
at
java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:591)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:689)
at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at
java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
at
org.apache.impala.catalog.events.EventExecutorService.clear(EventExecutorService.java:195)
at
org.apache.impala.catalog.events.MetastoreEventsProcessor.clear(MetastoreEventsProcessor.java:1210)
at
org.apache.impala.catalog.CatalogServiceCatalog.reset(CatalogServiceCatalog.java:2475)
at
org.apache.impala.service.CatalogOpExecutor.execResetMetadataImpl(CatalogOpExecutor.java:7591)
at
org.apache.impala.service.CatalogOpExecutor.execResetMetadata(CatalogOpExecutor.java:7407)
at
org.apache.impala.service.JniCatalog.lambda$resetMetadata$4(JniCatalog.java:369)
at
org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:253)
at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:267)
at org.apache.impala.service.JniCatalog.resetMetadata(JniCatalog.java:368)
Caused by: java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:496)
at
org.apache.impala.catalog.events.TableEventExecutor.clear(TableEventExecutor.java:440)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at
org.apache.impala.catalog.events.DbEventExecutor.clearInternal(DbEventExecutor.java:612)
at
org.apache.impala.catalog.events.DbEventExecutor.clear(DbEventExecutor.java:621)
at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at
java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at
java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
I20260529 08:01:44.538830 3797781 status.cc:129]
e64db92811da7b34:e680dd3700000000] IllegalStateException:
java.lang.IllegalStateException
CAUSED BY: IllegalStateException: null
@ 0x1367053 impala::Status::Status()
@ 0x1cdff17 impala::JniUtil::GetJniExceptionMsg()
@ 0x12e2c3f impala::JniCall::Call<>()
@ 0x12b07a0 impala::Catalog::ResetMetadata()
@ 0x132959a impala::CatalogServiceThriftIf::ResetMetadata()
@ 0x1270e51
impala::CatalogServiceProcessorT<>::process_ResetMetadata()
@ 0x129db63 impala::CatalogServiceProcessorT<>::dispatchCall()
@ 0x11e60c5 apache::thrift::TDispatchProcessor::process()
@ 0x16a6df5
apache::thrift::server::TAcceptQueueServer::Task::run()
@ 0x16930a5 impala::ThriftThread::RunRunnable()
@ 0x16940dc
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x1dc2ea8 boost::function0<>::operator()()
@ 0x1dc1d84 impala::Thread::SuperviseThread()
@ 0x1dc27e5 boost::detail::thread_data<>::run()
@ 0x291f967 thread_proxy
@ 0x7f689ac9d609 start_thread
@ 0x7f689a882133 clone
@ (nil) (unknown)
E20260529 08:01:44.555306 3797781 catalog-server.cc:431] IllegalStateException:
java.lang.IllegalStateException
CAUSED BY: IllegalStateException: null
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
