[
https://issues.apache.org/jira/browse/IMPALA-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803377#comment-17803377
]
ASF subversion and git services commented on IMPALA-10987:
----------------------------------------------------------
Commit 624b6c646819d7a15d550169ceca4ce5a3fa794f in impala's branch
refs/heads/master from Sai Hemanth Gantasala
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=624b6c646 ]
IMPALA-10987: (Addendum) Changing impala.disableHmsSync in Hive should not
break event processing
This patch removes the extra quote '"' in the
test_events_custom_config.py file. This ensures that tests are not
broken with different versions of Hive.
Change-Id: I1d29bf0bdf68d4da11f02dabff8c6a68d81276c8
Reviewed-on: http://gerrit.cloudera.org:8080/20860
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Michael Smith <[email protected]>
> Changing impala.disableHmsSync in Hive can break event processing
> -----------------------------------------------------------------
>
> Key: IMPALA-10987
> URL: https://issues.apache.org/jira/browse/IMPALA-10987
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Csaba Ringhofer
> Assignee: Sai Hemanth Gantasala
> Priority: Critical
> Fix For: Impala 4.4.0
>
>
> To reproduce, start Impala with event polling:
> {code}
> bin/start-impala-cluster.py --catalogd_args="--hms_event_polling_interval_s=2
> --catalog_topic_mode=minimal" --impalad_args="--use_local_catalog=1"
> {code}
> From Hive:
> {code}
> CREATE DATABASE temp;
> CREATE EXTERNAL TABLE temp.t (i int) PARTITIONED BY (p int)
> TBLPROPERTIES('impala.disableHmsSync'='true');
> ALTER TABLE temp.t SET TBLPROPERTIES ('impala.disableHmsSync'='false');
> {code}
> From this point event sync will be broken in Impala. It can be fixed only
> with global INVALIDATE METADATA (or restarting catalogd)
> catalogd log will include an exception like this:
> {code}
> E1026 10:30:16.151208 22514 MetastoreEventsProcessor.java:653] Event
> processing needs a invalidate command to resolve the state
> Java exception follows:
> org.apache.impala.catalog.events.MetastoreNotificationNeedsInvalidateException:
> EventId: 15956 EventType: ALTER_TABLE Detected that event sync was tur
> ned on for the table temp.t and the table does not exist. Event processing
> cannot be continued further. Issue a invalidate metadata command to reset
> the event processing state
> at
> org.apache.impala.catalog.events.MetastoreEvents$AlterTableEvent.process(MetastoreEvents.java:992)
> at
> org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:345)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:747)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:645)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> {code}
> and future events will be lead to a log like this:
> {code}
> W1026 10:30:18.151962 22514 MetastoreEventsProcessor.java:638] Event
> processing is skipped since status is NEEDS_INVALIDATE. Last synced event id
> is 15955
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
