[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854938#comment-16854938
]
Vihang Karajgaonkar commented on IMPALA-8567:
---------------------------------------------
Found a unrelated issue with EventsProcessor. The events processor is in
{{NEEDS_INVALIDATE}} state due to following sequence of events as seen in the
catalogd log. Unfortunately, the cause of the exception is not logged. I will
send out a patch to print the full stack trace so that we can identify what is
going on here.
{code}
I0601 15:22:49.529088 84 MetastoreEventsProcessor.java:466] Received 3 events.
Start event id : 11097
I0601 15:22:49.529270 84 MetastoreEvents.java:378] EventId: 11098 EventType:
CREATE_TABLE Creating event 11098 of type CREATE_TABLE on table
functional.alltypesnopart_insert
I0601 15:22:49.529706 84 MetastoreEvents.java:378] EventId: 11099 EventType:
INSERT Creating event 11099 of type INSERT on table
functional.alltypesnopart_insert
I0601 15:22:49.529960 84 MetastoreEvents.java:378] EventId: 11100 EventType:
DROP_TABLE Creating event 11100 of type DROP_TABLE on table
functional.alltypesnopart_insert
I0601 15:22:49.530210 84 MetastoreEvents.java:367] EventId: 11098 EventType:
CREATE_TABLE Found table alltypesnopart_insert is removed later in event 11100
type DROP_TABLE
I0601 15:22:49.530269 84 MetastoreEvents.java:219] EventId: 11098 EventType:
CREATE_TABLE Filtering out this event since the object is either removed or
renamed later in the event stream
I0601 15:22:49.530330 84 MetastoreEvents.java:227] Total number of events
received: 3 Total number of events filtered out: 1
I0601 15:22:49.530428 84 CatalogServiceCatalog.java:1956] Refreshing table
metadata: functional.alltypesnopart_insert
E0601 15:22:49.532079 84 MetastoreEventsProcessor.java:505] Event processing
needs a invalidate command to resolve the state
Java exception follows:
org.apache.impala.catalog.events.MetastoreNotificationNeedsInvalidateException:
EventId: 11099 EventType: INSERT Refresh table {} failed. Event processing
cannot continue. Issue an invalidate metadata command to reset the event
processor state.
at
org.apache.impala.catalog.events.MetastoreEvents$InsertEvent.processTableInserts(MetastoreEvents.java:823)
at
org.apache.impala.catalog.events.MetastoreEvents$InsertEvent.process(MetastoreEvents.java:764)
at
org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:312)
at
org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:594)
at
org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:497)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
> Many random catalog consistency issues with catalog v2/event processor
> ----------------------------------------------------------------------
>
> Key: IMPALA-8567
> URL: https://issues.apache.org/jira/browse/IMPALA-8567
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Vihang Karajgaonkar
> Priority: Blocker
> Labels: broken-build, catalog, flaky
>
> [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the
> event processor is likely to blame here so I'll let you look. The general
> theme is tables and databases not existing when they should.
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_option____kudu_read_mode____READ_AT_SNAPSHOT____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_option____sync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/
> I'll include the output of each job in a follow-on comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
