[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16864212#comment-16864212
]
ASF subversion and git services commented on IMPALA-8567:
---------------------------------------------------------
Commit dec14abbd46467a73d1159508971e04da68fa508 in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=dec14ab ]
IMPALA-8633 : Insert event should not error when table does not exists
When a insert event is received by the events processor it is possible
that the table has already been removed from metastore. In such cases,
the reloadTable method throws a NoSuchObjectException which is
unhandled. This causes EventProcessor to go in NEEDS_INVALIDATE state.
The patch adds exception handling for such cases so that the error can
be ignored and a warning is logged.
Ideally, if the table has been removed from metastore, this insert event
should be followed by a drop-table event and hence the table is cleaned
up from the catalog subsequently. Hence Event processor does not need to
remove the table from the catalog.
Also, the patch adds the exception trace to the error messages during
processing of Insert events to improve debugging ability in case of
such errors.
Testing:
Refactored the existing test for insert events into util methods
which can be reused for other tests. Added a new test case which
generates a insert event on a table which has been dropped. This test
reproduces the scenario seen by debugging test failures in IMPALA-8567.
The newly added test succeeds after the patch.
Change-Id: I961cd7cbede4c248dba538c7fabb4bc708e49693
Reviewed-on: http://gerrit.cloudera.org:8080/13548
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Many random catalog consistency issues with catalog v2/event processor
> ----------------------------------------------------------------------
>
> Key: IMPALA-8567
> URL: https://issues.apache.org/jira/browse/IMPALA-8567
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Vihang Karajgaonkar
> Priority: Critical
> Labels: broken-build, catalog, flaky
>
> [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the
> event processor is likely to blame here so I'll let you look. The general
> theme is tables and databases not existing when they should.
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_option____kudu_read_mode____READ_AT_SNAPSHOT____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_option____sync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_option____batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/
> I'll include the output of each job in a follow-on comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
