[
https://issues.apache.org/jira/browse/IMPALA-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihaly Szjatinya updated IMPALA-15254:
--------------------------------------
Description:
`prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py`
creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls
`INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up
Hive-created tables automatically via the HMS event processor. However, when
the event processor is lagging or has crashed (e.g. during long exhaustive runs
on resource-constrained environments), the asynchronous propagation may not
complete before `_compare_tables` queries the table, causing:
{code:java}
AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
{code}
*Fix:* Add INVALIDATE METADATA at the end of
`prepare_encoded_test_table_compress`, consistent with the other helpers.
was:
`prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py`
creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls
`INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up
Hive-created tables automatically via the HMS event processor. However, when
the event processor is lagging or has crashed (e.g. during long exhaustive runs
on resource-constrained environments), the asynchronous propagation may not
complete before `_compare_tables` queries the table, causing:
{code:java}
AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
{code}
*Fix:* Add {{INVALIDATE METADATA
{db}
.
{encoded_table}
}} (or {{{}REFRESH{}}}) at the end of `prepare_encoded_test_table_compress`,
consistent with the other helpers.
> test_enc_dec_gen_compress in test_charcodec.py fails with AnalysisException
> when HMS event processor is slow
> ------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-15254
> URL: https://issues.apache.org/jira/browse/IMPALA-15254
> Project: IMPALA
> Issue Type: Bug
> Reporter: Mihaly Szjatinya
> Assignee: Mihaly Szjatinya
> Priority: Major
> Labels: test-failure
>
> `prepare_encoded_test_table_compress()` in
> `tests/query_test/test_charcodec.py` creates a table and loads data via Hive
> (`run_stmt_in_hive`) but never calls `INVALIDATE METADATA` or `REFRESH`
> afterward. Normally Impala picks up Hive-created tables automatically via the
> HMS event processor. However, when the event processor is lagging or has
> crashed (e.g. during long exhaustive runs on resource-constrained
> environments), the asynchronous propagation may not complete before
> `_compare_tables` queries the table, causing:
> {code:java}
> AnalysisException: Could not resolve table reference:
> '<db>.<charset>_gen_None'
> {code}
> *Fix:* Add INVALIDATE METADATA at the end of
> `prepare_encoded_test_table_compress`, consistent with the other helpers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
