Mihaly Szjatinya created IMPALA-15254:
-----------------------------------------
Summary: test_enc_dec_gen_compress in test_charcodec.py fails with
AnalysisException when HMS event processor is slow
Key: IMPALA-15254
URL: https://issues.apache.org/jira/browse/IMPALA-15254
Project: IMPALA
Issue Type: Bug
Reporter: Mihaly Szjatinya
Assignee: Mihaly Szjatinya
`prepare_encoded_test_table_compress()` in `tests/query_test/test_charcodec.py`
creates a table and loads data via Hive (`run_stmt_in_hive`) but never calls
`INVALIDATE METADATA` or `REFRESH` afterward. Normally Impala picks up
Hive-created tables automatically via the HMS event processor. However, when
the event processor is lagging or has crashed (e.g. during long exhaustive runs
on resource-constrained environments), the asynchronous propagation may not
complete before `_compare_tables` queries the table, causing:
{code}
AnalysisException: Could not resolve table reference: '<db>.<charset>_gen_None'
{code}
All other {{prepare_*}} helpers in the same file explicitly sync the catalog
after Hive writes:
- {{prepare_utf8_test_table}} -- calls REFRESH (line 117)
- {{prepare_encoded_test_table}} -- calls REFRESH (line 129)
- {{prepare_utf8_test_table_partitions}} -- calls REFRESH (line 223)
- {{prepare_encoded_test_table_partitions}} -- calls REFRESH (line 240)
{{prepare_encoded_test_table_compress}} (line 173) is the only one that omits
this step. This is a test-reliability bug, not a product bug.
*Fix:* Add {{INVALIDATE METADATA {db}.{encoded_table}}} (or {{REFRESH}}) at the
end of `prepare_encoded_test_table_compress`, consistent with the other helpers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
