Vihang Karajgaonkar created IMPALA-10598:
--------------------------------------------
Summary: test_cache_reload_validation is flaky
Key: IMPALA-10598
URL: https://issues.apache.org/jira/browse/IMPALA-10598
Project: IMPALA
Issue Type: Test
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
I noticed that when I run
{noformat}
bin/impala-py.test tests/query_test/test_hdfs_caching.py -k
test_cache_reload_validation
{noformat}
I see a the following failure on master branch.
{noformat}
TestHdfsCachingDdl.test_cache_reload_validation[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]
tests/query_test/test_hdfs_caching.py:269: in test_cache_reload_validation
assert num_entries_pre + 4 == get_num_cache_requests(), \
E AssertionError: Adding the tables should be reflected by the number of
cache directives.
E assert (2 + 4) == 7
E + where 7 = get_num_cache_requests()
{noformat}
This failure is reproducible for me every time but I am not sure why the
jenkins job don't show this test failure. When I looked into this I found that
the test depends on the method
get the number of cache directives on the hdfs.
{noformat}
def get_num_cache_requests_util():
rc, stdout, stderr = exec_process("hdfs cacheadmin -listDirectives -stats")
assert rc == 0, 'Error executing hdfs cacheadmin: %s %s' % (stdout, stderr)
return len(stdout.split('\n'))
{noformat}
This output of this command when there are no entries is
{noformat}
Found 0 entries
{noformat}
when there are entries the output looks like
{noformat}
Found 4 entries
ID POOL REPL EXPIRY PATH
BYTES_NEEDED BYTES_CACHED FILES_NEEDED FILES_CACHED
225 testPool 8 never /test-warehouse/cachedb.db/cached_tbl_reload
0 0 0 0
226 testPool 8 never /test-warehouse/cachedb.db/cached_tbl_reload_part
0 0 0 0
227 testPool 8 never
/test-warehouse/cachedb.db/cached_tbl_reload_part/j=1 0
0 0 0
228 testPool 8 never
/test-warehouse/cachedb.db/cached_tbl_reload_part/j=2 0
0 0 0
{noformat}
When there are no entries there is also a additional new line which is counted.
So when there are no entries the method outputs 2 and when there are 4 entries
the method outputs 7 which causes the failure because the test expects 2+4.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
