David Knupp created IMPALA-5330:
-----------------------------------
Summary: Impala tests never use or set a secondary FS, so
TestMultipleFilesystems is always skipped
Key: IMPALA-5330
URL: https://issues.apache.org/jira/browse/IMPALA-5330
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: David Knupp
Priority: Critical
Fix For: Impala 2.9.0
This issue came up while investigating how can set up Impala + ADLS testing on
remote clusters. Since testing Impala with ADLS is analogous to testing Impala
with s3, I used that testing as a starting point.
When we change the TARGET_FILESYSTEM to 's3', the end result is to overwrite
the defaultFS setting in core-site.xml to the s3 location. There's not a point
(or doesn't appear to be a point) where we ever treat s3 as a true secondary
filesystem.
Further more, if we look at our test suite that is supposed to check multiple
filesystems simultaneously, we can see there's a pytest SkipIf decorator set to
skip the test if a secondary FS is not set.
{noformat}
@SkipIf.no_secondary_fs
class TestMultipleFilesystems(ImpalaTestSuite)
{noformat}
>From what I can see by going through old Jenkins console logs, this test never
>gets executed. It's skipped during exhaustive runs with HDFS as the default,
>and is also skipped during s3 and local FS test runs as well.
{noformat}
04:27:57
query_test/test_multiple_filesystems.py::TestMultipleFilesystems::test_multiple_filesystems[exec_option:
{'disable_codegen': False, 'abort_on_error': 1,
'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} |
table_format: text/none] SKIPPED
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
