Sahil Takiar created IMPALA-9004:
------------------------------------
Summary: TestCompressedFormats is broken for text files
Key: IMPALA-9004
URL: https://issues.apache.org/jira/browse/IMPALA-9004
Project: IMPALA
Issue Type: Test
Reporter: Sahil Takiar
While working on IMPALA-8950, we made a fix to {{TestCompressedFormats}} so
that it actually checks the exit status of the {{hdfs dfs -cp}} command, turns
out that this command has been silently failing whenever
{{test_compressed_formats}} runs with {{file_format}} = {{text}}.
For some reason, data load writes compressed text files with their
corresponding file compression suffix, but for compressed seq/rc files, it does
not:
{code:java}
hdfs dfs -ls /test-warehouse/tinytable_seq_*
Found 1 items
-rwxr-xr-x 3 systest supergroup 325 2019-08-22 14:32
/test-warehouse/tinytable_seq_bzip/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 215 2019-08-22 14:32
/test-warehouse/tinytable_seq_def/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 260 2019-08-22 14:32
/test-warehouse/tinytable_seq_gzip/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 301 2019-08-22 14:32
/test-warehouse/tinytable_seq_record_bzip/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 209 2019-08-22 14:32
/test-warehouse/tinytable_seq_record_def/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 242 2019-08-22 14:32
/test-warehouse/tinytable_seq_record_gzip/000000_0
Found 1 items
-rwxr-xr-x 3 systest supergroup 233 2019-08-22 14:32
/test-warehouse/tinytable_seq_record_snap/000000_0
Found 2 items
-rwxr-xr-x 3 systest supergroup 243 2019-08-22 14:32
/test-warehouse/tinytable_seq_snap/000000_0
hdfs dfs -ls /test-warehouse/tinytable_text_*
Found 1 items
-rwxr-xr-x 3 systest supergroup 59 2019-08-22 14:32
/test-warehouse/tinytable_text_bzip/000000_0.bz2
Found 1 items
-rwxr-xr-x 3 systest supergroup 28 2019-08-22 14:32
/test-warehouse/tinytable_text_def/000000_0.deflate
Found 1 items
-rwxr-xr-x 3 systest supergroup 40 2019-08-22 14:32
/test-warehouse/tinytable_text_gzip/000000_0.gz
Found 2 items
-rwxr-xr-x 3 systest supergroup 87 2019-08-22 14:32
/test-warehouse/tinytable_text_lzo/000000_0.lzo
-rw-r--r-- 3 systest supergroup 8 2019-08-22 14:42
/test-warehouse/tinytable_text_lzo/000000_0.lzo.index
Found 1 items
-rwxr-xr-x 3 systest supergroup 41 2019-08-22 14:32
/test-warehouse/tinytable_text_snap/000000_0.snappy{code}
Not sure if that is by design or not, but it is causing the tests to fail for
all text files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
