[
https://issues.apache.org/jira/browse/IMPALA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17205081#comment-17205081
]
ASF subversion and git services commented on IMPALA-10062:
----------------------------------------------------------
Commit 63bf92a2d1e60f440ec1bf827d81e7e87de47d12 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=63bf92a ]
IMPALA-10062: TestCompressedNonText.test_insensitivity_to_extension can fail
due to wrong filename
Some tests in test_compressed_formats.py hard-coded the filename
of the tables. They used "000000_0" for filename. The number after the
underscore is the "attempt id" which can be non-zero if there were
failed attempts during file writing.
I modified the test to do a filesystem listing to retrieve the
filename.
Testing
* I manually renamed one of my files to 000000_1 and re-run the test.
Change-Id: I265faf8d2e7f4251b18264052eededbeb2296f57
Reviewed-on: http://gerrit.cloudera.org:8080/16518
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestCompressedNonText.test_insensitivity_to_extension can fail due to wrong
> filename
> ------------------------------------------------------------------------------------
>
> Key: IMPALA-10062
> URL: https://issues.apache.org/jira/browse/IMPALA-10062
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Priority: Critical
> Labels: broken-build, flaky
>
> The fix for IMPALA-10005 added a new TestCompressedNonText test. It relies on
> Hive generating specific file names when writing these compressed tables
> (i.e. it expects a file named 000000_0). It looks like that is not guaranteed
> by dataload, which can lead to failures like this:
> {noformat}
> query_test/test_compressed_formats.py:142: in test_insensitivity_to_extension
> unique_database, 'tinytable', db_suffix, '000000_0', src_extension, ext)
> query_test/test_compressed_formats.py:86: in _copy_and_query_compressed_file
> self.filesystem_client.copy(src_file, dest_file, overwrite=True)
> util/hdfs_util.py:79: in copy
> self.hdfs_filesystem_client.copy(src, dst, overwrite)
> util/hdfs_util.py:241: in copy
> '{0} copy failed: '.format(self.filesystem_type) + stderr + "; " + stdout
> E AssertionError: HDFS copy failed: cp:
> `/test-warehouse/tinytable_avro_snap/000000_0': No such file or directory
> E ;{noformat}
> The file list shows that the filename is actually
> "/test-warehouse/tinytable_avro_snap/000000_1"
> We should update the test to tolerate this. The actual base filename doesn't
> matter for this test.
> I have seen this exactly once so far.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
