[
https://issues.apache.org/jira/browse/IMPALA-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16514539#comment-16514539
]
Tianyi Wang edited comment on IMPALA-7169 at 6/16/18 12:10 AM:
---------------------------------------------------------------
We set fs.trash.interval = 1440 so that HDFS supposedly shouldn't do the
checkpointing within a day, which doesn't work. HDFS will do the checkpointing
at the next UNIX time which is a multiple of 1440 * 60. The failed build
happened to run this test at around 17:00:00 so it failed.
was (Author: tianyiwang):
We set fs.trash.interval = 1440 so that HDFS supposedly shouldn't do the
checkpointing within a day.
The build took less than a day but HDFS did it when this test is running.
My current theory is that there are some leftover in the trash in our HDFS
snapshot. The timestamp (which is part of the filename) on those leftover is
older than a day so that HDFS thought more than a day has passed.
> TestHdfsEncryption::()::test_drop_partition_encrypt fails to find file
> ----------------------------------------------------------------------
>
> Key: IMPALA-7169
> URL: https://issues.apache.org/jira/browse/IMPALA-7169
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.13.0
> Reporter: Lars Volker
> Assignee: Tianyi Wang
> Priority: Critical
> Labels: broken-build, flaky
>
> {noformat}
> F
> metadata/test_hdfs_encryption.py::TestHdfsEncryption::()::test_drop_partition_encrypt
> metadata/test_hdfs_encryption.py:202: in test_drop_partition_encrypt
> assert self.hdfs_client.exists(
> E assert <bound method PyWebHdfsClientWithChmod.exists of
> <tests.util.hdfs_util.PyWebHdfsClientWithChmod object at
> 0xba6ee90>>('/user/jenkins/.Trash/Current/test-warehouse/test_encryption_db.db/t1/j=1/j1.txt')
> E + where <bound method PyWebHdfsClientWithChmod.exists of
> <tests.util.hdfs_util.PyWebHdfsClientWithChmod object at 0xba6ee90>> =
> <tests.util.hdfs_util.PyWebHdfsClientWithChmod object at 0xba6ee90>.exists
> E + where <tests.util.hdfs_util.PyWebHdfsClientWithChmod object at
> 0xba6ee90> = <test_hdfs_encryption.TestHdfsEncryption object at
> 0xba6ed50>.hdfs_client
> E + and
> '/user/jenkins/.Trash/Current/test-warehouse/test_encryption_db.db/t1/j=1/j1.txt'
> = <built-in method format of str object at 0x3d6f780>('jenkins',
> 'test_encryption_db')
> E + where <built-in method format of str object at 0x3d6f780> =
> '/user/{0}/.Trash/Current/test-warehouse/{1}.db/t1/j=1/j1.txt'.format
> E + and 'jenkins' = <function getuser at 0xb722a8>()
> E + where <function getuser at 0xb722a8> = getpass.getuser
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
