Steve Loughran created HADOOP-16644: ---------------------------------------
Summary: Intermittent failure of ITestS3ATerasortOnS3A: timestamp differences Key: HADOOP-16644 URL: https://issues.apache.org/jira/browse/HADOOP-16644 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3, test Affects Versions: 3.3.0 Environment: -Dparallel-tests -DtestsThreadCount=8 -Dfailsafe.runOrder=balanced -Ds3guard -Ddynamo -Dscale h2. Hypothesis: the timestamp of the source file is being picked up from S3Guard, but when the NM does a getFileStatus call, a HEAD check is made -and this (due to the overloaded test system) is out of sync with the listing. S3Guard is updated, the corrected date returned and the localisation fails. Reporter: Steve Loughran Terasort of directory committer failing in resource localisaton -the partitions.lst file has a different TS from that expected Happens under loaded integration tests (threads = 8; not standalone); non-auth s3guard {code} 2019-10-08 11:50:29,774 [IPC Server handler 4 on 55983] WARN localizer.ResourceLocalizationService (ResourceLocalizationService.java:processHeartbeat(1150)) - { s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst, 1570531828143, FILE, null } failed: Resource s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst changed on src filesystem (expected 1570531828143, was 1570531828000 java.io.IOException: Resource s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst changed on src filesystem (expected 1570531828143, was 1570531828000 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org