[
https://issues.apache.org/jira/browse/HADOOP-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Fabbri updated HADOOP-15003:
----------------------------------
Attachment: HADOOP-15033-testfix-1.diff
(Attaching a small patch.)
Busy day today but I think I have a fix for the parallel test failure. What I
did:
- Instrumented test class configuration constructors/getters.
- Dumped a stack trace on {{S3AFileSystem#initialize()}}.
- Added the isMagicCommitEnabled bit to {{S3AFileSystem#toString()}} (included
in patch).
- Reproduced the failure.
What I saw:
- Beginning part of huge scale test, where the large file is created, was using
some existing {{S3AFileSystem}} instance with {{isMagicCommitEnabled}} false,
despite us doing the work to set up a {{Configuration}} with the option set to
true.
- Second part of huge test (post create assertions) that was failing on
asserting that the .pending file was there *was* getting a new S3AFileSystem
with the magic commit flag enabled, but it didn't matter since the creation did
not have it enabled.
Seems like the initial FS is coming from the FileSystem cache so I disabled
that (patch included) and re-ran and my reproduction vanished. Logging shows
that both test phases (create, post assert) are getting a fresh FileSystem with
the magic commit flag enabled now.
> Merge S3A committers into trunk: Yetus patch checker
> ----------------------------------------------------
>
> Key: HADOOP-15003
> URL: https://issues.apache.org/jira/browse/HADOOP-15003
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13786-041.patch, HADOOP-13786-042.patch,
> HADOOP-13786-043.patch, HADOOP-13786-044.patch, HADOOP-13786-045.patch,
> HADOOP-13786-046.patch, HADOOP-13786-047.patch, HADOOP-13786-048.patch,
> HADOOP-13786-049.patch, HADOOP-13786-050.patch, HADOOP-15033-testfix-1.diff
>
>
> This is a Yetus only JIRA created to have Yetus review the
> HADOOP-13786/HADOOP-14971 patch as a .patch file, as the review PR
> [https://github.com/apache/hadoop/pull/282] is stopping this happening in
> HADOOP-14971.
> Reviews should go into the PR/other task
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
