[
https://issues.apache.org/jira/browse/HADOOP-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-15003:
------------------------------------
Attachment: HADOOP-13786-051.patch
patch 051
h3. Test failure from reuse of non-magic FS.
* special check in file creation for __magic path and not magic fs, logs @
info. Warn maybe?
* committer troubleshooting mentions what this warning means
* testing.md lists what we want from good tests (parallelise, etc), and
mentions FS caching
* magic tests disable FS Caching.
we may want to simplify things here by always setting the magic build in our
test filesystems. That way: no need to play these games and no recurrent
problems. Thoughts?
h3. Isolate S3A filesystem from transitive dependencies on hadoop-mapreduce
Isolate all utility classes which use MR datatypes from the others into their
own package, and the magic path ones, splitting CommitUtils in three. This is
needed to ensure s3a loads when mapreduce JARs aren't on the classpath.
The previous patches probably meant S3AFS woudn't load without
hadoop-mapreduce on the CP just because some other static methods in
CommitUtils needed it. Partitioning things up and a full review of the s3
codepath *appears* to fix this. However, I plan to add a new downstream test
module somewhere just to verify classpath setup by opening the landsat
repository and listing it,
> Merge S3A committers into trunk: Yetus patch checker
> ----------------------------------------------------
>
> Key: HADOOP-15003
> URL: https://issues.apache.org/jira/browse/HADOOP-15003
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13786-041.patch, HADOOP-13786-042.patch,
> HADOOP-13786-043.patch, HADOOP-13786-044.patch, HADOOP-13786-045.patch,
> HADOOP-13786-046.patch, HADOOP-13786-047.patch, HADOOP-13786-048.patch,
> HADOOP-13786-049.patch, HADOOP-13786-050.patch, HADOOP-13786-051.patch,
> HADOOP-15033-testfix-1.diff
>
>
> This is a Yetus only JIRA created to have Yetus review the
> HADOOP-13786/HADOOP-14971 patch as a .patch file, as the review PR
> [https://github.com/apache/hadoop/pull/282] is stopping this happening in
> HADOOP-14971.
> Reviews should go into the PR/other task
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
