[
https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032962#comment-16032962
]
Steve Loughran commented on HADOOP-13998:
-----------------------------------------
it's time to merge in, as the branch is big enough we should merge in trunk to
the branch again, have everything working nicely; the troublesome SSE-C test
can be skipped and covered in docs/release notes.
Before then, last chance to do refactorings and renamings of options, etc, and
review.
h3. Code
Need to look at code to see if
# perf on non-s3guard codepaths is impacted, e.g. by new requests
# config options look good before freezing their names
# all those places which have TODO in them: are they critical? If not, do they
at least have JIRA coverage somewhere
h3. Docs
# docs: how well do they read for someone who hasn't worked on it (this can
evolve, obviously)
# release notes. Maybe: make clear it's experimental, safe in non-auth, auth is
more dangerous
h3. Testing!
what functional tests have people done?
I'm trying to do some downstream testing in my [spark cloud integration
module|https://github.com/hortonworks-spark/cloud-integration] project, mostly
on committer. But I've just moved the spark SQL Hive test suite
{{org.apache.spark.sql.sources.HadoopFsRelationTest}} & its ORC subclass in so
I can verify that the commit algorithms work there. I'm now trying to switch
over to the inconsistent client to see if I can make it easy to observe
inconsistencies in the classic "legacy" file commit algorithms. (ignoring
Parquet for technical reasons; that'll need a fix in spark itself).
Other logistics
# get some review from [~cnauroth] if he has the time
# submit a single patch to cover the merge
> initial s3guard preview
> -----------------------
>
> Key: HADOOP-13998
> URL: https://issues.apache.org/jira/browse/HADOOP-13998
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
>
> JIRA to link in all the things we think are needed for a preview/merge into
> trunk
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
