[
https://issues.apache.org/jira/browse/HADOOP-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883813#comment-16883813
]
Steve Loughran commented on HADOOP-16415:
-----------------------------------------
Thinking for of this
We're running the same operation (MR or terasort sequence) with different
cluster configs, which is exactly what parameterized test runs can do. So we
just need some parameterization which declares everything a specific test run
can do:
* config options
* extra callbacks on validation
* expected outcomes
then we have a test which brings up the mini yarn cluster in static setup,
destroys it in teardown, and has the sets run parameterized
The only thing we'd need to do is implement a significantly more complex
parameterization than normal, with each one being a class declaring all that is
needed for each one. Ideally, one which we could share between the Terasort and
the TestMRJob tests
> Speed up S3A test runs
> ----------------------
>
> Key: HADOOP-16415
> URL: https://issues.apache.org/jira/browse/HADOOP-16415
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Major
>
> S3A Test runs are way too slow.
> Speed them by
> * reducing test setup/teardown costs
> * eliminating obsolete test cases
> * merge small tests into larger ones.
> One thing i see is that the main S3A test cases create and destroy new FS
> instances; There's both a setup and teardown cost there, but it does
> guarantee better isolation.
> Maybe if we know all test cases in a specific suite need the same options, we
> can manage that better; demand create the FS but only delete it in an
> @Afterclass method. That'd give us the OO-inheritance based setup of tests,
> but mean only one instance is done per suite
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
