[
https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131213#comment-16131213
]
Thomas Marquardt edited comment on HADOOP-14553 at 8/17/17 8:32 PM:
--------------------------------------------------------------------
It is great to see the work that has been done here! We can now run the tests
in half the time or less! This is a *huge* improvement to the engineering
process! Thanks for enabling the tests to be run in parallel.
Here are my comments on the change:
1) *mvn test* must run all the tests. This behavior should not change. I
think this should be the standard command to run tests for a project. Also,
people will not be aware of the change we are making, so please add new
arguments to mvn but do not change existing behavior.
2) The Results summary is no longer consolidated. For example, when you run
*mvn -T 1C -Dparallel-tests clean verify*, at the end it looks like only a few
tests were run, but if you scroll up in the console output you see that there
were several runs and reports. Lets summarize the results in a single line at
the end.
3) The tests currently marked scale are not *all* scale tests. Several are
functional tests, like those in ITestBlockBlobInputStream.java. It is
important for these tests to be run prior to each check-in. Unless you have
added new scale tests (I did not look at the history of every scale test), all
of the tests need to be run before check-in when we run *mvn test*.
4) It took me 12 minutes to run *mvn -T 1C clean verify -Dscale*. Two things:
i) the Results summary looks almost identical to the output from *mvn -T 1C
clean verify* except the latter had 156 skipped tests in the 2nd set of
results. ii) it looks like it is running the same tests that are run when you
don't include -Dscale:
*mvn -T 1C clean verify -Dscale*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 120
*mvn -T 1C clean verify*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 156
5) Before this patch, the test results summary read *Tests run 775, Failures:
0, Errors: 0, Skipped: 155*. It appears that some tests were removed, as the
total Tests run is no longer 775. Here are the results I had for different
commands:
*mvn test*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Total time: 01:23 min
*mvn -T 1C clean verify*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 156
Total time: 21:36 min
*mvn -T 1C -Dparallel-tests clean verify*
Tests run: 213, Failures: 0, Errors: 0, Skipped: 35
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Tests run: 323, Failures: 0, Errors: 0, Skipped: 98
Tests run: 150, Failures: 0, Errors: 0, Skipped: 58
Total time: 07:12 min
*mvn -T 1C clean verify -Dscale*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 120
Total time: 12:14 min
I realize this is a big patch and we want to commit it, as it is expensive to
refresh. As long as we can fix 1) and determine why in 5) the total tests run
has been reduced from the former 775 tests, we can commit this and continue
working on it.
Thanks!
was (Author: tmarquardt):
It is great to see the work that has been done here! We can now run the tests
in half the time or less! This is a *huge* improvement to the engineering
process! Thanks for enabling the tests to be run in parallel.
Here are my comments on the change:
1) *mvn test* must run all the tests. This behavior should not change. I
think this should be the standard command to run tests for a project. Also,
people will not be aware of the change we are making, so please add new
arguments to mvn but do not change existing behavior.
2) The Results summary is no longer consolidated. For example, when you run
*mvn -T 1C -Dparallel-tests clean verify*, at the end it looks like only a few
tests were run, but if you scroll up in the console output you see that there
were several runs and reports. Lets summarize the results in a single line at
the end.
3) The tests currently marked scale are not *all* scale tests. Several are
functional tests, like those in ITestBlockBlobInputStream.java. It is
important for these tests to be run prior to each check-in. Unless you have
added new scale tests (I did not look at the history of every scale test), all
of the tests need to be run before check-in when we run *mvn test*.
4) It took me 12 minutes to run *mvn -T 1C clean verify -Dscale*. Two things:
i) the Results summary looks almost identical to the output from *mvn -T 1C
clean verify* except the latter had 156 skipped tests in the 2nd set of
results. ii) it looks like it is running the same tests that are run when you
don't include -Dscale:
*mvn -T 1C clean verify -Dscale*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 120
*mvn -T 1C clean verify*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 156
4) Before this patch, the test results summary read *Tests run 775, Failures:
0, Errors: 0, Skipped: 155*. It appears that some tests were removed, as the
total Tests run is no longer 775. Here are the results I had for different
commands:
*mvn test*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Total time: 01:23 min
*mvn -T 1C clean verify*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 156
Total time: 21:36 min
*mvn -T 1C -Dparallel-tests clean verify*
Tests run: 213, Failures: 0, Errors: 0, Skipped: 35
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Tests run: 323, Failures: 0, Errors: 0, Skipped: 98
Tests run: 150, Failures: 0, Errors: 0, Skipped: 58
Total time: 07:12 min
*mvn -T 1C clean verify -Dscale*
Tests run: 214, Failures: 0, Errors: 0, Skipped: 35
Tests run: 519, Failures: 0, Errors: 0, Skipped: 120
Total time: 12:14 min
I realize this is a big patch and we want to commit it, as it is expensive to
refresh. As long as we can fix 1) and determine why the total tests run has
been reduced from the former 775 tests, we can commit this and continue working
on it.
Thanks!
> Add (parallelized) integration tests to hadoop-azure
> ----------------------------------------------------
>
> Key: HADOOP-14553
> URL: https://issues.apache.org/jira/browse/HADOOP-14553
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-14553-001.patch, HADOOP-14553-002.patch,
> HADOOP-14553-003.patch, HADOOP-14553-004.patch, HADOOP-14553-005.patch,
> HADOOP-14553-006.patch, HADOOP-14553-007.patch, HADOOP-14553-008.patch,
> HADOOP-14553-009.patch, HADOOP-14553-010.patch, HADOOP-14553-011.patch,
> HADOOP-14553-012.patch, HADOOP-14553-014.patch
>
>
> The Azure tests are slow to run as they are serialized, as they are all
> called Test* there's no clear differentiation from unit tests which Jenkins
> can run, and integration tests which it can't.
> Move the azure tests {{Test*}} to integration tests {{ITest*}}, parallelize
> (which includes having separate paths for every test suite). The code in
> hadoop-aws's POM &c show what to do.
> *UPDATE August 4, 2017*: Adding a list of requirements to clarify the
> acceptance criteria for this JIRA:
> # Parallelize test execution
> # Define test groups: i) UnitTests - self-contained, executed by Jenkins, ii)
> IntegrationTests - requires Azure Storage account, executed by engineers
> prior to check-in, and if needed, iii) ScaleTests – long running performance
> and scalability tests.
> # Define configuration profiles to run tests with different settings. Allows
> an engineer to run “IntegrationTests” with fs.azure.secure.mode = true and
> false. Need to review settings to see what else would benefit.
> # Maven commands to run b) and c). Turns out it is not easy to do with
> Maven, so we might have to run it multiple times to run with different
> configuration settings.
> # Document how to add and run tests and the process for contributing to
> Apache Hadoop. Steve shared an example at
> https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
>
> # UnitTests should run in under 2 minutes and IntegrationTests should run in
> under 15 minutes, even on slower network connections. (These are rough goals)
> # Ensure test data (containers/blobs/etc) is deleted. Exceptions for large
> persistent content used repeatedly to expedite test execution.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
