[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168439#comment-16168439
]
Jason Lowe commented on MAPREDUCE-4980:
---------------------------------------
bq. it's patching code that isn't (last I checked) in branch-2.
I'm confused by that statement. The latest 636K patch applies about 591K of
it as-is on branch-2. Granted I didn't try to build it, but I'm curious what
part makes this patch unapplicable for branch-2. It's interesting to note that
some of the patch conflicts are context issues because JIRAs like HADOOP-14729
and HADOOP-10392 were applicable to branch-2 but not applied there.
There is a _lot_ of whitespace changes in this patch, so I did a comparison of
file sizes both with and without the whitespace changes. If the patch focuses
just on fixing the problems with the tests wrt. making them run parallel, the
patch is around 293K. (I can post it if interested.) That's less than half
the size of the current patch, making it easier to review, easier to port to
other branches, and more durable against other checkins to trunk. If we really
want to clean up whitespace that's seems better done and discussed in a
separate JIRA given the extent of the changes rather than negatively impacting
this one.
Other comments on patch 015:
Commented out code in the following places should be removed:
{code}
@@ -130,8 +135,8 @@ public void testMRTimelineEventHandling() throws Exception {
+ cluster.getApplicationHistoryServer().getPort());
TimelineStore ts = cluster.getApplicationHistoryServer()
.getTimelineStore();
- String localPathRoot = System.getProperty("test.build.data",
- "build/test/data");
+ String localPathRoot = GenericTestUtils.getRandomizedTempPath();
+ //getTempPath(TestMRTimelineEventHandling.class.getSimpleName());
Path inDir = new Path(localPathRoot, "input");
Path outDir = new Path(localPathRoot, "output");
RunningJob job =
@@ -176,9 +181,9 @@ public void testMRTimelineEventHandling() throws Exception {
public void testMRNewTimelineServiceEventHandling() throws Exception {
LOG.info("testMRNewTimelineServiceEventHandling start.");
- String testDir =
- new File("target", getClass().getSimpleName() +
- "-test_dir").getAbsolutePath();
+ String testDir = GenericTestUtils.getRandomizedTestDir().getAbsolutePath();
+ // new File("target", getClass().getSimpleName() +
+ // "-test_dir").getAbsolutePath();
String storageDir =
testDir + File.separator + "timeline_service_data";
{code}
There's some TBD stuff that was added in TestMultipleLevelCaching and TestPipes
should either be addressed or removed if unnecessary, i.e.:
{code}
- mr.waitUntilIdle();
- mr.shutdown();
+ // TBD mr.waitUntilIdle();
+ mr.stop();
and
- mr.waitUntilIdle();
+ //TBD mr.waitUntilIdle();
{code}
I really like your idea of unsetting HADOOP_CONF_DIR in the pom if we can get
away with that for the tests. Will help keep dev environments from
contaminating the unit tests in unexpected ways. Best done as a separate JIRA?
Seems like that would apply to more than just the jobclient tests.
Thanks again for the hard work pushing this forward. It's far from glamorous
work, but it really is appreciated.
> Parallel test execution of hadoop-mapreduce-client-core
> -------------------------------------------------------
>
> Key: MAPREDUCE-4980
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
> Project: Hadoop Map/Reduce
> Issue Type: Test
> Components: test
> Affects Versions: 3.0.0-alpha1
> Reporter: Tsuyoshi Ozawa
> Assignee: Andrey Klochkov
> Attachments: MAPREDUCE-4980.010.patch, MAPREDUCE-4980.011.patch,
> MAPREDUCE-4980.012.patch, MAPREDUCE-4980.013.patch, MAPREDUCE-4980.014.patch,
> MAPREDUCE-4980.015.patch, MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
> MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
> MAPREDUCE-4980--n7.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980--n8.patch,
> MAPREDUCE-4980.patch
>
>
> The maven surefire plugin supports parallel testing feature. By using it, the
> tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
