[
https://issues.apache.org/jira/browse/HADOOP-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806438#action_12806438
]
Stephen Watt commented on HADOOP-6332:
--------------------------------------
@Sharad/Cos
1) Thanks for the offer to post the patch. FYI, you might want to check the
pathing in the new patch, as the existing ones go back one too many directories
so it cannot find build.xml. Its not too big of an issue as I am running
everything from eclipse at present.
2) Its nice to have the TestCluster sample JUnit Test as a starting point in
using the framework.
3) In the current patch there is a "Cluster" class referenced on line 15 of
JTClient, but not implemented anywhere in the patch.
3) Design: At present, we have the framework for cluster management and we have
the M/R JobTracker Client. As to how someone would use this to run system tests
on Hadoop... is the intention that one write their own Test Case as a normal
Hadoop Job (such as TeraSort) as a separate activity and then one would get a
handle to the MRCluster in the @before method, and then start the test by
calling Job.submit() in the @test method and then be able to pass the jobID
back to the JTClient to do whatever you needed to with it at that point ?
> Large-scale Automated Test Framework
> ------------------------------------
>
> Key: HADOOP-6332
> URL: https://issues.apache.org/jira/browse/HADOOP-6332
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.21.0
>
> Attachments: 6332_v1.patch, 6332_v2.patch, HADOOP-6332-MR.patch,
> HADOOP-6332-MR.patch, HADOOP-6332.patch, HADOOP-6332.patch
>
>
> Hadoop would benefit from having a large-scale, automated, test-framework.
> This jira is meant to be a master-jira to track relevant work.
> ----
> The proposal is a junit-based, large-scale test framework which would run
> against _real_ clusters.
> There are several pieces we need to achieve this goal:
> # A set of utilities we can use in junit-based tests to work with real,
> large-scale hadoop clusters. E.g. utilities to bring up to deploy, start &
> stop clusters, bring down tasktrackers, datanodes, entire racks of both etc.
> # Enhanced control-ability and inspect-ability of the various components in
> the system e.g. daemons such as namenode, jobtracker should expose their
> data-structures for query/manipulation etc. Tests would be much more relevant
> if we could for e.g. query for specific states of the jobtracker, scheduler
> etc. Clearly these apis should _not_ be part of the production clusters -
> hence the proposal is to use aspectj to weave these new apis to
> debug-deployments.
> ----
> Related note: we should break up our tests into at least 3 categories:
> # src/test/unit -> Real unit tests using mock objects (e.g. HDFS-669 &
> MAPREDUCE-1050).
> # src/test/integration -> Current junit tests with Mini* clusters etc.
> # src/test/system -> HADOOP-6332 and it's children
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
