[
https://issues.apache.org/jira/browse/HADOOP-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837968#action_12837968
]
Konstantin Boudnik commented on HADOOP-6332:
--------------------------------------------
@Stephen: the main reason to use code injection is to completely hide testing
handles from any chance of misusing by a stranger. Apparently many of the
contracts (interfaces, APIs) we are interested in a course of testing either
unveil internal states of key Hadoop components or allow to perform
'undesirable' actions such as killing a job, a tasktracker, or a datanode it'd
be unwise to keep them in the A-grade production code. Therefore, code
injection seems to be the right technique for this.
Next version of the patch is coming any minute now. It will be clear that all
interfaces exposed to test are defined statically. Their implementation is
injected though, which shouldn't concern anyone but framework developers.
Now, a particular implementation of injection doesn't really matter. We
could've go with ASM or BCEL for the purpose. It happens that we have readily
available AspectJ providing high-level language capabilities, Eclipse
integration, etc. That explain the choice of the framework.
As for an extra burden for future contributors: instrumentation is used for
internal framework mechanics and shouldn't be exposed to the test developers.
Thus, if one simply want to develop a cluster test she/he can do it from a
vanilla Eclipse without AJDT installed. Or from IDEA (which I personally prefer
and use all the time, except when I need to develop/fix some aspects). Or from
vim (not like I suggest to do it :-)
> Large-scale Automated Test Framework
> ------------------------------------
>
> Key: HADOOP-6332
> URL: https://issues.apache.org/jira/browse/HADOOP-6332
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.21.0
>
> Attachments: 6332.patch, 6332_v1.patch, 6332_v2.patch,
> HADOOP-6332-MR.patch, HADOOP-6332-MR.patch, HADOOP-6332.patch,
> HADOOP-6332.patch
>
>
> Hadoop would benefit from having a large-scale, automated, test-framework.
> This jira is meant to be a master-jira to track relevant work.
> ----
> The proposal is a junit-based, large-scale test framework which would run
> against _real_ clusters.
> There are several pieces we need to achieve this goal:
> # A set of utilities we can use in junit-based tests to work with real,
> large-scale hadoop clusters. E.g. utilities to bring up to deploy, start &
> stop clusters, bring down tasktrackers, datanodes, entire racks of both etc.
> # Enhanced control-ability and inspect-ability of the various components in
> the system e.g. daemons such as namenode, jobtracker should expose their
> data-structures for query/manipulation etc. Tests would be much more relevant
> if we could for e.g. query for specific states of the jobtracker, scheduler
> etc. Clearly these apis should _not_ be part of the production clusters -
> hence the proposal is to use aspectj to weave these new apis to
> debug-deployments.
> ----
> Related note: we should break up our tests into at least 3 categories:
> # src/test/unit -> Real unit tests using mock objects (e.g. HDFS-669 &
> MAPREDUCE-1050).
> # src/test/integration -> Current junit tests with Mini* clusters etc.
> # src/test/system -> HADOOP-6332 and it's children
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
