[
https://issues.apache.org/jira/browse/HADOOP-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753299#action_12753299
]
Alex Loddengaard commented on HADOOP-6248:
------------------------------------------
h1. Overview
This proposal defines a system testing framework for Hadoop, named Circus.
Circus will let Hadoop testers and users run one or more tests on a Hadoop
"context." A context might be a local pseudo-distributed cluster, an EC2
cluster, two Hadoop clusters-each running a different version of Hadoop-or
anything else. A test might be the canned pi example, HDFS manipulations, an
HDFS upgrade, custom MapReduce code that is compiled and checked for validity,
a distcp across two different versions of Hadoop, or anything else. This
proposal will provide more examples of contexts and jobs in its later sections.
h1. System Testing vs. Unit Testing
One might wonder how Circus is different from a unit testing framework such as
JUnit. A unit testing framework is meant to run short-lived tests, one after
another. Unit tests, by definition, are meant to test small, independent pieces
of a larger entity. A system test, in general, is larger than a unit test, in
that a system test might run for several hours, analyze logs, run on a large
cluster of machines, or test the runtime and performance of a job. We will see
more use cases later.
h1. Motivation and Use Cases
# Hadoop users want to know if their MapReduce jobs are compatible with new
Hadoop releases. They should have a tool that lets them easily spin up a
cluster running a particular version of Hadoop and compile and run MapReduce
jobs on that cluster. Ideally users will put their MR jobs in Circus and put
Circus in their QA pipeline. Then, when considering an upgrade, they can make
minimal changes to test those same jobs in a newer release of Hadoop.
# Hadoop users want to know how their MapReduce jobs will perform in a new
Hadoop release.
# Hadoop QA engineers want to run complicated tests such as HDFS upgrades,
distcp jobs between two different versions of HDFS, etc. Testers want these
tests to all be part of the same framework so a suite of regression tests can
be run with a single command.
# Hadoop QA engineers want to continually be running Hadoop jobs to ensure API
compatibility between versions. Circus could power a community-driven website,
where community members upload Hadoop jobs and sample data that help generate
reports on backwards compatibility, both at the API level and at the runtime
level.
h1. Expected Usage
I imagine that large users of Hadoop can port their MapReduce jobs to work in
this framework, and add Circus to their QA pipeline. They might also configure
the local context to work with their dev cluster. I imagine Hadoop QA
engineers can use Circus to test some of the tricky use cases of Hadoop, such
as cross-version distcp jobs, HDFS upgrades, or anything else. Testing these
use cases will require that the QA engineer write contexts and tests.
h1. Implementation Details
Please see the source code. A README file exists that explains interfaces and
other implementation details.
h1. Open Issues and Questions
# Can this framework be useful to Hadoop developers? And if so, how?
## Should Circus provide hooks for inducing failure in a Hadoop cluster?
## Might a trunk context be useful so developers can spin up a cluster very
quickly from their local SVN checkout?
## Can the local context allow the user to specify a core jar to be used,
instead of a release?
# What other canned contexts would be useful besides the local context?
Probably an EC2 context.
# How can this framework cater to performance testing? Is just providing time
information sufficient?
# Should tests specify the contexts they are supposed to run in? Seems like
tests should at least have the option to. This will be particularly
interesting for tests that need to run on two Hadoop clusters.
# Should the environment variables specified by a context be validated? That
is, should the interface for contexts be more rigid?
# Should Circus provide a facility for analyzing the Hadoop logs generated
during the run of a test?
> Circus: Proposal and Preliminary Code for a Hadoop System Testing Framework
> ---------------------------------------------------------------------------
>
> Key: HADOOP-6248
> URL: https://issues.apache.org/jira/browse/HADOOP-6248
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Environment: Python, bash
> Reporter: Alex Loddengaard
>
> This issue contains a proposal and preliminary source code for Circus, a
> Hadoop system testing framework. At a high level, Circus will help Hadoop
> users and QA engineers to run system tests on a configurable Hadoop cluster,
> or distribution of Hadoop. See the comment below for the proposal itself.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
