[jira] Commented: (MAPREDUCE-1154) Large-scale, automated test framwork for Map-Reduce

[Arun C Murthy (JIRA)]

    [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769762#action_12769762
 ] 

Arun C Murthy commented on MAPREDUCE-1154:
------------------------------------------

ClusterTest.java:

{noformat}
public class ClusterTest extends TestCase {

  Cluster cluster;
  Configuration conf;
  
  @Override
  protected void setUp() throws Exception {
    conf = new Configuration();
    ClusterTestUtils.setupCluster(conf);
    cluster = new Cluster(conf);
  }

  @Test
  public void testCluster() throws Exception {
    // Load data
    DataLoader dataLoader = new DistcpDataLoader();
    List<Path> inputs = new ArrayList<Path>();
    inputs.add(new Path("in"));
    dataLoader.load(inputs, new Path("out"));
    
    // Setup job
    Job job = Job.getInstance(cluster);
    job.setJobName("cluster-test-mr-job");

    // Run job
    long start = System.currentTimeMillis();
    job.submit();
    job.waitForCompletion(true);
    long end = System.currentTimeMillis();
    
    FileContext fc = FileContext.getLocalFSFileContext();

    // Fetch daemon logs and validate them
    Path daemonLogs = new Path("daemon-logs");
    assertFalse("JobTracker logs contain FATAL warnings!", 
                MRClusterTestUtils.checkDaemonLogs(
                    cluster, LogSource.JOBTRACKER, start, end, 
                    "FATAL", true, daemonLogs) == false);
    assertFalse("TaskTracker logs contain FATAL warnings!", 
                MRClusterTestUtils.checkDaemonLogs(
                    cluster, LogSource.TASKTRACKER, start, end, 
                    "FATAL", true, daemonLogs) == false);
    fc.delete(daemonLogs, true);
    
    // Fetch job logs, outputs etc. and validate them
    Path jobLogs = new Path("daemon-logs");
    Path jobOutputs = new Path("daemon-logs");
    TaskAttemptID m0 = null; // map 0
    TaskAttemptID m1 = null; // map 1
    TaskAttemptID r0 = null; // reduce 0
    assertFalse("Task log contains FATAL warnings!", 
                MRClusterTestUtils.checkJobArtifacts(job, 
                    Artifact.LOGS, "FATAL", 
                    true, jobLogs) == false);
    assertFalse("Task log contains WARN warnings!", 
                MRClusterTestUtils.checkJobArtifacts(
                    job, Artifact.LOGS, "WARN", 
                    false, jobLogs, m0, m1) == false);
    assertTrue("Task outputs do not contain expected output!", 
               MRClusterTestUtils.checkJobArtifacts(
                   job, Artifact.OUTPUTS, 
                   "hello, world!", true, jobOutputs, r0));
    fc.delete(jobLogs, true);
    fc.delete(jobOutputs, true);
  }
  
  @Override
  protected void tearDown() throws Exception {
    ClusterTestUtils.tearDownCluster(conf);
  }

}
{noformat}

> Large-scale, automated test framwork for Map-Reduce
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-1154
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1154
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: test
>            Reporter: Arun C Murthy
>             Fix For: 0.21.0
>
>         Attachments: testing.patch
>
>
> HADOOP-6332 proposes a large-scale, automated, junit-based test-framework for 
> Hadoop.
> This jira is meant to track relevant work to Map-Reduce.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.