Hello,
I have a JobControl with an unknown number of jobs. The way I execute
it is add a job, start it and upon finnishing
decide whether to add another job.
When I add another a job and run it,I get the following(see below)
What does the "Unknown Source" mean? Does it have anything do with
"08/11/16 16:11:16 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
with processName=JobTracker, sessionId= - already initialized "
08/11/16 16:16:41 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
[output of first job (job_1) skipped....]
08/11/16 16:16:51 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics
with processName=JobTracker, sessionId= - already initialized
08/11/16 16:16:51 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to
process : 1
08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to
process : 1
08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to
process : 1
08/11/16 16:16:51 INFO mapred.FileInputFormat: Total input paths to
process : 1
08/11/16 16:16:51 INFO mapred.MapTask: numReduceTasks: 1
08/11/16 16:16:51 INFO mapred.MapTask: io.sort.mb = 200
08/11/16 16:16:52 INFO mapred.MapTask: data buffer =
159383552/199229440
08/11/16 16:16:52 INFO mapred.MapTask: record buffer = 524288/655360
08/11/16 16:16:52 WARN mapred.LocalJobRunner: job_local_0002
java.lang.NullPointerException
at org.saptarshiguha.clusters.ClusterCenter
$ClosestCenterMR.map(Unknown Source)
at org.saptarshiguha.clusters.ClusterCenter
$ClosestCenterMR.map(Unknown Source)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.LocalJobRunner
$Job.run(LocalJobRunner.java:157)
08/11/16 16:16:57 INFO mapred.LocalJobRunner: file:/tmp/input/
sample.data:0+308
The map function is not run at all.
===BEGIN_CODE====
JobConf jobConf_1 = ColumnRanges.createSetupJob(inPaths_1, outdir);
Job job_1 = new Job(jobConf_1, null);
JobControl theControl = new JobControl("Test");
theControl.addJob(job_1);
Thread theController = new Thread(theControl);
theController.start();
while (!theControl.allFinished()) {
System.out.println("Jobs in waiting state: " +
theControl.getWaitingJobs().size());
System.out.println("Jobs in ready state: " +
theControl.getReadyJobs().size());
System.out.println("Jobs in running state: " +
theControl.getRunningJobs().size());
System.out.println("Jobs in success state: " +
theControl.getSuccessfulJobs().size());
System.out.println("Jobs in failed state: " +
theControl.getFailedJobs().size());
System.out.println("\n");
try {
Thread.sleep(5000);
} catch (Exception e) {
System.out.println("Exception\n");
}
}
theControl.stop();
if ( job_1.getState() == Job.SUCCESS) {
System.out.println("Choosing initial centers");
createSeedCenters();
FileUtil.fullyDelete(outdir.getFileSystem(jobConf_1 ),outdir);
JobConf jobConf = ClusterCenter.createSetupJob(inPaths_1, outdir);
Job job = new Job(jobConf, null);
theControl.addJob(job);
Thread threadControl = new Thread(theControl);
threadControl.start();
while (!theControl.allFinished()){
try {
Thread.sleep(5000);
} catch (Exception e) {
System.out.println("Exception\n");
}
}
}
