[
https://issues.apache.org/jira/browse/GIRAPH-822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Yost updated GIRAPH-822:
-----------------------------
Description:
Hi Everyone,
I've run into this really weird error with the following stacktrace:
Exception in thread "main" java.lang.ClassCastException:
org.apache.giraph.conf.GiraphConfiguration cannot be cast to
org.apache.hadoop.mapred.JobConf
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at
com.prototype.analytics.TwitterSpringRunner.run(TwitterSpringRunner.java:58)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
com.prototype.analytics.TwitterSpringRunner.main(TwitterSpringRunner.java:78)
Everything was working fine with my locally-built giraph version, but when I
added the Giraph 1.0.0 maven dependency, this Exception was thrown. Any ideas?
UPDATE:
When I revert back to using GiraphRunner, the job completes successfully, so I
must have dorked up something along the way within my Tool class.
Also, this ClassCastException is thrown when I make the following method
invocation:
job.getInternalJob().waitForCompletion(false);
When I remove this, the job starts, but no tasks are executed
Here's the run method implementation:
@Override
public int run(String[] parameters) throws Exception
{
String jobName = parameters[0];
String zConfig = parameters[1];
String inputPath = parameters[2];
String outputPath = parameters[3];
String seedVertex = parameters[4];
int numWorkers = Integer.parseInt(parameters[5]);
GiraphConfiguration gConf = new GiraphConfiguration(getConf());
gConf.set("mapred.job.tracker","localhost:9001");
gConf.set("fs.default.name","hdfs://localhost:9000");
gConf.setVertexClass(com.johnyost.analytics.io.TwitterVertex.class);
gConf.setMasterComputeClass(TwitterCompute.class);
gConf.setZooKeeperConfiguration(zConfig);
gConf.setWorkerConfiguration(numWorkers,numWorkers,100.0f);
gConf.set("seedVertex",seedVertex);
gConf.setVertexInputFormatClass((Class<? extends
VertexInputFormat>) JsonTwitterInputFormat.class);
gConf.setVertexOutputFormatClass((Class<? extends
VertexOutputFormat>)
org.apache.giraph.io.formats.IdWithValueTextOutputFormat.class);
GiraphJob job = new GiraphJob(gConf,jobName);
FileInputFormat.setInputPaths(job.getInternalJob(),new
Path(inputPath));
FileOutputFormat.setOutputPath(job.getInternalJob(),new
Path(outputPath));
GiraphFileInputFormat.setVertexInputPath(gConf,new
Path(inputPath));
log.info(job.getInternalJob().getConfiguration().get("mapred.job.tracker"));
log.info(job.getInternalJob().getConfiguration().get("fs.default.name"));
job.getInternalJob().waitForCompletion(false);
return job.run(true) ? 0 : -1;
}
Any ideas as to why my custom Tool class is failing while GiraphRunner is not?
was:
Hi Everyone,
I've run into this really weird error with the following stacktrace:
Exception in thread "main" java.lang.ClassCastException:
org.apache.giraph.conf.GiraphConfiguration cannot be cast to
org.apache.hadoop.mapred.JobConf
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at
com.prototype.analytics.TwitterSpringRunner.run(TwitterSpringRunner.java:58)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
com.prototype.analytics.TwitterSpringRunner.main(TwitterSpringRunner.java:78)
Everything was working fine with my locally-built giraph version, but when I
added the Giraph 1.0.0 maven dependency, this Exception was thrown. Any ideas?
UPDATE:
When I revert back to using GiraphRunner, the job completes successfully, so I
must have dorked up something along the way within my Tool class. Still would
like to know if anyone else has encountered this and how to fix it, thanks.
> java.lang.ClassCastException: org.apache.giraph.conf.GiraphConfiguration
> cannot be cast to org.apache.hadoop.mapred.JobConf
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: GIRAPH-822
> URL: https://issues.apache.org/jira/browse/GIRAPH-822
> Project: Giraph
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 1.0.0
> Environment: CentOS 6.5
> Reporter: John Yost
>
> Hi Everyone,
> I've run into this really weird error with the following stacktrace:
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.giraph.conf.GiraphConfiguration cannot be cast to
> org.apache.hadoop.mapred.JobConf
> at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
> at
> com.prototype.analytics.TwitterSpringRunner.run(TwitterSpringRunner.java:58)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at
> com.prototype.analytics.TwitterSpringRunner.main(TwitterSpringRunner.java:78)
> Everything was working fine with my locally-built giraph version, but when I
> added the Giraph 1.0.0 maven dependency, this Exception was thrown. Any
> ideas?
> UPDATE:
> When I revert back to using GiraphRunner, the job completes successfully, so
> I must have dorked up something along the way within my Tool class.
> Also, this ClassCastException is thrown when I make the following method
> invocation:
> job.getInternalJob().waitForCompletion(false);
> When I remove this, the job starts, but no tasks are executed
> Here's the run method implementation:
> @Override
> public int run(String[] parameters) throws Exception
> {
> String jobName = parameters[0];
> String zConfig = parameters[1];
> String inputPath = parameters[2];
> String outputPath = parameters[3];
> String seedVertex = parameters[4];
> int numWorkers = Integer.parseInt(parameters[5]);
>
> GiraphConfiguration gConf = new GiraphConfiguration(getConf());
>
> gConf.set("mapred.job.tracker","localhost:9001");
> gConf.set("fs.default.name","hdfs://localhost:9000");
>
> gConf.setVertexClass(com.johnyost.analytics.io.TwitterVertex.class);
> gConf.setMasterComputeClass(TwitterCompute.class);
> gConf.setZooKeeperConfiguration(zConfig);
> gConf.setWorkerConfiguration(numWorkers,numWorkers,100.0f);
> gConf.set("seedVertex",seedVertex);
> gConf.setVertexInputFormatClass((Class<? extends
> VertexInputFormat>) JsonTwitterInputFormat.class);
> gConf.setVertexOutputFormatClass((Class<? extends
> VertexOutputFormat>)
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat.class);
>
> GiraphJob job = new GiraphJob(gConf,jobName);
>
> FileInputFormat.setInputPaths(job.getInternalJob(),new
> Path(inputPath));
> FileOutputFormat.setOutputPath(job.getInternalJob(),new
> Path(outputPath));
>
> GiraphFileInputFormat.setVertexInputPath(gConf,new
> Path(inputPath));
>
>
> log.info(job.getInternalJob().getConfiguration().get("mapred.job.tracker"));
>
> log.info(job.getInternalJob().getConfiguration().get("fs.default.name"));
>
> job.getInternalJob().waitForCompletion(false);
>
> return job.run(true) ? 0 : -1;
> }
> Any ideas as to why my custom Tool class is failing while GiraphRunner is not?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
