Weird, I'll create a simple main file to repoduce it.
Antonio Magnaghi a écrit :
From the trace you have attached below it looks like the connection to the job
tracker on the cluster failed, before getting to the point to set job specific
properties and before launching a job to the cluster.
I tried out what I suggested yesterday by setting some custom keys after
creating a pig server and was able to see such keys in the job conf shipped to
the cluster.
Can you please attach a complete code sample that reproduces the problem so
that I can reproduce the issue and further debug it?
Thanks,
-a.
-----Original Message-----
From: Benjamin Francisoud [mailto:[EMAIL PROTECTED]
Sent: Wednesday, February 06, 2008 12:49 AM
To: [email protected]
Subject: Re: [jira] Created: (PIG-93) Impossible to set jobconf parameters
Antonio Magnaghi a écrit :
If I understand correctly the problem, one possible way to configure the
job conf before launching the job to a given cluster could be:
PigContext pigContext = new PigContext(PigServer.ExecType.MAPREDUCE);
PigServer pigServer = new PigServer(pigContext);
Properties props = new Properties();
//
// set specific values in props...
// ...
// set these properties in the job conf. They will be present in the job
conf used when MR is executed
pigContext.getExecutionEngine().getConfiguration().putAll(props);
// register queries with the Pig Server that will trigger MR jobs(s)
Yes you understood correctly :)
There is a slight difference between your example above and the ones I
putted in PIG-93.
You are creating the pigServer object using pigContext object before
setting the properties in pigContext with putAll() method.
But it doesn't seem to solve the problem:
java.lang.RuntimeException: Bad mapred.job.tracker: local
at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:711)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:149)
at org.apache.pig.impl.PigContext.connect(PigContext.java:180)
at org.apache.pig.PigServer.<init>(PigServer.java:123)
-----Original Message-----
From: Benjamin Francisoud (JIRA) [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 05, 2008 8:20 AM
To: [email protected]
Subject: [jira] Created: (PIG-93) Impossible to set jobconf parameters
Impossible to set jobconf parameters
------------------------------------
Key: PIG-93
URL: https://issues.apache.org/jira/browse/PIG-93
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.1.0
Reporter: Benjamin Francisoud
Priority: Critical
