If I understand correctly the problem, one possible way to configure the job conf before launching the job to a given cluster could be:
PigContext pigContext = new PigContext(PigServer.ExecType.MAPREDUCE); PigServer pigServer = new PigServer(pigContext); Properties props = new Properties(); // // set specific values in props... // ... // set these properties in the job conf. They will be present in the job conf used when MR is executed pigContext.getExecutionEngine().getConfiguration().putAll(props); // register queries with the Pig Server that will trigger MR jobs(s) -----Original Message----- From: Benjamin Francisoud (JIRA) [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 05, 2008 8:20 AM To: [email protected] Subject: [jira] Created: (PIG-93) Impossible to set jobconf parameters Impossible to set jobconf parameters ------------------------------------ Key: PIG-93 URL: https://issues.apache.org/jira/browse/PIG-93 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.1.0 Reporter: Benjamin Francisoud Priority: Critical I'm trying to set jobconf parameter before launching a pig job using pig api. I tried 2 different ways but with no success: {code:java} PigContext pigContext = new PigContext(ExecType.MAPREDUCE); pigContext.getExecutionEngine().getConfiguration().putAll(properties); PigServer pigServer = new PigServer(pigContext); .... {code} Throw a NPE because the internal executionEngine var is initialize only when calling connect(). So I tried: {code:java} PigContext pigContext = new PigContext(ExecType.MAPREDUCE); pigContext.connect(); pigContext.getExecutionEngine().getConfiguration().putAll(properties); PigServer pigServer = new PigServer(pigContext); ... {code} My properties have been replace with a "new JobConf()" {noformat} java.lang.RuntimeException: Bad mapred.job.tracker: local at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:711) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExe cutionEngine.java:149) at org.apache.pig.impl.PigContext.connect(PigContext.java:180) {noformat} "properties" contains "mapred.job.tracker" and "hadoop.tmp.dir values" Before PIG-32 I use to do (and it was working): {code:java} PigContext pigContext = new PigContext(ExecType.MAPREDUCE); pigContext.setConf(myJobConf); PigServer pigServer = new PigServer(pigContext); ... {code} Any idea before I start to work on a patch ? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
