[
https://issues.apache.org/jira/browse/PIG-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573665#action_12573665
]
Eugene Kirpichov commented on PIG-93:
-------------------------------------
I am having a similar problem while trying to get started with Pig.
I've successfully installed Hadoop on 2 machines and hadoop's examples finally
worked.
Now, I built Pig and that's what I get when launching it:
<code>
[EMAIL PROTECTED] trunk$ scripts/pig.pl -v -cp
pig.jar:~/hadoop/hadoop-0.16.0/conf
I can't find HOD configuration for , hopefully you weren't planning on using
HOD.
2008-02-29 13:19:22,102 [main] DEBUG org.apache.hadoop.conf.Configuration -
java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:144)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:112)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.<init>(HExecutionEngine.java:88)
at org.apache.pig.impl.PigContext.connect(PigContext.java:179)
at org.apache.pig.PigServer.<init>(PigServer.java:132)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:41)
at org.apache.pig.Main.main(Main.java:247)
2008-02-29 13:19:22,259 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
hadoop file system at: file:///
2008-02-29 13:19:22,263 [main] DEBUG org.apache.hadoop.conf.Configuration -
java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:144)
at
org.apache.pig.backend.hadoop.datastorage.HConfiguration.getConfiguration(HConfiguration.java:38)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:36)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:140)
at org.apache.pig.impl.PigContext.connect(PigContext.java:181)
at org.apache.pig.PigServer.<init>(PigServer.java:132)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:41)
at org.apache.pig.Main.main(Main.java:247)
2008-02-29 13:19:22,418 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
map-reduce job tracker at: local
2008-02-29 13:19:22,428 [main] DEBUG org.apache.hadoop.conf.Configuration -
java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:144)
at
org.apache.pig.backend.hadoop.datastorage.HConfiguration.getConfiguration(HConfiguration.java:38)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:149)
at org.apache.pig.impl.PigContext.connect(PigContext.java:181)
at org.apache.pig.PigServer.<init>(PigServer.java:132)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:41)
at org.apache.pig.Main.main(Main.java:247)
2008-02-29 13:19:22,610 [main] ERROR org.apache.pig.Main -
java.lang.RuntimeException: Bad mapred.job.tracker: local
</code>
I've been grepping the sources for a couple of hours already to find out where
these properties really should have been set to something senseful, but I found
that the HExecutionEngine constructor is actually invoked in its 1-argument
version, where it creates an 'empty' JobConf() and does not seem to be going to
read my hadoop-site.conf.
I am pretty sure I made a mistake somewhere, because it can't be that pig
simply doesn't work at all from command line, and I'd be much pleased to be
pointed at this mistake :)
It would be even better if you had a from-scratch description like 'How to
install pig and hadoop and launch a simple example on two machines' in the wiki.
> Impossible to set jobconf parameters
> ------------------------------------
>
> Key: PIG-93
> URL: https://issues.apache.org/jira/browse/PIG-93
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.1.0
> Reporter: Benjamin Francisoud
> Priority: Critical
> Attachments: PIG93Main.java
>
>
> I'm trying to set jobconf parameter before launching a pig job using pig api.
> I tried 2 different ways but with no success:
> {code:java}
> PigContext pigContext = new PigContext(ExecType.MAPREDUCE);
> pigContext.getExecutionEngine().getConfiguration().putAll(properties);
> PigServer pigServer = new PigServer(pigContext);
> ....
> {code}
> Throw a NPE because the internal executionEngine var is initialize only when
> calling connect().
> So I tried:
> {code:java}
> PigContext pigContext = new PigContext(ExecType.MAPREDUCE);
> pigContext.connect();
> pigContext.getExecutionEngine().getConfiguration().putAll(properties);
> PigServer pigServer = new PigServer(pigContext);
> ...
> {code}
> My properties have been replace with a "new JobConf()"
> {noformat}
> java.lang.RuntimeException: Bad mapred.job.tracker: local
> at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:711)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:149)
> at org.apache.pig.impl.PigContext.connect(PigContext.java:180)
> {noformat}
> "properties" contains "mapred.job.tracker" and "hadoop.tmp.dir values"
> Before PIG-32 I use to do (and it was working):
> {code:java}
> PigContext pigContext = new PigContext(ExecType.MAPREDUCE);
> pigContext.setConf(myJobConf);
> PigServer pigServer = new PigServer(pigContext);
> ...
> {code}
> Any idea before I start to work on a patch ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
