Hi Prashant, I needed a way to control the hadoop configuration based on the deployed environment. I found a work-around by putting empty core-site.xml in the classpath and providing properties in the constructor. Thanks for your help! -Rakesh
> Date: Mon, 6 Feb 2012 16:27:03 -0800 > Subject: Re: Setting HDFS configuration in PigServer > From: [email protected] > To: [email protected] > > I think I jumped the gun here, I was under the impression you want to > override certain properties. > > In MapReduce mode, Pig always looks for "hadoop-site.xml" or > "core-site.xml". I am not aware of anyway to bypass this. > > Curious, why do you want to not provide the config files? > > Thanks, > Prashant > > On Mon, Feb 6, 2012 at 4:17 PM, rakesh sharma > <[email protected]>wrote: > > > > > Hi Prashant, > > I tried it before posting the question. Here is the exception that I get > > when "new PigServer(ExecType.MAPREDUCE, properties)" gets executed. Note > > that properties has the value of property "fs.default.name": > > org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Cannot > > find hadoop configurations in classpath (neither hadoop-site.xml nor > > core-site.xml was found in the classpath).If you plan to use local mode, > > please put -x local option in command line at > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:169) > > at > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134) > > at org.apache.pig.impl.PigContext.connect(PigContext.java:183) > > at org.apache.pig.PigServer.<init>(PigServer.java:226) at > > org.apache.pig.PigServer.<init>(PigServer.java:215) at > > org.apache.pig.PigServer.<init>(PigServer.java:211) > > Any suggestions? > > Thanks,Rakesh > > > > > Date: Mon, 6 Feb 2012 15:00:24 -0800 > > > Subject: Re: Setting HDFS configuration in PigServer > > > From: [email protected] > > > To: [email protected] > > > > > > Rakesh, > > > > > > You could use > > > > > http://pig.apache.org/docs/r0.9.1/api/org/apache/pig/PigServer.html#PigServer%28org.apache.pig.ExecType,%20java.util.Properties%29 > > > > > > You can specify your own properties via this constructor. > > > > > > Something like, > > > > > > Properties properties = PropertiesUtil.loadDefaultProperties(); //If > > > you want to load defaults > > > properties.set("foo","bar"); > > > PigServer pigServer = new PigServer(ExecType.MAPREDUCE, properties); > > > > > > > > > Thanks, > > > Prashant > > > > > > On Mon, Feb 6, 2012 at 2:56 PM, rakesh sharma > > > <[email protected]>wrote: > > > > > > > > > > > Hi All, > > > > I am using PigServer to execute pig scripts. It picks up the HDFS > > > > configuration from hadoop configuration files residing in the > > classpath. I > > > > am wondering if there is a way to pass these values to PigServer and > > > > eliminate the need of having HDFS configuration files. > > > > Thanks,Rakesh > > > >
