[
https://issues.apache.org/jira/browse/PIG-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566853#comment-13566853
]
Cheolsoo Park commented on PIG-3135:
------------------------------------
[~prkommireddi] Overall looks good to me. I have 3 questions.
* Can we call the property "pig.use.overriden.hadoop.configs"? What we're
overriding here are basically Hadoop confs, so I think that a more specific
name is better. Do you agree?
* On a related note, can you update the following comment in
testJobConfGeneration?
From:
{code}
// This should fail as pig expects classpath to be set
{code}
To something like:
{code}
// This should fail as pig expects Hadoop configs are present in classpath.
{code}
* Can you add this new property to conf/pig.properties with some explanation,
so people can know about it? It would be nice if we could mention that this is
a Mr-mode-specific property too. Please let me know if you have a better
suggestion regarding how to document this.
Thanks!
> HExecutionEngine should look for resources in user passed Properties
> --------------------------------------------------------------------
>
> Key: PIG-3135
> URL: https://issues.apache.org/jira/browse/PIG-3135
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.10.0
> Reporter: Prashant Kommireddi
> Assignee: Prashant Kommireddi
> Attachments: PIG-3135.patch
>
>
> Looking at this snippet:
> {code}
> private void init(Properties properties) throws ExecException {
> .
> .
> .
> // Check existence of hadoop-site.xml or core-site.xml
> Configuration testConf = new Configuration();
> ClassLoader cl = testConf.getClassLoader();
> URL hadoop_site = cl.getResource( HADOOP_SITE );
> URL core_site = cl.getResource( CORE_SITE );
>
> if( hadoop_site == null && core_site == null ) {
> throw new ExecException("Cannot find hadoop configurations in
> classpath (neither hadoop-site.xml nor core-site.xml was found in the
> classpath)." +
> " If you plan to use local mode, please put -x local
> option in command line",
> 4010);
> }
> {code}
> This assumes the resources (*-site.xml) are set on the classpath, but this
> will not always be the case when run with Pig's Java APIs. One could want to
> programatically set the resources and the code here should additionally check
> if they are available in there.
> Example: When a Configuration object is created and resources are added
> before passing it on to Pig.
> {code}
> Configuration conf = new Configuration(false);
> conf.addResource("foo/core-site.xml");
> conf.addResource("bar/hadoop-site.xml");
> PigServer pServer = new PigServer(ExecType.MAPREDUCE, conf);
> {code}
> The above conf is not used right now to obtain resources.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
