Re: ConfigKeys wrappers for MapReduce source code base
Hi Jay, In trunk, these keys are in MRJobConfig (extended by JobContext) and MRConfig. I'm not sure what the state of things are in branch-1. Agreed that it would be nice for the naming convention to be consistent with HDFS, but personally I like *Config more than *ConfigKeys better because the class holds default values as well. -Sandy On Sun, Apr 21, 2013 at 12:29 PM, Jay Vyas jayunit...@gmail.com wrote: Hi guys: A breif check wiht find ./ -name *ConfigKeys* doesn't seem to indicate that there is a MapRedConfigKeys class... Should there be one to help get rid of magic numbers and unify the namespace? This seems to be the goal of the DFSConfigKeys class in the HDFS source tree. ... Or are there differences in the way configuration values are handled in mapred versus hdfs code bases ? For example : job.getInt(JobContext.IO_SORT_FACTOR, 100) in the ReduceTask class would more typically be implemented (if in hdfs) using the DFSConfigKeys static class which stores defaults and configuration parameter names. Just curious wether there is any goal to take the Mapred configuration values and unify their namespace in mapred/common in the same way that seems to have been done in hdfs using the DFSConfigKeys class. -- Jay Vyas http://jayunit100.blogspot.com
MAPREDUCE-5069
Hi, I just wanted to ping the committers again for your review of the patch for MAPREDUCE-5069. I would greatly appreciate it if you could take a look at the patch, and let me know if you have any feedback or accept the patch. On a related note, do I need to generate a separate patch for branch-2 for it to get accepted onto branch-2? Or is the trunk patch sufficient? Thanks in advance! Regards, Sangjin
Re: MAPREDUCE-5069
On Mon, Apr 22, 2013 at 10:51AM, Sangjin Lee wrote: Hi, I just wanted to ping the committers again for your review of the patch for MAPREDUCE-5069. I would greatly appreciate it if you could take a look at the patch, and let me know if you have any feedback or accept the patch. On a related note, do I need to generate a separate patch for branch-2 for it to get accepted onto branch-2? Or is the trunk patch sufficient? Thanks in advance! If trunk patch cleanly applies on branch-2 then you don't need a separate patch. Otherwise, please generate a separate one. Cos
[jira] [Created] (MAPREDUCE-5171) Expose info about blacklisted nodes from MR AM REST API
Sandy Ryza created MAPREDUCE-5171: - Summary: Expose info about blacklisted nodes from MR AM REST API Key: MAPREDUCE-5171 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5171 Project: Hadoop Map/Reduce Issue Type: Bug Components: applicationmaster Affects Versions: 2.0.3-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: MAPREDUCE-5069
Thanks for the clarification. Sangjin On Mon, Apr 22, 2013 at 11:38 AM, Konstantin Boudnik c...@apache.org wrote: On Mon, Apr 22, 2013 at 10:51AM, Sangjin Lee wrote: Hi, I just wanted to ping the committers again for your review of the patch for MAPREDUCE-5069. I would greatly appreciate it if you could take a look at the patch, and let me know if you have any feedback or accept the patch. On a related note, do I need to generate a separate patch for branch-2 for it to get accepted onto branch-2? Or is the trunk patch sufficient? Thanks in advance! If trunk patch cleanly applies on branch-2 then you don't need a separate patch. Otherwise, please generate a separate one. Cos
[jira] [Created] (MAPREDUCE-5172) Expose number of non-local maps in MR app metrics
Sandy Ryza created MAPREDUCE-5172: - Summary: Expose number of non-local maps in MR app metrics Key: MAPREDUCE-5172 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5172 Project: Hadoop Map/Reduce Issue Type: Improvement Components: applicationmaster Affects Versions: 2.0.3-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5173) Need to specify master principal run local job in a secure setup
Sandy Ryza created MAPREDUCE-5173: - Summary: Need to specify master principal run local job in a secure setup Key: MAPREDUCE-5173 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5173 Project: Hadoop Map/Reduce Issue Type: Bug Components: client, security Affects Versions: 2.0.3-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza When security is turned on, running a job involves asking the filesystems it uses for delegation tokens. Currently, this fails if the kerberos master principal is not set, even if the job is run against a filesystem that doesn't use delegation tokens -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
