[ http://issues.apache.org/jira/browse/HADOOP-619?page=comments#action_12455648 ] Sanjay Dahiya commented on HADOOP-619: --------------------------------------
Runping: The patterns you mentioned work, basically it breaks the input path in parts and matches files/directories at each step going forward(without recursing) so it should be ok with these types of patterns. >The list of matched files > should be part of the job's status so that the user can examine them > through web UI. The list of input paths will be a part of jobclient so it will be available through the JobTracker UI anyways. Does this sound ok ? > Unify Map-Reduce and Streaming to take the same globbed input specification > --------------------------------------------------------------------------- > > Key: HADOOP-619 > URL: http://issues.apache.org/jira/browse/HADOOP-619 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Reporter: eric baldeschwieler > Assigned To: Sanjay Dahiya > > Right now streaming input is specified very differently from other map-reduce > input. It would be good if these two apps could take much more similar input > specs. > In particular -input in streaming expects a file or glob pattern while MR > takes a directory. It would be cool if both could take a glob patern of > files and if both took a directory by default (with some patern excluded to > allow logs, metadata and other framework output to be safely stored). > We want to be sure that MR input is backward compatible over this change. I > propose that a single file should be accepted as an input or a single > directory. Globs should only match directories if the paterns is '/' > terminated, to avoid massive inputs specified by mistake. > Thoughts? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
