We do this in Pig by using our own InputSplits. ben
On Friday 12 October 2007, Owen O'Malley wrote: > On Oct 12, 2007, at 5:51 AM, Shailendra Mudgal wrote: > > I am adding two input dir in a job. Both the input dirs have same > > <Key.class, > > Value.class>. Inside the map method i want to know that which > > pair<key, > > value> has come from which input dir. How can i do this ? Any help > > will be > > appreciated.. > > *sigh* We've _almost_ had that feature for a long time now, see > HADOOP-372. > > The work around is to use the information on: > http://wiki.apache.org/lucene-hadoop/TaskExecutionEnvironment > and get the "map.input.file" from the map's JobConf and match against > the prefix.
