How about using the line number as the key and the string at that line as value.

-------
Please excuse typos and brief nature of this email sent from my mobile device

On Oct 20, 2010, at 9:07 PM, Shi Yu <[email protected]> wrote:

> Hi,
> 
> I have a problem of comparing two huge files (100G each) consist of string 
> sequence. It is more like the file text compare problem. I would like to find 
> out how many strings are different within these two files in the natural 
> order. Can this task be modeled as a map/reduce job? Currently I have no idea 
> how to control the split of map and make sure the two input threads in one 
> map task are pointing to the same positions in the files.
> 
> 
> Shi

Reply via email to