number of reducers

[jamal sasha]

Hi,

  I wrote a simple map reduce job in hadoop streaming.



I am wondering if I am doing something wrong ..

While number of mappers are projected to be around 1700.. reducers.. just 1?

It’s couple of TB’s worth of data.

What can I do to address this.

Basically mapper looks like this



For line in sys.stdin:

    Print line



Reducer

For line in sys.stdin:

    New_line = process_line(line)

    Print new_line





Thanks