I've already been able to replicate the problem using just two reducers, on a completely fresh table. So it seemed to me when I did that the problem was independent of the number of reducers...
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Stack Sent: Wednesday, September 14, 2011 8:47 AM To: [email protected] Subject: Re: scanner deadlock? On Wed, Sep 14, 2011 at 8:42 AM, Geoff Hendrey <[email protected]> wrote: > 17 MR nodes, 8 reducers per machine = 138 concurrent reducers. > (machines are 12-core, and I've found 8 reducers with 1GB allocated heap to be a happy medium that doesn't freeze out the data nodes or the region servers - or so I think :-). > Are you swapping at all? What if you restored your config. to something sane -- 100 handlers with queue size of 10, default timeout -- with 1/4 of the reducers? What does this MR job do? St.Ack
