, but it still blocks on reduce phase. Because the long job has
taken all the reduce slots (because the long job starts first and its reducers
are started shortly after).
The long job’s reducer won’t finish until all its mappers
have finished. So my short job is still blocked by the long job…. Making
Amar, Thanks for the pointer.
-Original Message-
From: Amar Kamat [mailto:[EMAIL PROTECTED]
Sent: Monday, November 24, 2008 8:43 PM
To: core-user@hadoop.apache.org
Subject: Re: do NOT start reduce task until all mappers are finished
Haijun Cao wrote:
Hi,
I am using 0.18.2
I have seen the opposite case where the maps are shown as 100% done
while there are still some maps running. I have seen this on trunk and
there were some failed/killed tasks.
Amar
Andreas Kostyrka wrote:
On Wednesday 09 July 2008 05:56:28 Amar Kamat wrote:
Andreas Kostyrka wrote:
To speed up the overall map operation time, the last few map tasks are
sent to multiple machines. The machine that finishes first wins and
that block is passed onto the reduce phase while the other map tasks
are killed and their results ignored.
-Daniel
On Wed, Jul 16, 2008 at 9:47 AM, Amar
On Wednesday 09 July 2008 05:56:28 Amar Kamat wrote:
Andreas Kostyrka wrote:
See attached screenshot, wonder how that could happen?
What Hadoop version are you using? Is this reproducible? Is it possible
to get the JT logs?
Hadoop 0.17.0
Reproducible: As such no. I did notice that
