On Sep 29, 2006, at 12:20 AM, Benjamin Reed wrote:
I please correct me if I'm reading the code incorrectly, but it seems
like submitJob puts the submitted job on the jobInitQueue which is
immediately dequeued by the JobInitThread and then initTasks() will
get
the file splits and create Tasks. Thus, it doesn't seem like there is
any difference in memory foot print.
Agreed, it won't cost more memory. In fact, it will be less because
we won't have the init task thread running and creating InputFormats
and running user code. Of course, once we allow user-defined
InputSplits we will be back in exactly the same boat of running user-
code on the JobTracker, unless we also ship over the preferred hosts
for each InputFormat too.
-- Owen
