I think the accepted pattern for this is to accumulate your top N and bottom N values while you reduce and then output them in the close() call. The files from your config can be obtained during the configure() call.
Jeff -----Original Message----- From: Jimmy Wan [mailto:[EMAIL PROTECTED] Sent: Thursday, March 06, 2008 10:40 AM To: [email protected] Subject: Equivalent of cmdline head or tail? I've got some jobs where I'd like to just pull out the top N or bottom N values. It seems like I can't do this from the map or combine phases (due to not having enough data), but I could aggregate this data during the reduce phase. The problem I have is that I won't know when to actually write them out until I've gone through the entire set, at which point reduce isn't called anymore. It's easy enough to post-process with some combination of sort, head, and tail, but I was wondering if I was missing something obvious. -- Jimmy
