I've got some jobs where I'd like to just pull out the top N or bottom N values.

It seems like I can't do this from the map or combine phases (due to not having enough data), but I could aggregate this data during the reduce phase. The problem I have is that I won't know when to actually write them out until I've gone through the entire set, at which point reduce isn't called anymore.

It's easy enough to post-process with some combination of sort, head, and tail, but I was wondering if I was missing something obvious.

--
Jimmy

Reply via email to