Found it. Never knew all those helpful metrics were there in the GUI!
Looks like we are right at the 2x threshold on our spilled records vs map output records ratio. I will play with this this week. Thanks again! -----Original Message----- From: Gabriel Reid [mailto:gabriel.r...@gmail.com] Sent: Tuesday, September 01, 2015 6:43 AM To: user@phoenix.apache.org Subject: Re: Help Tuning CsvBulkImport MapReduce On Tue, Sep 1, 2015 at 11:29 AM, Riesland, Zack <zack.riesl...@sensus.com> wrote: > You say I can find information about spills in the job counters. Are > you talking about “failed” map tasks, or is there something else that > will help me identify spill scenarios? "Spilled records" is a counter that is available at the job level and at individual task level -- you can see it in the Counters view of a job or task in the web interface of the YARN Resource Manager or History Server. This is a counter that will only be included if a task was successful (this is the case for all counters in MapReduce jobs). "Map Output Records" is the other counter that you'll want to compare with this. - Gabriel
