The problem was that my custom built UDAF with subqueries caused all UDAF logics to run only on reducers. I created a jira relating to this (HIVE-6230<https://issues.apache.org/jira/browse/HIVE-6230> )
*Benjamin Kim* *benkimkimben at gmail* On Tue, Jan 14, 2014 at 3:25 PM, Ben Kim <[email protected]> wrote: > Hello! > > When i run hive job, it creates large temporary files on > > /tmp/hive-ben/hive_2014-01-14_15-01-39_521_3015861149916225685-1/ > > somewhere around 300GB. this number tends to get larger if i use lower > number of reducers. > > with 5 reducers the size goes up to 1TB > > my input files are total 1GB, but it's distributed with large number of > keys on MR. > > > what are these temp files for? > > Thanks! > > *Benjamin Kim* > *benkimkimben at gmail* >
