that's a solution, but, since user may not clearly know whic step will produce tempoary table, i think setting reduce number automatically will improve user experience. I think maybe we can set reduce number as 1/3 mapper number before submitting jobs if one of the job inputs is temporary table.
2016-10-14 18:59 GMT+08:00 David Ortiz <[email protected]>: > You can manually set the reducer number using the conf object among other > things. > > On Fri, Oct 14, 2016, 5:43 AM 陈竞 <[email protected]> wrote: > >> hi, i found that if the pipeline produce temporary table , the reduce >> number of the temporary table whose input table is temporary table become >> to small, since temporary table has no content . >> > -- 陈竞，中科院计算技术研究所，高性能计算机中心 Jing Chen HPCC.ICT.AC China
