Looking at ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java, hive.merge.mapredfiles is effective if there is a reducer for your job. Otherwise you should have set hive.merge.mapfiles to true.
On Sat, Jun 12, 2010 at 11:22 PM, Sammy Yu <[email protected]> wrote: > Hi, > I'm running the latest version of trunk r953172. I'm doing doing a > dynamic partition insert overwrite query which generates a lot of small > files in each of the partition. I was hoping this could be solved by > setting hive.merge.mapredfiles to true. However, it seems like whenever the > job is submitted it is always set to false, thus it doesnt seem to have any > effect. I also tried to modified this property in the hive-default.xml, but > it didn't work either. > > Thanks, > Sammy > > >
