[ https://issues.apache.org/jira/browse/HIVE-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971987#action_12971987 ]
Ning Zhang commented on HIVE-1806: ---------------------------------- dyn_part_empty.q is having intermittent failures. I tried the clean trunk and it sometimes fails as well. The failure is not critical -- exception was printed to log file rather than console. It should not be caused by this patch. However, I did find 2 diffs in skewjoin.q and bucketmapjoin2.q. The former has a simple fix. I'm looking at why bucketmapjoin2.q has a slightly different plan. > The merge criteria on dynamic partitons should be per partiton > -------------------------------------------------------------- > > Key: HIVE-1806 > URL: https://issues.apache.org/jira/browse/HIVE-1806 > Project: Hive > Issue Type: Bug > Reporter: Ning Zhang > Assignee: Ning Zhang > Attachments: HIVE-1806.patch > > > Currently the criteria of whether a merge job should be fired on dynamic > generated partitions are is the average file size of files across all dynamic > partitions. It is very common that some dynamic partitions contains mostly > large files and some contains mostly small files. Even though the average > size of the total files are larger than the hive.merge.smallfiles.avgsize, we > should merge those partitions containing small files only. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.