Yes, it will try to run another map-reduce job to merge the files ________________________________________ From: lei liu [[email protected]] Sent: Monday, August 09, 2010 8:57 AM To: [email protected] Subject: Re: How to merge small files
Could you tell me whether the query is slower if I two parameters both are true? 2010/8/9 Namit Jain <[email protected]<mailto:[email protected]>> That's right ________________________________________ From: lei liu [[email protected]<mailto:[email protected]>] Sent: Sunday, August 08, 2010 7:18 PM To: [email protected]<mailto:[email protected]> Subject: Re: How to merge small files Thank you for your reply. Your mean is I will execute below statement: statement.execute("set hive.merge.mapfiles=true"); statement.execute("set hive.merge.mapredfiles=true"); The two parementers are both true, right? 2010/8/6 Namit Jain <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> HIVEMERGEMAPFILES("hive.merge.mapfiles", true), HIVEMERGEMAPREDFILES("hive.merge.mapredfiles", false), Set the above parameters to true before your query. ________________________________________ From: lei liu [[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>] Sent: Thursday, August 05, 2010 8:47 PM To: [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> Subject: How to merge small files When I run below sql: INSERT OVERWRITE TABLE tablename1 select_statement1 FROM from_statement, there are many files which size is zero are stored to hadoop, How can I merge these small files? Thanks, LiuLei
