On Tue, Aug 10, 2010 at 8:13 AM, lei liu <[email protected]> wrote: > Thank you for your reply. > > Could you tell me why it is slower if the two paremeters are true and how > slow it is? > > 2010/8/10 Namit Jain <[email protected]> >> >> Yes, it will try to run another map-reduce job to merge the files >> ________________________________________ >> From: lei liu [[email protected]] >> Sent: Monday, August 09, 2010 8:57 AM >> To: [email protected] >> Subject: Re: How to merge small files >> >> Could you tell me whether the query is slower if I two parameters both are >> true? >> >> 2010/8/9 Namit Jain <[email protected]<mailto:[email protected]>> >> That's right >> >> ________________________________________ >> From: lei liu [[email protected]<mailto:[email protected]>] >> Sent: Sunday, August 08, 2010 7:18 PM >> To: [email protected]<mailto:[email protected]> >> Subject: Re: How to merge small files >> >> Thank you for your reply. >> >> Your mean is I will execute below statement: >> >> statement.execute("set hive.merge.mapfiles=true"); >> statement.execute("set hive.merge.mapredfiles=true"); >> >> The two parementers are both true, right? >> >> 2010/8/6 Namit Jain >> <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> >> HIVEMERGEMAPFILES("hive.merge.mapfiles", true), >> HIVEMERGEMAPREDFILES("hive.merge.mapredfiles", false), >> >> >> Set the above parameters to true before your query. >> >> >> >> ________________________________________ >> From: lei liu >> [[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>] >> Sent: Thursday, August 05, 2010 8:47 PM >> To: >> [email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> >> Subject: How to merge small files >> >> When I run below sql: INSERT OVERWRITE TABLE tablename1 select_statement1 >> FROM from_statement, there are many files which size is zero are stored to >> hadoop, >> >> How can I merge these small files? >> >> Thanks, >> >> >> >> LiuLei >> >> >> > >
How slow it is is relevant to how much data you have. We can not answer questions like that, try it both ways and find out for yourself. Edward
