Export/Import and # of regions
Hi, If I export existing table using Export MR job, truncate the table, increase region size, do a Import will it make use of the new region size? thanks V
Re: Export/Import and # of regions
Yes but you'll start with a single region, instead of truncating you probably want instead to create a pre-split table. J-D On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh vramanatha...@aol.com wrote: Hi, If I export existing table using Export MR job, truncate the table, increase region size, do a Import will it make use of the new region size? thanks V
Re: Export/Import and # of regions
Pre-splitting was discussed a few times on the mailing list today, and a few times in the past weeks, for example: http://search-hadoop.com/m/XB9Vr1gQc66 Import works on a pre-existing table so it won't recreate it. Also it doesn't know how your key space is constructed, so it cannot guess the start/stop row keys for you. J-D On Tue, Mar 29, 2011 at 2:33 PM, Venkatesh vramanatha...@aol.com wrote: Thanks J-D We have way too much data it won't fit in 1 region.Is Import smart enough create reqd # of regions? Cld u pl. elaborate on pre-split table creation? steps? Reason I'm doing this exercise is reduce # of regions in our cluster (in the absence of additional hardware 25K regions on 20 node) -Original Message- From: Jean-Daniel Cryans jdcry...@apache.org To: user@hbase.apache.org Sent: Tue, Mar 29, 2011 5:29 pm Subject: Re: Export/Import and # of regions Yes but you'll start with a single region, instead of truncating you probably want instead to create a pre-split table. J-D On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh vramanatha...@aol.com wrote: Hi, If I export existing table using Export MR job, truncate the table, increase region size, do a Import will it make use of the new region size? thanks V
Re: Export/Import and # of regions
Thanks J-D..Using 0.20.6..I don't see that method with pre-split in 0.20.6 API spec 1) Will the data still be accessible if I Import the data to a new table? (purely for backup reasons) I tried on small data set..I could.. Before I do export/Import on large table, want to make sure.. 2) Data exported using 0.20.6, can it be imported using 0.90.1? (i could use pre-split in this case) -Original Message- From: Jean-Daniel Cryans jdcry...@apache.org To: user@hbase.apache.org Sent: Tue, Mar 29, 2011 5:38 pm Subject: Re: Export/Import and # of regions Pre-splitting was discussed a few times on the mailing list today, and a few times in the past weeks, for example: http://search-hadoop.com/m/XB9Vr1gQc66 Import works on a pre-existing table so it won't recreate it. Also it doesn't know how your key space is constructed, so it cannot guess the start/stop row keys for you. J-D On Tue, Mar 29, 2011 at 2:33 PM, Venkatesh vramanatha...@aol.com wrote: Thanks J-D We have way too much data it won't fit in 1 region.Is Import smart enough create reqd # of regions? Cld u pl. elaborate on pre-split table creation? steps? Reason I'm doing this exercise is reduce # of regions in our cluster (in the absence of additional hardware 25K regions on 20 node) -Original Message- From: Jean-Daniel Cryans jdcry...@apache.org To: user@hbase.apache.org Sent: Tue, Mar 29, 2011 5:29 pm Subject: Re: Export/Import and # of regions Yes but you'll start with a single region, instead of truncating you probably want instead to create a pre-split table. J-D On Tue, Mar 29, 2011 at 2:27 PM, Venkatesh vramanatha...@aol.com wrote: Hi, If I export existing table using Export MR job, truncate the table, increase region size, do a Import will it make use of the new region size? thanks V
Re: Export/Import and # of regions
Thanks J-D..Using 0.20.6..I don't see that method with pre-split in 0.20.6 API spec It's new from 0.89, please consider upgrading. 1) Will the data still be accessible if I Import the data to a new table? (purely for backup reasons) I tried on small data set..I could.. Before I do export/Import on large table, want to make sure.. It's indeed a way to backup a table http://blog.sematext.com/2011/03/11/hbase-backup-options/ 2) Data exported using 0.20.6, can it be imported using 0.90.1? (i could use pre-split in this case) Yes. J-D
