Re: table creation is failing now and then (CDH3b3)
Thank you, sir. Do you happen to know if hbase swtich from 0.89 CDH3b3 to 0.90.1 is going to be file-level compatible? I.e. do i have to export and re-import data in between? Thank you. On Wed, Feb 23, 2011 at 12:05 PM, Ryan Rawson ryano...@gmail.com wrote: You should consider upgrading to hbase 0.90.1, a lot of these kinds of issues were fixed. -ryan On Wed, Feb 23, 2011 at 12:02 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Hi all, from time to time we come to a sitation where .META. table seems to be stuck in some corrupted state. In particular, attempts to create more tables cause ERROR: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region LEAD_DATA,,1298484698945.8226d7a44286402e80c9b991d2c00cc3. The server is not even under any load worth mentioning at the moment. Table manipulations are failing with both shell and HBAdmin, so it's not an isolated shell issue (there seems to be similar issue related to shell only, 872, but it's pretty old one). I scanned for solutions for this particular error, and it seems to be revolving around flushing .META. , major compact it and restarting the master(s). In our case, only restarting the master helped to remove the condition. logs seem to be stacktrace-free. My questions are -- -- is there a way to figure out what causes those inconsistences in .META.? -- is there a way to fix that other than restarting master? Is restarting master in this case implies a downtime? Thanks. -Dmitriy
Re: table creation is failing now and then (CDH3b3)
See http://hbase.apache.org/upgrading.html#upgrade0.90 Also if you want to stay on CDH, CDH3b4 was just released. J-D On Thu, Feb 24, 2011 at 4:08 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Thank you, sir. Do you happen to know if hbase swtich from 0.89 CDH3b3 to 0.90.1 is going to be file-level compatible? I.e. do i have to export and re-import data in between? Thank you. On Wed, Feb 23, 2011 at 12:05 PM, Ryan Rawson ryano...@gmail.com wrote: You should consider upgrading to hbase 0.90.1, a lot of these kinds of issues were fixed. -ryan On Wed, Feb 23, 2011 at 12:02 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Hi all, from time to time we come to a sitation where .META. table seems to be stuck in some corrupted state. In particular, attempts to create more tables cause ERROR: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region LEAD_DATA,,1298484698945.8226d7a44286402e80c9b991d2c00cc3. The server is not even under any load worth mentioning at the moment. Table manipulations are failing with both shell and HBAdmin, so it's not an isolated shell issue (there seems to be similar issue related to shell only, 872, but it's pretty old one). I scanned for solutions for this particular error, and it seems to be revolving around flushing .META. , major compact it and restarting the master(s). In our case, only restarting the master helped to remove the condition. logs seem to be stacktrace-free. My questions are -- -- is there a way to figure out what causes those inconsistences in .META.? -- is there a way to fix that other than restarting master? Is restarting master in this case implies a downtime? Thanks. -Dmitriy
Re: table creation is failing now and then (CDH3b3)
Thank you, sir. On Thu, Feb 24, 2011 at 4:12 PM, Jean-Daniel Cryans jdcry...@apache.orgwrote: See http://hbase.apache.org/upgrading.html#upgrade0.90 Also if you want to stay on CDH, CDH3b4 was just released. J-D On Thu, Feb 24, 2011 at 4:08 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Thank you, sir. Do you happen to know if hbase swtich from 0.89 CDH3b3 to 0.90.1 is going to be file-level compatible? I.e. do i have to export and re-import data in between? Thank you. On Wed, Feb 23, 2011 at 12:05 PM, Ryan Rawson ryano...@gmail.com wrote: You should consider upgrading to hbase 0.90.1, a lot of these kinds of issues were fixed. -ryan On Wed, Feb 23, 2011 at 12:02 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Hi all, from time to time we come to a sitation where .META. table seems to be stuck in some corrupted state. In particular, attempts to create more tables cause ERROR: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region LEAD_DATA,,1298484698945.8226d7a44286402e80c9b991d2c00cc3. The server is not even under any load worth mentioning at the moment. Table manipulations are failing with both shell and HBAdmin, so it's not an isolated shell issue (there seems to be similar issue related to shell only, 872, but it's pretty old one). I scanned for solutions for this particular error, and it seems to be revolving around flushing .META. , major compact it and restarting the master(s). In our case, only restarting the master helped to remove the condition. logs seem to be stacktrace-free. My questions are -- -- is there a way to figure out what causes those inconsistences in .META.? -- is there a way to fix that other than restarting master? Is restarting master in this case implies a downtime? Thanks. -Dmitriy
Re: table creation is failing now and then (CDH3b3)
You should consider upgrading to hbase 0.90.1, a lot of these kinds of issues were fixed. -ryan On Wed, Feb 23, 2011 at 12:02 PM, Dmitriy Lyubimov dlie...@gmail.com wrote: Hi all, from time to time we come to a sitation where .META. table seems to be stuck in some corrupted state. In particular, attempts to create more tables cause ERROR: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region LEAD_DATA,,1298484698945.8226d7a44286402e80c9b991d2c00cc3. The server is not even under any load worth mentioning at the moment. Table manipulations are failing with both shell and HBAdmin, so it's not an isolated shell issue (there seems to be similar issue related to shell only, 872, but it's pretty old one). I scanned for solutions for this particular error, and it seems to be revolving around flushing .META. , major compact it and restarting the master(s). In our case, only restarting the master helped to remove the condition. logs seem to be stacktrace-free. My questions are -- -- is there a way to figure out what causes those inconsistences in .META.? -- is there a way to fix that other than restarting master? Is restarting master in this case implies a downtime? Thanks. -Dmitriy
