Lars, Can you dump your documents to external storage (either HDFS or ordinary file space storage)?
On 1/4/08 10:01 PM, "larsgeorge" <[EMAIL PROTECTED]> wrote: > > Jim, > > I have inserted about 5million documents into HBase and translate them into > 15 languages (means I end up with about 75million in the end). That data is > only recreatable if we process them costly again. So I am in need of a > migration path. > > For me this is a definitely +1 for a migration tool. > > Sorry to be a hassle like this. :\ > > Lars > > ---- > Lars George, CTO > WorldLingo > > > Jim Kellerman wrote: >> >> Do you have data stored in HBase that you cannot recreate? >> >> HADOOP-2478 will introduce an incompatible change in how HBase >> lays out files in HDFS so that should the root or meta tables >> be corrupted, it will be possible to reconstruct them from >> information in the file system alone. >> >> The problem is in building a migration utility. Anything that >> we could build to migrate from the current file structure to >> the new file structure would require that the root an meta >> regions be absolutely correct. If they are not, the migration >> would fail, because there is not enough information on disk >> currently to rebuild the root and meta regions. >> >> Is it acceptable for this change to be made without the provision >> of an upgrade utility? >> >> If not, are you willing to accept the risk that the upgrade >> may fail if you have corruption in your root or meta regions? >> >> After HADOOP-2478, we will be able to build a fault tolerant >> upgrade utility, should HBase's file structure change again. >> Additionally, we will be able to provide the equivalent of >> fsck for HBase after HADOOP-2478. >> >> --- >> Jim Kellerman, Senior Engineer; Powerset >> >> No virus found in this outgoing message. >> Checked by AVG Free Edition. >> Version: 7.5.516 / Virus Database: 269.17.13/1207 - Release Date: 1/2/2008 >> 11:29 AM >> >> >>
