morningman opened a new pull request #436: Redesign meta version URL: https://github.com/apache/incubator-doris/pull/436 Because the meta version is only be used in catalog saving and loading. So currently this version is a field of Catalog class. And we can get this version only by calling Catalog.getCurrentCatalogJournalVersion(). But in restore process, we need to read the meta data which is saved with a specified meta version. So we need a flexible way to read a meta data using a specified meta version, not only the version from Catalog. So we create a new class called MetaContext. Currently it only has one field, 'journalVersion', to save the current journal version. And it is a thread local variable, so that we can create a MetaContext anywhere we want, and setting the 'journalVersion' which we want to use for reading meta. Currently, there are 4 threads which is related to meta data saving and loading. 1. The Frontend starting thread, which will call Catalog.initialize() to load the image. 2. the Frontend state listener thread, which will listen the state changing, and call transferToMaster() or transferToNonMaster(). 3. Edit log replayed thread, which is created when calling transferToNonMaster(). It will replay edit log 4. Checkpoint thread, which is created when calling transferToMaster(). It will do the checkpoint periodically. We create MetaContext thread local variable for these 4 threads, and thread 2,3,4's meta context inherit from thread 1's meta context. Because thread 1 will load the origin image file and get the very first meta version. On the other hand, we add the current meta version in backup job info file when doing backup job. So that when restoring from a backup snapshot, we can know which meta version we should use for read the meta. And also , we add a new property "meta_version" for Restore stmt, so that we can specify the meta version used for reading backup meta. It is for those old backup snapshots which do not has meta version saving in backup job info file.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
