helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/12075 )
Change subject: KUDU-2636: LBM supports deleting dead and full containers ...................................................................... Patch Set 6: (11 comments) There are more than one LogBlock which will reference the same LogBlockContainer. If these LogBlocks are deleted in different LogBlockDeletionTransaction concurrently, then the LogBlockContainer will be deleted repeatedly. I am trying to fix it. http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.h File src/kudu/fs/log_block_manager.h: http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.h@276 PS6, Line 276: // Removes dead containers from disk. : // : // Must call RemoveFullContainersUnlocked() first. > This is a good place to expand about the interaction with FileCache. You co Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc File src/kudu/fs/log_block_manager.cc: http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1237 PS6, Line 1237: // Due to the shared ownership of LogBlockDeletionTransaction, the destructor is : // responsible for destroying all registered blocks. This includes: : // 1. Punching holes in deleted blocks, and : // 2. Deleting dead containers outright. > The formatting issue persists. Take a look at the diff in gerrit; you'll se done. a new line character '\r\n' in MS-DOS causes this problem. http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1256 PS6, Line 1256: std::unordered_map<internal::LogBlockContainer*, : std::vector<BlockInterval>> deleted_interval_map_; > Nit: while you're here, could you drop the unnecessary std:: prefixes here Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1260 PS6, Line 1260: unordered_map<DataDir*, set<string>> dead_containers_by_dir_; > The value would probably be more efficient as an unordered_set rather than Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1631 PS6, Line 1631: // Access to container_ must be before calling log_block_.reset(). : // : // Because log_block_.reset() may cause the LogBlock() to lose its last reference : // and be destroyed. This may cause a registered LogBlockDeletionTransaction to lose : // its last reference and be destroyed, which will trigger the asynchronous code that : // will eventually destroy the container which is dead. Thus, the ordering of operations : // here is very important, otherwise it is unsafe. > Nit: reword a bit: Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1965 PS6, Line 1965: // The Descriptor() in the FileCache() will be destroyed : // along with LogBlockContainer's destructor. > Nit: I would drop this comment. It's not really necessary to improve the un Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1980 PS6, Line 1980: DataDir* dir, const std::vector<std::string>& names) { > Don't need std:: prefixes. Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@1982 PS6, Line 1982: for (const auto& name : names) { > There are some extra line formatting issues here. Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@2008 PS6, Line 2008: // Remove from the memory. > Nit: "Remove the containers from memory." Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@2014 PS6, Line 2014: // Remove from the disk. > Nit: "Remove the containers from disk." Done http://gerrit.cloudera.org:8080/#/c/12075/6/src/kudu/fs/log_block_manager.cc@2580 PS6, Line 2580: // After the deletions, the data directory is sync'ed to reduce the chance : // of a data file existing without its corresponding metadata file (or vice : // versa) in the event of a crash. The block manager would treat such a case : // as corruption and require manual intervention. : // : // TODO(adar) the above is not fool-proof; a crash could manifest in between : // any pair of deletions. That said, the odds of it happening are incredibly : // rare, and manual resolution isn't hard (just delete the existing file). > Move this into the body of RemoveDeadContainersFromDisk, just above the cal Done -- To view, visit http://gerrit.cloudera.org:8080/12075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7d3672ae3c3dd9acd489120d653c44a641537f10 Gerrit-Change-Number: 12075 Gerrit-PatchSet: 6 Gerrit-Owner: helifu <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Hao Hao <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: helifu <[email protected]> Gerrit-Comment-Date: Wed, 19 Dec 2018 09:18:04 +0000 Gerrit-HasComments: Yes
