Hi, the process can be abstracted into two transaction, and there are four operations in the transactions:
Del SG (delete storage group), Del SE (delete schema), Get SG (get or create storage group), Get SE (get or create schema), So, the deletion command (Transaction T1) is: T1: Del SG, Del SE the insertion command (Transaction T2) is: T2: Get SE, Get SG Then after a concurrent deletion (transaction T1) and insertion (transaction T2), the correct state of the DB can be: - State 1: no SG and no SE - State 2: both SG and SE exist - State 3: SE exists but SG does not exist - State 4 (bad state): SG exist but SE does not exist (no SG means both the memory structure and the data folder of SG do not exist) If we want the system clean, State 1 and state 2 are the only choices, and T1 and T2 should be executed serialized. If we relax the constraint, State 3 can also be acceptable: as if there is someone call the insertion operation in parallel, he/she may write new data into the SG later. We deleted the first part of the data from the user reasonably: another user remove the data in parallel. So I think whether choose accept State 3 depends on how much performance effort we have to lose to reach state 1 or state 2. Anyway, I think the process of T1 should change to : T1: Del SE, Del SG If so, state 4 can be avoided even though we have no lock. Best, ----------------------------------- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Haiming Zhu <[email protected]> 于2021年8月27日周五 下午8:03写道： > > Hi, > > After reviewing the code of delete storage group SQL statement, i have some > questions about the implementation. > > First, when deleting several storage groups in one statement, the metadata > in MManger will be deleted together after all storage groups are deleted. > But, if there are concurrent write operations, the write operations will > use old metadata, after the 'delete storage group' statement is executed > successfully and the metadata is deleted from MManger, the data written in > this period cannot be queried any more. > > Secondly, processorMap in StorageEngine class may need a > ReentrantReadWriteLock. For example, if we execute delete storage group > statement in cli and there are concurrent write operations to these storage > groups. Current code first remove storage group from processorMap and then > delete it, so other write operations will create a new > VirtualStorageGroupManager in processorMap and the new VSGManager's recover > method will conflict with the delete procedure, because tsfile may be > truncated in recover and delete needs flushing memtable to the same tsfile. > However, if we first delete a storage group and then remove it from > processorMap, other write operations may create a new TsFileProcessor in > the interval, making this delete statement incorrect. So, we need a > ReentrantReadWriteLock to lock processorMap. > > Best, > ------------------------------------ > Haiming Zhu > School of Software, Tsinghua University > > 朱海铭 > 清华大学 软件学院
