keith-turner commented on issue #3823: URL: https://github.com/apache/accumulo/issues/3823#issuecomment-1783250146
Met with @dlmarion, @cshannon, @ddanielr , and @EdColeman to discuss this issue, below are notes from the meeting for thing I can remember. Please add more comments if there are things I missed. Meeting info [here](https://the-asf.slack.com/archives/CERNB8NDC/p1698351188844809) With table locks the current behavior is that a merge operation will wait on a compaction or bulk import. Consensus was it would be good to maintain this behavior. With the current table locks, a merge can prevent compactions and bulk imports from running even the ranges are disjoint. Like in the the following situation. 1. Really long running user compaction is started on range (a,m] 2. Merge is initiated on range (o,p], however it can not start because of the compaction. Places a write lock entry in the zookeeper table lock queue. 3. A bulk import for range (x,z] is initiated, but it blocks when is sees there is a write lock queued for the above merge. So in the situation above a bulk import is blocked by a merge that is blocked by a long running user compaction. Since these ranges are disjoint, if locking were moved to the tablet level all of the operations could proceed. Adding a tablet state of disabled would be useful for manual metadata edits. We discussed how this aligns with the tablet hosting goal. If a tablet has a hosting goal of never, then it will not be hosted but in elasticity compactions, splits, bulk import, etc could still change the tablet metadata. For exporting a table, potentially having a partially disabled tablet is kinda confusing. The export table operation could verify that all tablets are in the disabled state when it starts, but there is no way to prevent this from changing later. Discussed have another tablet state of readonly or snapshot (snapshot is a terminal read only state). So tablet export could require all the tablet to be in the snapshot state before it would start. Discussed where the valid state transitions for tablet might exists in the code. This could exists in ample, where it checks the transitions before its made. If write locks were removed and fate storage was moved to a zookeeper table, then bulk imports may have zero zookeeper writes which may help with scalability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
