keith-turner commented on PR #3563: URL: https://github.com/apache/accumulo/pull/3563#issuecomment-1618902100
> Curious if you have looked at https://github.com/apache/accumulo/pull/3200 and your thoughts on using ZK to signal Tablet metadata changes. ZK is not very scalable in terms of writes (I think each write goes to the leader which then forwards it to all other ZK servers and waits for a majority to respond). This potential write bottleneck seem like an issue when considering it as an option for signaling tablet metadata refresh. As for the tablet metadata update in the tablet server I have not had a chance to look into it in detail. Some of the issues that I know will arise for refresh off the top of my head are : * Needs to properly synchronize with the cut over from in memory map to minor compacted file w.r.t. initiating new scan. A new scan should always use one or the other, never both or neither. * Needs to properly synchronize the tracking of scan file entries in tablet metadata and active scans. * Needs to properly synchronize with write ahead log recovery and tablets update around that. * Needs to properly synchronize with a tablet that is loading or unloading. The tablet server code has synchronization around bulk import, compactions, and splits that can be remove. I am thinking after we remove that code we can evaluate what needs are left and look into simplifying the tablets synchronization model that refresh will have to fit into. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
