ddanielr commented on issue #3211: URL: https://github.com/apache/accumulo/issues/3211#issuecomment-1458835343
> What does it mean to exceed? I believe that the limitation to a TabletServer hosting a Tablet is the amount of memory given to the TabletServer. If a TabletServer can host N Tablets, it will still be limited as to how many tablets can read/write concurrently based on other properties. If a TabletServer can't host N tablets, then it it needs more memory, or more TabletServers need to be stood up so that the load can be distributed. I would say that determining how many Tablets a TabletServer can host would be difficult as the TabletMetadata is not the same size for each Tablet. It might be possible to use some rough upper bound on the amount of memory required to host each Tablet to determine how many TabletServers are needed. "Exceed "is exactly what you stated, no TabletServers with available memory exist to host the assigned tablet. I'm curious what the default balancer does for tablet assignment. If it's just based off # of tablets per tserver that doesn't seem optimal, but out of scope of this current conversation. > The way that I currently have this wired up, the tablets are loaded when they are required by the client. The client tries to determine the location (which TabletServer) a tablet is hosted on, so that it can send read or write requests. If the tablet is not hosted, and the table is ondemand, it sends a request for the tablet to be hosted which incurs some latency cost. So, the number of tablets that need to be hosted for an ondemand table are the sum of the tablets involved in read and write requests from all of the clients. If you do an unbounded scan, then it's going to eventually host the entire table, but that's what the user requested of the client. Thanks for the explanation, I think what wasn't clicking in my head was how the client interaction worked for a given write. This makes more sense. > If we end up with a mechanism for determining reliably how many tablets a TabletServer can host, then I agree we could introduce some priority scheme. Without that, and in its current form, the code just does what is asked. In https://github.com/apache/accumulo/issues/3212 we will create some mechanism for unloading ondemand tablets, not sure what that will look like yet. I think there are several possibilities, for example: a TabletServer could host N (configurable) ondemand tablets and unload them in an LRU fashion. Yeah, I could see priority being used for both hosting and unloading. But I agree that a mechanism for # of possible hosted tablets is required for that to work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
