-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Owen,
On 8/12/17 12:47 PM, Owen Rubel wrote: > What I am talking about is something that improves communication as > we notice that communication channel needing more resources. Not > caching what is communicated... improving the CHANNEL for > communicating the resource (whatever it may be). If the channel is an HTTP connection (or TCP; the application protocol isn't terribly relevant), then you are limited by the following: 1. Network bandwidth 2. Available threads (to service a particular request) 3. Hardware resources on the server (CPU/memory/disk/etc.) Let's ignore 1 and 3 for now, since you are primarily concerned with concurrency, and concurrency is useless if the other resources are constrained or otherwise limiting the equation. Let's say we had "per endpoint" thread pools, so that e.g. /create had its own thread pool, and /show had another one, etc. What would that buy us? (Let's ignore for now the fact that one set of threads must always be used to decode the request to decide where it's going, like /create or /show.) If we have a limited total number of threads (e.g. 10), then we could "reserve" some of them so that we could always have 2 threads for /create even if all the other threads in the system (the other 8) were being used for something else. If we had 2 threads for /create and 2 threads for /show, then only 6 would remain for e.g. /edit or /delete. So if 6 threads were already being used for /edit or /delete, the 7th incoming request would be queued, but anyone making a request for /show or /create would (if a thread in those pools is available) be serviced immediately. I can see some utility in this ability, because it would allow the container to ensure that some resources were never starved... or, rather, that they have some priority over certain other services. In other words, the service could enjoy guaranteed provisioning for certain endpoints. As it stands, Tomcat (and, I would venture a guess, most if not all other containers) implements a fair request pipeline where requests are (at least roughly) serviced in the order in which they are received. Rather than guaranteeing provisioning for a particular endpoint, the closest thing that could be implemented (at the application level) would be a resource-availability-limiting mechanism, such as counting the number of in-flight requests and rejecting those which exceed some threshold with e.g. a 503 response. Unfortunately, that doesn't actually prioritize some requests, it merely rejects others in order to attempt to prioritize those others. It also starves endpoints even when there is no reason to do so (e.g. in the 10-thread scenario, if all 4 /show and /create threads are idle, but 6 requests are already in process for the other endpoints, a 7th request for those other endpoints will be rejected). I believe that per-endpoint provisioning is a possibility, but I don't think that the potential gains are worth the certain complexity of the system required to implement it. There are other ways to handle heterogeneous service requests in a way that doesn't starve one type of request in favor of another. One obvious solution is horizontal scaling with a load-balancer. An LB can be used to implement a sort of guaranteed-provisioning for certain endpoints by providing more back-end servers for certain endpoints. If you want to make sure that /show can be called by any client at any time, then make sure you spin-up 1000 /show servers and register them with the load-balancer. You can survive with only maybe 10 nodes servicing /delete requests; others will either wait in a queue or receive a 503 from the lb. For my money, I'd maximize the number of threads available for all requests (whether within a single server, or across a large cluster) and not require that they be available for any particular endpoint. Once you have to depart from a single server, you MUST have something like a load-balancer involved, and therefore the above solution becomes not only more practical but also more powerful. Since relying on a one-box-wonder to run a high-availability web service isn't practical, provisioning is necessarily above the cluster-node level, and so the problem has effectively moved from the app server to the load-balancer (or reverse proxy). I believe the application server is an inappropriate place to implement this type of provisioning because it's too small-scale. The app server should serve requests as quickly as possible, and arranging for this kind of provisioning would add a level of complexity that would jeopardize performance of all requests within the application server. > But like you said, this is not something that is doable so I'll > look elsewhere. I think it's doable, just not worth it given the orthogonal solutions available. Some things are better-implemented at other layers of the application (as a whole system) and perhaps not the application server itself. Someone with intimate experience with Obidos should be familiar with the benefits of separation of these kinds of concerns ;) If you are really more concerned with threads that are tied-up with I/O-bound work, then Websocket really is your friend. The complex threading model of Websocket allows applications to do Real Work on application threads and then delegate the work of pushing bytes across the wire to the container, resulting in very few I/O-bound threads. But the way you have phrased your questions seems like you were more interested in guaranteed provisioning than avoiding I/O-bound threads. - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlmQTK8ACgkQHPApP6U8 pFiSqw/+LB0K2z2wMZkZca7hqpOTnC3wyhr/8tAsJhPKNWMu9A/MzTAbDLhHM6Q3 anRBSEzAPU1RR0YDh4ym0yi81C+5LWf92i74ITHhZOqnOsHJpP2NpENdumHNeq5C USwbaa2BAycL0SxKdSmm5kiXDs6HQcH/dspudIHcna2Wx9mOWaW7/jcnmc4XZcFe Na/Xi6Ita+oky8yadjt8k5GTqPBD0AFDu6KYXfhIaqkoa5OXTn8A1HuCsMoDYJQj jYMd58ahbKGjhPgwPq0D/1gtFf6VcTAxK7d7T4EvKXvIYgv3vj+4ddAXRk6y6Ac4 AMw70PjvZpIZdslHwTwGk3AJ2u+fxBYIXmF3dDh7oIh00+HXow9V9WqLfkW9jDV1 vIC5ofjsiztNCZnhGH4eTIRohn0mou3mZnIbM1dtc+NmLGArGYjxU2Q1rHcWqjlM QjKQimdPEaAT0iwtz6iY8hMI4PHJ9B8BnFHrZMm6wnYkMBbA0IHM2ofl1BgtgdIH IKfm2yo4cGcUKFXYvWTKHFslV5Seqs5rc0NlaRO8OYt4FvxjEt3THS6b8Wog7qzs EMGTrFouq2SyW+4cKp6cajOUAAU7u2PqkUxbEEZcf1ITwhw4aNgdS+bVhTwDApw4 w1hDPV/IsNHpgSFRiOzKpOWFtRjsCbtKIwf3WNEO6EfgmGZGQpA= =Lrxc -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
