On Mon, 14 Oct 2019 at 05:16, Andrea Aime <andrea.a...@geo-solutions.it> wrote:
> On Fri, Oct 11, 2019 at 8:53 PM Gabriel Roldan <gabriel.rol...@gmail.com> > wrote: > >> >>> Good point, they should be enclosed, meaning that the layer end needs to >>> be issued from the rendering thread. It can be done, matter of >>> implementation, something to be checked in the PR. >>> >> > Checked, it's indeed happening in the rendering thread, thus it should be > enclosing feature events: > > https://github.com/geotools/geotools/pull/2606/files#diff-0c2a1d63c591affa14372de83ef72814R3876 > > >> Speaking of MT, the rendering is only lightly multi-threaded, there are >>> two fixed threads, one reading the data, preprocessing it, preparing the >>> styles to be compatible with Java 2D, >>> and one actually rendering the geometries with the desired styles, >>> connected by a blocking queue. It was done like this to compensate for the >>> Ductus rasterizer >>> scalability issues, needs to be evaluated if it's still beneficial once >>> we move away from Java 8 (Oracle Java 8 is the last implementation based on >>> it, from Java 9 >>> onwards Marlin is built-in into the JDK). >>> >> Thanks for the clarification. So, SimpleFeatureRenderer's setThreadPool() >> says "Sets a thread pool to be used in parallel rendering". I guess its >> primary purpose is then not doing parallel rendering itself, but being >> given the thread pool it can borrow those two threads from, and allowing to >> set a limit on parallel rendering among multiple renderer instances (if the >> pool has an upper limit)? >> > > It's even simpler than that. The primary thread doing the reads and > semplification is the one you call StreamingRenderer.paint on, only the > java2d painting thread is borrowed from the pool. "Rendering" is the > activity that the StreamingRenderer does, it encompasses everything > from reading the data, reprojecting, evaluating symbolizer, and painting. > The pool provides a single thread that makes the rendering activity > parallel. > > I'm guessing you're thinking about true layer parallel rendering... I > think it would be a good idea for a desktop system, with all the resources > of a computer at the disposal of a single user, but it may prove very > problematic for a server setup under load (if light load, we're basically > going back to the single user case). Think many concurrent requests, > against a multi-layer map (e.g., seeding a OSM like map), each > one allocating separate rendering surfaces for each layer, maybe more than > one per layer to handle multiple feature type styles... the > amount of resource consumption would make the system thrash quite > quickly... Posing limits so that one ensures no more than X threads > rendering globally, no more than M megabytes of rendering surfaces > allocated, would be possible but definitely not trivial... and it would > have to be "global", the rendering surfaces keep on existing past the end > of the paint call, up until the image is fully encoded in output > in server side apps, so the system would have to allow external control. > > Totally get it, thanks. As a matter of fact, I've used multiple StreamingRenderer's with a shared thread pool for a desktop JavaFX app with great success. Yes, you gotta control how many of them run in parallel nonetheless. One pain point is labeling though, it'd be awesome if multiple renderes could share a single LabelCache and you can progressively render the labels as they get moved around to a top level pane. Next year :) > A fun project for someone with a lot of spare time (both to design it, and > then to follow it up for a year or two when it starts > actually rolling in production and the concurrency/load control issues > start popping up) :-D > The dividends would be interesting though, IMHO the ideal scalability > curve is not "linear" as many say, it's flat (full speed no > matter how many threads hit the server, using all the cores for a single > request on light load, and distributing them across > requests as the load goes up). > Screams for a reactive design IMHO. Hope I can find the time to experiment on it. Cheers and thanks for the insightful replies. Gabrie. > > Cheers > Andrea > > == > > GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf > Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa > (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 > http://www.geo-solutions.it http://twitter.com/geosolutions_it > ------------------------------------------------------- *Con riferimento > alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - > Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni > circostanza inerente alla presente email (il suo contenuto, gli eventuali > allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i > destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per > errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le > sarei comunque grato se potesse darmene notizia. This email is intended > only for the person or entity to which it is addressed and may contain > information that is privileged, confidential or otherwise protected from > disclosure. We remind that - as provided by European Regulation 2016/679 > “GDPR” - copying, dissemination or use of this e-mail or the information > herein by anyone other than the intended recipient is prohibited. If you > have received this email by mistake, please notify us immediately by > telephone or e-mail.* > -- Gabriel Roldán
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
