shooting from the hip with some feedback on these ideas On 06/22/2010 06:18 PM, Chris Holmes wrote: > I've been thinking a bit about how we can bring GeoWebCache in to > GeoNode, to get at some of the great performance enhancements it can > bring. Ideally we seamlessly cache all layers viewed in GeoNode, both > local and remote, even when those change. There are twists with each, > and both revolve around stale caches. > > With local caches we need a way to truncate the cache if the style > changes. Ideally when one is in style edit mode we don't use GWC at > all, only when someone does a final 'save' does it start caching the > change. > > With remote caches we need a way for a user to manage the cache, to > invalidate it when the remote server changes, either data or style. > Ideally it would have a GeoRSS feed of changes that GWC automatically > truncates based on. Less ideally there's a manual way to restart the > caching. > > A rough roadmap of how we might achieve the end goal: > > * Start with just caching remote layers. So when anyone puts in a > remote WMS it automatically gets added as a GWC layer. Gabriel is about > to commit a Least Recently Used cache to GWC, which will allow an admin > to set a total max for the cache. So we could let people add any layer, > but the admin of GeoNode can configure it to just cache the most used > tiles, up to a limit they set, be it 100 megs or 2 terabytes. For this > first step the caches may just get invalid, but the admin would have the > ability to truncate them in the GWC admin. > LRU worries me a bit; if we set the disk limit too low we may just end up with a lot of cache churn for little/no performance benefit. And the disk requirements can grow with minimal warning, since anyone can add a layer. There's also an easy DOS attack - anyone can fetch 18 zoom levels of some layer nobody uses and trash the cache (not a huge deal, how long would it take an attacker to do that anyway?). I'm not saying an LRU is a bad idea. I think caching will be a great improvement. It's just that there is a lot of room for refinement here (probably once we have better usage tracking we can use that to prioritize tilesets, for example.)
> * Cache local layers, coordinating with Style changes. I think Arne may > have coded this up, at least for the embedded GWC. We could perhaps > start with just doing the cache on the embedded maps, since those won't > have people switching to 'style mode'. Maybe that intermediary step > isn't necessary, but when we're in the map composer view we want to be > sure that when people are styling they're not seeing GWC tiles. When > they finish styling we should then truncate the existing cache and start > over. Another simplifying assumption we could also consider making is > only cache on the default style. Not sure how much that actually helps. > I don't think we need to avoid caching alternative styles. I do think we need to skip the cache while editing styles. It would be nice if we could use cached layers everywhere, and have only the layer being styled switch to "straight" WMS when styling is active. > * Remote layer management. This is sort of more general, I think in the > future we should figure out some more full representation in each > GeoNode of a remote layer. Right now remote layers can be added, but no > metadata can be found out about it. This is another whole topic, but > the implication for here is that such a page should/could have a way to > manage the cache of the local GeoNode. So you could truncate the cache > there (maybe just the person who added? Maybe you can set permissions > of who can truncate?). And then possibly also add a GeoRSS location to > automatically truncate from. > Yeah, it would be awesome if adding a WMS to the composer application got that service added to the GeoNode's GeoNetwork index, complete with metadata pages in the Django web app. And GeoNode can periodically scan the capabilities for added/removed layers, updated descriptions, new styles. These would be reflected in GeoNetwork and GeoWebCache as well as the Django database. It might be nice to also provide a listing of indexed services so users can track down the originating WMS services if they want. > The cool thing this set of things should lead to is to give a benefit to > people adding remote servers. They get increased speed and reliability > if they just add it to a map on a geoNode. So we can come in with a > GeoNode to an existing nice SDI implementation that already has a bunch > of WMS services, and then people can start creating maps on top of it, > and those maps perform even faster than the straight WMS. > > Thoughts? I think this could be a nice performance win, as most all our > maps are tiled. Should obviously be complemented by other > optimizations, like on the javascript side, but the two together should > make things quite zippy. > Having the WMS capabilities handled on the server side (and cached there) would probably be a nice win for loading services. We could do away with reading capabilities entirely until the user pulls up the add layers dialog (which is not available in the embedded viewers). We don't do GFI requests now but it might be worth thinking about how they interact with the cache. I also don't see this map caching doing much for offline/distributed data management, which seems like caching of another sort. It would be good to work out some answers related to that. -- David Winslow OpenGeo - http://opengeo.org/
