On Tue, Dec 05, 2006 at 12:47:57PM -0600, Reid Priedhorsky wrote: > Christopher Schmidt wrote: > > > >In order to improve the scalability of Open Source mapping on the Web, > >MetaCarta Labs has developed a tool we call TileCache. TileCache acts as > >a buffer between your WMS server and its clients. It accepts WMS > >requests constrained by scale and bounding box (i.e., WMS-C, as > >described by the OSGeo WMS Tiling Client Recommendation[1]), and returns > >images that can be viewed in clients such as OpenLayers. TileCache also > >supports the OSGeo Tile Map Service Specification[2], and requests in the > >format supported by the WorldWind 3D globe client. > > Chris, > > Very interesting! We need a tool to do this. > > We had previously planned on using a generic HTTP caching proxy server > such as Squid. I would be curious to read your take on the advantages > and disadvantages of choosing your tool over something like Squid.
>From the WMS-C page: "Although some users of WMS have attempted to use naive HTTP proxies to solve the problems of WMS caching, caching HTTP proxies alone don't take full advantage of prior knowledge of the WMS protocol, and don't account for the various idiosyncrasies that WMS clients can present." Some of the various idiosyncranasies: WMS clients can request parameters in different orders, each one getting a slightly different set of cache hits. WMS clients can request areas which are not meant to be cached -- requesting on a non-gridded bounding box -- and thus poisoning your cache. Clients can do this unintentionally: we've seen well-behaving web mapping clients which simply don't request tiles on a grid all the time. Maintaining a WMS cache requires the ability to control the cache: you may wish to clear out an area of bad data, or to see how large your cache is. Typically, HTTP Proxy Caches are very opaque as far as access to them goes: logging is available, but oftentimes other information about what is currently in the cache is not. There are exceptions to this, but the most popular HTTP Cache (squid) has very little in the way of cache control. It all comes down to who controls the clients. If you can control the clients, and can ensure that they are doing the right thing, and you have no need to have a pluggable caching mechanism, then it's likely that Squid is for you. If you're looking to provide a high-speed service to users of disparate clients, then TileCache might be for you. If you're looking to create a cache of regularly changing data (editable maps, for example) then TileCache might be for you. If you have a data source which is not available as WMS, and want to write a WMS wrapper around it, then TileCache might be fore you. Or, if you'd just rather have a couple hundred lines of Python code, rather than a full HTTP caching tool, then maybe TileCache is for you too. The decision to use TileCache came for us after repeated attempts to use HTTP Proxies turned up flaws in their place in our pipeline. With the WMS Tiling Client recommendation and TileCache, those problems have gone away, and I get to see fun things like how long it takes to render tiles from a variety of sources to boot. TileCache will also work if you want to create a cache with no rendering backend. Use the precaching tools included in the distribution to build up all the cache you care about, and turn off your WMS server. No more dynamic requests, and TileCache will serve out your URLs at 300/second. TileCache basically fits the same niche as pre-rendering all your data... only it does it on the fly. TileCache was written with the idea of being able to scale to high loads without problems: if the max requests/second we can serve from a DiskCache drops below 300 from a single thread, I start to figure out where TileCache can be optimized. If pre-rendering all your data is a reasonable expectation given your setup, then you probably don't need TileCache. But if you want to cache, for example, 20kb tiles for the whole world down to 16 zoom levels, you're caching 20TB of data. This is not a reasonable expecation for most people. The MetaCarta Labs demo of TileCache is serving nearly a dozen layers now, and all the data that has been browsed so far is less than 100GB, and most of those layers started off with the first 12 layers pre-cached. I like TileCache because it's simple. I like TileCache because it works. I haven't found another tool that meets the needs of accelerating WMS that solves most of the use cases I can find. Regards, -- Christopher Schmidt MetaCarta _______________________________________________ Geowanking mailing list [email protected] http://lists.burri.to/mailman/listinfo/geowanking
