Hello everyone,
I just wanted to let you know that I have committed a new feature to
mod_tile / renderd that might be of interest to some.
So far both mod_tile and renderd had the assumption that (meta)tiles
would be stored on a posix filesystem deeply baked into their
architecture. While the posix file system it self is a rather flexible
pluggable storage system with many different backends you can choose
from, (e.g. from temporary memory based filesystem, over traditional
local filesystems to network filesystems to cluster filesystems to
arbitrary usermode filesystems) with growing diversity in osm based
tileserver setups, this might not be enough.
I have therefore refactored the code of mod_tile and renderd to cleanly
separate out all storage access routines into a separate API[0], which
can then easily be implemented by various different storage backends and
methods.
In the initial commit I have implemented 3 different backends so far.
1) Posix file system storage backend: This backend is equivalent to how
mod_tile and renderd have so far worked. It uses any posix compliant
file system to store metatiles.
2) RADOS ( Reliable Autonomic Distributed Object Storage) [1]: Rados is
a cluster based large scale distributed and redundant object storage,
which is part of the ceph project. It allows to store objects across
many servers allowing it to scale up to many terrabyte of data to very
high performance on commodity hardware.
3) memcached: Memcached is a pure memory based storage system which
allows for very high performance. As it can also cluster many servers
together, it can create a large scale memory storage.
I very much suspect that the majority of single server tileserver setups
will continue to use the posix filesystem backend as it is the easiest
to use, most robust and for local setups possibly also the fastest
thanks to built in OS level caching. However, for those setups that need
to scale up to multi tileserver setups, be it for performance or for
redundancy / high availability reasons other backends like the cluster
based rados backend might be preferable.
Renderd has supported multi-server distributed rendering for a long time
now, in which a central renderd performs queueing and request
de-duplication and then passing those requests on to an arbitrary number
of slave renderers who do the actual rendering. Recently mod_tile has
gained the ability to talk to renderd via a tcp socket (previously it
was limited to a unix domain socket), so mod_tile and renderd can now
also live on different servers than renderd and multiple mod_tile
servers can connect to the dirstributed renderd infrastructure. This
should make all components of the tile rendering stack arbitrarily and
separately scalable and redundant, allowing for very large scale
deployments of many thousand requests per second using mod_tile.
However, the setup requires all components to have a shared view to the
storage backend. While this was previously already possible using
cluster based posix filesystems like glusterFS or using central network
attached storage, hooking more directly into a cluster storage like
rados might be beneficial.
In the future hopefully further backends will be implemented. I can
think of a couple of potential interesting candidates. For example one
could implement a MBTiles[2] backend, or a backend that stores tiles on
Amazon S3, or a backend that combines the various other backends in a
hierarchical way like a combination of memcached for local caching and
Amazon S3 for large scale storage.
To remain compatible with tirex, I hope to port over this feature to
tirex as well, at least to the mapnik render backend.
For the moment the rados and memcached backends should still be
considered somewhat experimental until more thorough testing can be
conducted, but they should already be fully functional.
Any comments, suggestions, improvements or bug reports are welcome.
Kai
[0] https://github.com/openstreetmap/mod_tile/blob/master/store.h
[1] http://ceph.com/docs/master/rados/
[2] http://mapbox.com/developers/mbtiles/
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
