Hi Ryan! The ultimate goal is to combine three geo data related projects that are currently supported by Wikimedia Germany and are in different stages of development: - WIWOSM (Wikipedia where in OSM) [0] - Multilingual Wikipedia Maps [1] - OpenGeoServer.org
Kolossos from WIWOSM made a rough sketch how such a combination could look like: https://toolserver.org/~kolossos/openlayers/kml-on-ol-json3.php?lang=de&title=Donau&layers=00000B0FTTTT WIWOSM & Multilingual Maps runs fine using the current toolserver infrastructure. But with OpenGeoServer.org I'm currently doing a lot of low level stuff like tile cache performance optimization that would be difficult to do on toolserver.org. The next step on my side is be to get a full Landsat [2] layer with about 3TB of data up and running and see how to scale it well across multiple servers. Thanks for the documentation and I will take a look at saltstack too. cu andreas [0] https://wiki.openstreetmap.org/wiki/WIWOSM [1] https://wiki.openstreetmap.org/wiki/Multilingual_maps_Wikipedia_project [2] https://en.wikipedia.org/wiki/Landsat 2012/10/4 Ryan Lane <[email protected]> > On Thu, Oct 4, 2012 at 4:13 AM, Andreas Trawoeger > <[email protected]> wrote: > > Hi everybody! > > > > Hi! > > > I'm currently running a project called opengeoserver.org that collects > and > > redistributes freely available Open Geodata (at the moment I'm at 1.3 TB > of > > data) . The project is currently receiving some funding from Wikimedia > > Germany and is a likely candidate to be migrated to the Wikimedia Labs > > sometime in the near future. > > > > Do you mean that your test and development will move to Labs, or that > your production site will? The former is ok, the latter is not. Labs > is not meant for production services. If the production service is to > be moved, then we should be getting it going in a test and development > mode in Labs, then move it to production. > > > But being a sysadmin that neither wants to replace his fine tuned XFS > > storage with Gluster nor his Fabric [0] deployment scripts with Puppet > > without a lot of testing. I would actually prefer to set up my own > OpenStack > > instance on my existing systems, test different configurations options > and > > then migrate the services to Wikimedia Labs. > > > > That's a decent amount of data. We have about 40TB of gluster space > available in the pmtpa cluster and will have another 40+TB in eqiad. > It'll be slower than direct XFS access, but we don't really have a > means of providing that much space in a direct access mode. > > We'll be using saltstack for deployment soon, which works in a very > fabric-like way. It's a little more flexible than fabric, in fact. > Puppet really shouldn't be used for deployment, just configuration and > dependency management. > > > It's easy to figure out that Wikimedia Labs is using Ubuntu, OpenStack & > > Gluster, but I couldn't find any detailed information about what versions > > are currently used and how the current hardware & storage configuration > > actually looks like. > > > > We have some slightly outdated docs here: > > http://wikitech.wikimedia.org/view/OpenStack > > - Ryan > > _______________________________________________ > Labs-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/labs-l >
_______________________________________________ Labs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/labs-l
