Re: [Wikitech-l] [Maps-l] How should I generate static OSM maps for Wikimedia with the SlippyMap extension?

2009-07-15 Thread Brion Vibber
Ævar Arnfjörð Bjarmason wrote:
 On Wed, Jul 15, 2009 at 12:42 AM, Brion Vibberbr...@wikimedia.org wrote:
 Static maps of arbitrary size presumably have much the same problem here as 
 the map tile images. How is it handled there?
 
 The same way I want to do static map generation. Just put an arbitrary
 expiry time on it  serve it to the client.

Handy that!

 It might be good to use a Google Maps Static-like model here; the
 MediaWiki end can just make itself an img and stick all the relevant
 info onto the URL, calling out to the static map server.
 
 That's a very good idea, but I'd been assuming that I wouldn't be able
 to do that -- that each apache server was supposed to do things like
 EasyTimeline generation / image rescaling and likewise static map
 generation on its own  write it to the shared image NFS.
 
 But just being able to call out to an existing server makes things a
 whole lot easier.

Yep... this way is usually easier to manage for us, and also helps to 
isolate services for performance and security reasons. If the map 
rendering goes crazy and uses ten bajillion gigs of RAM and disk space, 
we don't want that taking down the site. :) If it's isolated and nothing 
depends on synchronous result returns, then you just don't get new maps 
until it's fixed.

For things like the PDF rendering where we need to be able to return 
data through the main web servers, isolation is still better -- we just 
need a timeout on fetches to make sure we don't get a million reqs stuck 
if the service hangs.

 This also means that we can set up the Wikimedia Deutschland servers
 to do tile rendering and then easily test it on a big wiki (like
 dewiki) by just enabling the SlippyMap extension which won't do
 anything more fancy than point static/tile URLs to the external
 service.
 
 So yay!

:D

 Either storing it on local disk there or simply leaving it to be cached by 
 upper-level squids
 
 Throwing it at the squids and making it their problem would be simpler
 for *me* at least.

Hehehe...

-- brion

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Maps-l] How should I generate static OSM maps for Wikimedia with the SlippyMap extension?

2009-07-15 Thread Brion Vibber
I wrote:
 For things like the PDF rendering where we need to be able to return 
 data through the main web servers, isolation is still better -- we just 
 need a timeout on fetches to make sure we don't get a million reqs stuck 
 if the service hangs.

Funny story: the first prototype of our Lucene search server went live 
around Christmas 2004 and immediately hung the whole site thanks to an 
overloaded server and an incredibly long default timeout for HTTP 
fetches from PHP. :)

Ahh, those heady early cowboy days...

-- brion

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


[Wikitech-l] Upload fileserver upgrade

2009-07-15 Thread Brion Vibber
We're applying a Solaris kernel patch for the ZFS performance problem on 
ms1, our main media file server.

While in progress, uploads will be temporarily disabled. Cached images 
should still display, but you may see some missing images in the meantime...

-- brion vibber (brion @ wikimedia.org)

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l


Re: [Wikitech-l] Upload fileserver upgrade

2009-07-15 Thread Brion Vibber
Brion Vibber wrote:
 We're applying a Solaris kernel patch for the ZFS performance problem on 
 ms1, our main media file server.
 
 While in progress, uploads will be temporarily disabled. Cached images 
 should still display, but you may see some missing images in the meantime...

The kernel patch is applied, the machine's rebooted, and uploads  math 
are re-enabled. So far all looks well; this'll keep things running 
smoothly while we continue work on rearranging the files (moving thumbs 
out to a separate server will simplify things and keep the main 
filesystem backup snapshots from filling up with short-lived files, for 
instance).

-- brion


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l