Daniel, sure, I can give you the full scoop. Considering that I designed
and developed most of it, I do have some knowledge here. Big kudos to Max
Semenik for developing all the initial data queries and php work, all the
amazing UI done by Julien Girault, and the current maps engineering team
for a lot of good recent work, especially the multilingual support.
Summary:
* OSM data is imported a postgres DB with osm2pgsql (initially used default
schema, now migrating to ClearTables)
* Tilerator (Kartohterian's tile generation manager) is used to generate
vector tiles from SQL queries. One of the SQL output fields is a
JSON-formatted value with all the available languages -- {"en":'"...,"
"fr":"...", ...}. Mapnik creates vector tiles from the SQL output, and
babel decodes them, expands languages into multiple fields, and
recompresses them. Tilerator then stores them into Cassandra database
(WMF), but you could use any other storage, e.g. postgress or mbtiles. So
as the result, tilerator's pipeline produces vector tiles with all
available languages. No rendering is done yet. Tilerator generates just
the z0..z14. My guesstimate is ~300GB, but these numbers could have
changed. Sadly, tiles tiles are not as optimized as what Open Map Tiles
produce, hence slightly oversized.
* When user requests a tile in a given language, vector tile is loaded from
storage, modified on the fly by using babel - unpacks the tile and this
time choose just one language. The tile is repacked and passed to mapnik
for rendering. The resulting tile is not stored on disk, but it does get
cached by Varnish.
So as you can see, there is nearly no difference between one language and
unlimited number of languages with this approach - except for the slightly
slower generation and rendering, and bigger cache fragmentation. See
various Kartotherian repos in github for more information.
-- Yuri / @nyuriks
On Fri, May 4, 2018 at 4:14 AM Daniel Koć <daniel@koć.pl> wrote:
> W dniu 04.05.2018 o 02:30, Yuri Astrakhan pisze:
> > Daniel, the only real difference between serving every available
> > language and serving just one is cache fragmentation, and that's may
> > be different in your case.
>
> Sure, but you're talking about just one link of the chain. The WMF
> vector tiles are produced from a database, rasterized somewhere down the
> line, then probably written on the disk, served, cached etc, so it's not
> that simple.
>
> That's why I ask about the bottom line in this specific case. There are
> many factors which will be different (like how complex the style is, for
> example), but it's good to estimate the order of magnitude at least. But
> all the details are also interesting to me.
>
> --
> "My method is uncertain/ It's a mess but it's working" [F. Apple]
>
>
>
_______________________________________________
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
