Re: [Server-devel] [IAEP] [XSCE] RE: [UKids] Re: [support-gang] Taking, OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC
The idea of pre-rendering some of the tiles and doing the rest on the fly has always appealed to me too. In fact I thought this is what IIAB was doing. But I now think that in order to render anything you need the 500+G database, which Braddock and now Anish have put on the large rendering machine, but which seems to me like a lot of space for the schoolserver itself. In terms of the efficiency of the tile rendering machine itself there are probably improvements that could be made. I don't know why we need all the data in ssd and not just the region of interest, but I think Anish said it took 1 1/2 weeks just to load the map data into the database, before any rendering. I have wondered if a distributed model is possible, like the seti and other projects, where people could sign up to have their machines do a small portion of the rendering from a small portion of the data with perhaps btsync to distribute the results. This would take some work both on the software and the organizational side. In terms of a disconnected client, the maps likely have to be tiles as we don't expect client machines to have a spare 500G of storage. There are ajax client-side apps that stitch together tiles (I think slippy map is one), but they would have to be coaxed into caching tiles on the client and using that cache when disconnected. -Original Message- From: unleashk...@googlegroups.com [mailto:unleashk...@googlegroups.com] On Behalf Of Tony Anderson Sent: Monday, June 15, 2015 5:43 AM To: xsce-de...@googlegroups.com; unleashk...@googlegroups.com; gagno...@gmail.com; Jaakko Helleranta; server-devel; IAEP SugarLabs; i...@lists.sugarlabs.org; Community Support Volunteers -- who help respond to help AT laptop. org; gagno...@gmail.com; Jaakko Helleranta; server-devel aep; i...@lists.sugarlabs.org; Community Support Volunteers -- who help respond to help AT laptop. org; Nick Doiron Subject: Re: Re: [IAEP] [XSCE] RE: [UKids] Re: [support-gang] Taking, OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC There seem to be two independent problems. First, make an updated version of the IIAB OSM. The goal is to understand what are the computer requirements to do this and how long it takes and then be able to make a new version whenever needed. Second, make available more detail than IIAB OSM provides for a region selected by a deployment while allowing the user (learner) to display, edit, and save it as a new map from this data. On the first problem, I am confused. I would guess the job is cpu intensive. However, many comments about SSD suggest it is i/o limited. In a former life, I worked on parallel programming in which the technical problem was to overlap disk i/o with computing. One thread bringing the data for the next case into memory while the processor worked on the current case. The trick was to determine the size of data needed to keep the processor busy and not waiting on i/o. The goal was to overlap i/o with processing to minimize processor idle time. I assume the processing of the world can be broken down into independent pieces (e.g. one tile at the global zoom level). The experiment with Nepal should show the processing time required as well as the disk i/o time. Based on that information, the work plan would be to load the global data on hard drive and then set up double buffered pipelines from hard disk to SSD to memory which can keep the processor busy. On the second, it is not clear that the greater detail needs to be OSM tiles. Something like Nick's map.activity/mappack with the map data on the school server (separate from OSM) and a client application (Sugar or Sugar-web or html) accessing the data, displaying it, and enabling the learner to add to it (à la GIS). After discovering that umapper is not umap, umap is clearly open source and could be a good base for the client application. This application should be able to get data from the school server and, as always, it should be possible for the learner to do meaningful work when not connected to the schoolserver or internet. Thia means knowing what data the learner needs to display and edit a map locally (even though the local store may be as low as 1GB) and how to specify that data to be downloaded to the Journal when connected. Tony On 06/15/2015 08:24 AM, iaep-requ...@lists.sugarlabs.org wrote: > Send IAEP mailing list submissions to > i...@lists.sugarlabs.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.sugarlabs.org/listinfo/iaep > or, via email, send a message with subject or body 'help' to > iaep-requ...@lists.sugarlabs.org > > You can reach the person managing the list at > iaep-ow...@lists.sugarlabs.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of IAEP digest..." > > > Today's Topics: > > 1. Re: [XSCE] RE: [UKids] Re: [support-gang] Taking >Open
Re: [Server-devel] [IAEP] [XSCE] RE: [UKids] Re: [support-gang] Taking, OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC
There seem to be two independent problems. First, make an updated version of the IIAB OSM. The goal is to understand what are the computer requirements to do this and how long it takes and then be able to make a new version whenever needed. Second, make available more detail than IIAB OSM provides for a region selected by a deployment while allowing the user (learner) to display, edit, and save it as a new map from this data. On the first problem, I am confused. I would guess the job is cpu intensive. However, many comments about SSD suggest it is i/o limited. In a former life, I worked on parallel programming in which the technical problem was to overlap disk i/o with computing. One thread bringing the data for the next case into memory while the processor worked on the current case. The trick was to determine the size of data needed to keep the processor busy and not waiting on i/o. The goal was to overlap i/o with processing to minimize processor idle time. I assume the processing of the world can be broken down into independent pieces (e.g. one tile at the global zoom level). The experiment with Nepal should show the processing time required as well as the disk i/o time. Based on that information, the work plan would be to load the global data on hard drive and then set up double buffered pipelines from hard disk to SSD to memory which can keep the processor busy. On the second, it is not clear that the greater detail needs to be OSM tiles. Something like Nick's map.activity/mappack with the map data on the school server (separate from OSM) and a client application (Sugar or Sugar-web or html) accessing the data, displaying it, and enabling the learner to add to it (à la GIS). After discovering that umapper is not umap, umap is clearly open source and could be a good base for the client application. This application should be able to get data from the school server and, as always, it should be possible for the learner to do meaningful work when not connected to the schoolserver or internet. Thia means knowing what data the learner needs to display and edit a map locally (even though the local store may be as low as 1GB) and how to specify that data to be downloaded to the Journal when connected. Tony On 06/15/2015 08:24 AM, iaep-requ...@lists.sugarlabs.org wrote: Send IAEP mailing list submissions to i...@lists.sugarlabs.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.sugarlabs.org/listinfo/iaep or, via email, send a message with subject or body 'help' to iaep-requ...@lists.sugarlabs.org You can reach the person managing the list at iaep-ow...@lists.sugarlabs.org When replying, please edit your Subject line so it is more specific than "Re: Contents of IAEP digest..." Today's Topics: 1. Re: [XSCE] RE: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC (Nick Doiron) -- Message: 1 Date: Sun, 14 Jun 2015 23:24:11 -0700 From: Nick Doiron To: xsce-de...@googlegroups.com Cc: Internet In a Box Working Group ,Unleash Kids! , J?r?me Gagnon-Voyer , Jaakko Helleranta , server-devel , iaep , "Community Support Volunteers -- who help respond to help AT laptop.org" Subject: Re: [IAEP] [XSCE] RE: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC Message-ID: Content-Type: text/plain; charset="utf-8" I don't think there's any technical issues with rendering the world at 10 and specific countries at 16, other than the human knowing where they can and cannot zoom -- Nick On Sun, Jun 14, 2015 at 11:58 AM, Tim Moody wrote: couple of observations: As expected, the new tiles have a lot more detail. There are more levels of zoom in the new ones. Some names have changed - the old map had Lalitpur and the new one has Patan (both are used) I don't see any boxes for unprintable characters, but there is a lot less Devanagari. (Google maps has more) Is it possible to merge individually generated regional tiles? for example if you rendered India and Nepal separately would you get both? What happens if you render the world at level 10 and then specific countries at 16? *From:* xsce-de...@googlegroups.com [mailto:xsce-de...@googlegroups.com] *On Behalf Of *Anish Mangal *Sent:* Saturday, June 13, 2015 10:06 PM *To:* J?r?me Gagnon-Voyer *Cc:* xsce-devel; Community Support Volunteers -- who help respond to help AT laptop.org; Unleash Kids!; server-devel; iaep; Internet In a Box Working Group; Jaakko Helleranta *Subject:* Re: [XSCE] RE: [UKids] Re: [support-gang] Taking OpenStreetMap Offline - DESIGN Call - Thur June 11, 10AM EDT / 2PM UTC Hi, Following from the skype call this week, I uploaded Nepal's OSM data in postgres
