On Tue, Aug 11, 2009 at 11:58:57PM +0200, andrzej zaborowski wrote: > 2009/8/11 MilesTogoe <[email protected]>: > > On 08/11/2009 10:27 AM, Jeffrey Johnson wrote: > >> I concur. The largest obstacle here in infrastructure. I have recently > >> completed processing and tiling 10s of TB of imagery for DoD and just > >> moving > >> that amount of data around is costly and complicated, let alone trying to > >> serve it. > >> > >> Amazon has indicated they would be willing to help us with this kind of > >> project, but I agree that a distributed registry approach makes the most > >> sense in the short/medium term, and think we should focus our efforts > >> there. > > > > I also think it is a good idea to distribute the server data regionally to > > spread the storage requirements. > > Agreed, and I think a "crowdsourced storage" approach is worth > considering. Lots of people have apache running on their desktops > with a not too fast internet connection but with a 1.0 or 1.5TB > desktop-class disk. They'd just need to run a server side script that > could also accept tile uploads from some kind of distpatcher.
The problem with "many low reliability machines" -- with an insuficiently large number of 'many' -- is that you trade latency, the key feature in many web mapping apps. I have spent a lot of time getting tiles in a map to load 1 second faster by request from customers. In cases when a single host is unavailable, you have to try that, then fall back to something else... increasing latency significantly. It's an interesting approach, but after soem experimenting with it for TileCache, we decided that it probably wasn't practical for the common cases. This isn't a blanket refusal to pursue this approach, just an explanation of some things that need to be considered. -- Chris > On the client side the dispatching could be done inside the browser, > by computing some simple hash in javascript, so as to avoid asking a > central server for the address of every tile. The tiles should not be > mapped to servers (people's desktops) geographically but instead using > a hash function so that e.g. when someone browses Germany imagery all > his requests don't go to a single computer, and instead every desktop > hosts a tiny bit of Germany and a tiny bit of Massachuetts. If > there's enough hosts then there could be 10 or 20 computers hosting > copies of each tile and the central server serving the website would > just send the mapping of hash values to a subset of mirrors, which > would be rotated. > > Cheers > > _______________________________________________ > talk mailing list > [email protected] > http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org -- Christopher Schmidt Web Developer _______________________________________________ talk mailing list [email protected] http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org
