Hey, this is great. Say what you will about voting and processes, but you guys are churning out great ideas. Why don't we break down this very large set of linked problems, and get volunteers to take a lead on each angle.
We can break the problem down into the following parts: 1) Implementing the catalog server. Needs to be written. Has to be hosted somewhere to be tested. Fortunately, there's already the start of a codebase in Github, and I can host a demo instance on the freemap.in server while we work out the features. Until we have serious storage, we can still fill out the catalog with WMS servers to cache, and do a modicum of local caching. That way, we can derive near-term benefit while the bigger technical issues are worked out. 2) Figuring out the cheapest way to get dozens, hundreds, or even thousands of terabytes. Find options that will scale horizontally, with minimal points of failure. Evaluate all options, including Amazon and this cool Backblaze thing that Richard posted. Let's plan for petabytes, so let's figure out the costs and prepare a funding proposal. 3) Finding potential funders or hosts. Say -- *just for example* -- we got $15k for a pair of Backblaze machines, we'd have 130 TB on RAID-6 in 12U. That wouldn't be a bad start. Where can we get money to buy the things? Who will host them, and, ideally, provide free bandwidth? What kind of SLA will we have with the host? Can we get two geographically distributed hosts for redundancy? Who will fix things when they break? Surely the answer to all these questions is ultimately affirmative in some way; someone just has to do the leg work to sort it out. 4) Start thinking of ways to distribute things across the Internet, so that we can start building *some* kind of cache without first needing tens of thousands of dollars in hardware and hosting. This can happen after #1 is well along, but that way we don't wind up stuck waiting on #2 or #3. Did I leave anything out? Please, rather than wearing ourselves figuring out what we should do, pick a sub-problem that you think is worthwhile and interests you, and start telling us what you're doing about it. Note that not all the problems are technical, e.g. #3 above, so there's room for people to engage lots of different skills. For myself, I plan to start contributing on item #1 while on holiday next week. The proposed OAM project is huge in potential scope, bigger in volume than anything I've ever built personally. But I'm confident that we have the combined talent on this list to find a way to build it. SDE _______________________________________________ talk mailing list talk@openaerialmap.org http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org
