On Fri, Oct 16, 2009 at 7:06 PM, Sean Gillies <[email protected]>wrote:
> GIS data would be herded through processing steps by the framework, which > calls your specific code at the right point. It probably allows you to chain > processing steps, split features into several streams, and merge them again. > Maybe it should allow one to distribute work among the machines of a network > a la Hadoop or octo.py. > I built a prototype a while back of doing GeoProcessing via MapReduce. The use case was things that ArcGIS sucks at: running a set of relatively simple "steps" over a large dataset (for those who know ArcGIS, it tends to grind away for many hours/days doing stuff, then crashes). The specific use case was figuring out the walking distance between every property in a city and the nearest park. The idea was to define relatively simple MapReduce processing steps in Javascript/Python. Then drop them into Hadoop, let it distribute each step, and have the steps chained together. Rob :)
