On Oct 15, 2009, at 6:00 PM, Eric Lemoine wrote:
On Thursday, October 15, 2009, Sean Gillies <[email protected]>
wrote:
Hi all,
I launched this group soon after chatting with a bunch of different
folks about feature processing. Mainly in the context of WPS
services. I don't do a lot of what GIS people call "geoprocessing"
these days, but enjoy thinking about it. I wonder if we might be
able to move past one-off scripts to frameworks for processing like
we've moved past one-off CGI scripts to web frameworks. What would
a geoprocessing framework look like? Does anyone have one they'd
like to introduce? FWIW, I think a WPS service isn't a framework by
itself, but could be backed up by a framework.
Hi Sean. What do you think makes processing web services different
than other web services?
Because processing web services take longer to respond? We could
envision using queues and polling if that makes sense from the client
perspective.
One thing that WPS has is chaining. That makes me think that WSGI
might be a potential architecture, with chain elements being
implemented as WSGI middleware.
Cheers,
A framework is something that runs my/your code. Write a Python
callable or class that carries out a specific feature or raster
processing task, register it with the framework, and it's executed at
the right step.
In Django, iirc (it's been awhile), you write a controller or view,
register it with the URL dispatcher, and it's executed when that URL
is dereferenced. In Zope, you can plugin in almost anywhere with the
application registry and the framework runs your code on URL
traversal, URL viewing, whatever. Safe's FME includes a framework that
will run Python scripts, right? I'm not at all familiar with its entry
points or API, but it seems to be there. I've never used it, so
correct me if I'm wrong.
This is the kind of framework I was thinking of, thought there are
certainly variations. GIS data would be herded through processing
steps by the framework, which calls your specific code at the right
point. It probably allows you to chain processing steps, split
features into several streams, and merge them again. Maybe it should
allow one to distribute work among the machines of a network a la
Hadoop or octo.py.
Jáchym Čepický's PyWPS will run your code that uses its entry
points, but isn't a full-fledged framework. If it permits chaining, in
Python, I didn't see it.
I've had an introduction to ZOO, but not the time to login and look
through the code. It uses the familiar unix pipe paradigm, but I got
the sense that you'd still have to roll your own framework.
WPS could be useful to expose occasionally used geoprocessing to
clients that have no GIS toolset at all, or to expose a proprietary
process (like a street routing algorithm) when you don't want to share
the code. If I worked in a data-processing shop, I'd want something
than runs on my own cluster of machines, with optimal performance and
agility. WPS wouldn't be part of that.
