On Sat, Oct 29, 2011 at 1:36 PM, Jody Garnett <[email protected]> wrote:
> Hi,
> in the next weeks I'll be working to add asynchronous execution
> I'd like to give you some heads up and discuss some design details.
> support for GeoServer.
>
> That is actually a pretty big job; there is some support in GeoTools for
> "ProcessExecutor" with the idea
> that GeoServer should provide its own implementation (with suitable ability
> to audit currently running processes
> and kill any that get out of control).
> In particular a ProcessExecutor works with a subclass of Future called
> Progress. The Progress object allows status to be returned; with an eye
> towards fulfilling the Async response document.
> If it is not too much trouble I would encourage you to stick with this
> framework (with a custom GeoServer implementation). Mostly so processes that
> do chaining can be written at the GeoTools API
> level; and as long as they make use of the GeoServer provided
> ProcessExecutor we can still see what is going on.
Looked at that, seems quite a bit more than what I need, and harder to use
from the point of view of polling.
I'll stick to something more similar to what Gabriel did.
> The specification allows asynchronous requests when the caller asks for
> storeResponse=true and status=true, meaning the actual response document
> is stored somewhere and the status contained in it is updated while the
> process
> proceeds.
> (by spec, If status="true" and storeExecuteResponse is "false" then the
> service
> shall raise an exception)
>
> I was under the impression you could also ask that a result be stored to an
> FTP site or something. Perhaps that is part of "storedRespose=true".
The spec says you have to return a URL with the location of the document,
a FTP is unlikely what they had in mind.
There is a sequence diagram showing a FTP server for the results, but the
response to be updated is shown as stored on a HTTP server.
Both of them seem highly overkill to me in the simple case, though being
able to store the results (especially large ones) on a FTP server could
be a nice idea since it allows to restart downloads.
Anyways, a problem for another day, I'm not getting there right now :)
> The location of the document is reported in the execute response and then
> shall be updated while the process performs the computation.
> The spec does not say where this document should be located, but for ease
> of implementation I propose to make it into another service call, looking
> like:
> wps?service=WPS&version=1.0&request=executionStatus&identifier=xyz
> This makes it rather natural to implement with our current framework.
>
> See above; I was expecting the ProcessExecutor to track this information;
> and the Progress data structure be the core of this response document.
>
> Process wise, we already have the factories pass down a ProgressListener
> among the call arguments, so the process can update its status.
Yep, that's what I want to use. But Progress is no good, does not allow
for cancellation, does not have the notion of "paused" state
and forces the usage of Future, when doing restartable processes
against persistent storage and checkpoints people should be free to
use whatever they want imho.
A progress listener is all we need.
> Now, how to handle the process asynch execution and tracking?
> I was thinking to have a ProcessManager interface that the WPS service
> code submits processes to and can ask about their status too.
> It might look roughly like this:
> interface ProcessManager {
> /**
> * Submits the process for execution, returns a id to refer to the
> execution later
> */
> String submit(String processName, Map<String, Object> inputs);
> Status getStatus(String executionId);
> }
> Where Status is:
> Status {
> StatusType status; /* queued, paused, executing, complete, ... */
> double progress;
> Map<String, Object> output;
> }
> The default implementation of process manager would use a fixed size thread
> pool, callable and futures to handle the execution, but the interface will
> allow
> to plugin (from spring context) other custom managers.
> For example people might want to roll very long processes (several
> hours or more)
> that can be restarted from known checkpoints, in that case the manager
> would also
> need persistent storage of the processes in flight to allow the same
> to be resumed in case
> of a crash and restart of the WPS server, or enforce their own
> particular execution
> policies (e.g, link priority and amount of processes executed to the user).
>
> The design is the same; ProcessManager extends ProcessExecutor; Status
> extends Progress.
As said above, the interfaces over there do not seem suitable for what I need,
I need interfaces that talk about what can be done in a WPS server allow
for polling state and cancellation.
Afaik the GeoTools ones are probably suitable for a desktop tool instead, but
in practice nothing is using them?
If so maybe we can do like with the processes, port back reusable portions
to GeoTools once the dust is settled.
In any case I'll keep them on the desk along with Gabriel work and see what
I can learn from them while building asynch support for GeoServer
Cheers
Andrea
--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 962313
http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf
-------------------------------------------------------
------------------------------------------------------------------------------
Get your Android app more play: Bring it to the BlackBerry PlayBook
in minutes. BlackBerry App World™ now supports Android™ Apps
for the BlackBerry® PlayBook™. Discover just how easy and simple
it is! http://p.sf.net/sfu/android-dev2dev
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
