On Tue, Jun 2, 2015 at 10:37 AM, Julian Atkinson <[email protected] > wrote:
> Hi All, > > We are attempting to integrate a WPS service into Geoserver 2.7.0 and have > some issues around resource management. > > Briefly, our WPS Process involves, > - Implementing a GeoServerProcess that uses the execute() method to > return a class deriving from RawData. > - Implementing the RawData.getInputStream() method to return an > InputStream object. This object holds refs to a Geotools transaction object > and a jdbc connection taken from a Geotools jdbcdatastore. These resources > are used to consume and transform data - which is returned via calls on the > InputStream's read() methods. > > We found that, > > - During synchronous WPS calls, if the remote client prematurely > terminates the connection, there is no corresponding call back on the > InputStream's close() method. Without this notification we are unable to > properly cleanup the geotools transaction and jdbc connection resources. > Sounds like a bug. Please open a ticket, and if you can, follow with a pull request to solve the issue. > > - As soon as the InputStream is returned to the WPS executor, the job is > marked as complete, even though it may take many hours to process and > stream all the data. Additionally, any new WPS jobs are run immediately > without respecting queue limits. > The queue is just for the "execute" phase. If you can change the code so that it also does also the encoding in the execute thread, that would be appreciated. > > With the above issues in mind, we modified our approach to use a > FileOutputStream created via resourceManager.getOutputResource(). The > prevents the WPS job from being prematurely marked as complete until all > data is written to the file. However it's not our preferred choice for > web-browsers since there's no indication of any connection/download > activity. > Marked as complete... where? In the WPS protocol, or in the status store? > > We also used the ProgressListener to monitor job state, by testing the > progressListener.isCanceled() while processing. However we noticed, > > - That if the remote client disconnects we still don't get notification of > completion via the ProgressListener. > Same as above, ticket and pull request welcomed. Mind though, a client disconnection cannot be determined as it happens, the servlets only tell us about the disconnection via an exception thrown as we try to write the output on the response output stream. > > > A couple of other minor issues, > - timeouts begin from the time the job is queued, not the time the job > run is started. > Yes, I can see that... does not seem like it's easy to change that though, we have nothing marking the actual execution beginning... well, unless we wrapped the process with something that notifies the MaxExecutionTimeListener and then cascades the execute call. It's probably the less painful way. Maybe this wrapper could also be a way to push the encoding in the execution thread pool. > - It's not possible to set the max synchronous jobs to zero via the GUI > as an indirect way to disable synchronous WPS jobs > This is indeed not possible, and not formally correct either: as far as I know, there is no way to communicate the client that a process cannot be executed in synchronous mode, so denying it would probably be a protocol violation. I can see how it's desirable though, some processes are just running for too much time to be usable in synch mode, but having a simple blanket statement that no process should be run in synch mode is too restrictive. I'd try to add a per process setting instead. > > > Any advice would be greatly appreciated, > > Cheers, > Julian > > > > University of Tasmania Electronic Communications Policy (December, 2014). > This email is confidential, and is for the intended recipient only. > Access, disclosure, copying, distribution, or reliance on any of it by > anyone outside the intended recipient organisation is prohibited and may be > a criminal offence. Please delete if obtained in error and email > confirmation to the sender. The views expressed in this email are not > necessarily the views of the University of Tasmania, unless clearly > intended otherwise. > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Geoserver-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-devel > > -- == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. -------------------------------------------------------
------------------------------------------------------------------------------
_______________________________________________ Geoserver-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-devel
