On Wed, 2008-02-13 at 15:06 -0200, Matias Alberto Gavinowich wrote: > Thanks again! > > I also see what you say about the background services, though I am > intrigued as to why I would *not* want to do it. That is, I've seen > several portlets that don't use abstractions and persist the url > handle from GramJob.
The difference I can see is that it's easier to serialize a URL to disk than some cog object tree. Which means one can more easily persist state if tomcat is shut down and then re-started. > But with this schema I could use the > abstractions, and it would only require me to have the listeners > living in an object in the server, and it seems a doable approach. I > wonder why it seems that this is not so common, are there any > disadvantages I am missing or may it be just that the abstractions are > relatively new and there are not so many portlets that work on them > yet? How are you measuring how common it is? For example, as far as I understand, GridSphere and its GridPortlets have a similar scheme though they don't use the cog abstractions but instead have their own. > > If I didn't use the abstractions and want to do some multi step tasks, > then I guess it means I would have to build the graph 'by hand' in the > RSL right? Seems more complicated at first. I guess the expression of mutliple tasks combined would be called "workflow" in many places. And there are many choices there. > > I also wonder what the handle returned by the abstractions (the > urn:cog-1234567890123 one) could be used for. It's a unique id. Mihael > > Thanks a lot again for all the info you provided, was really useful. > > Matt > > > On Feb 12, 2008 7:20 PM, Mihael Hategan <[EMAIL PROTECTED]> wrote: > > > > On Tue, 2008-02-12 at 19:04 -0200, Matias Alberto Gavinowich wrote: > > > Hi, > > > > > > Thanks for the answers I received. > > > > > > I see how the schema your mention would work, although I am not sure > > > how I would implement it. I envision a web service that could receive > > > a listener as a parameter and return some kind of id to retrieve it, > > > or perhaps something based on the whole collections of jobs for a > > > user, which would resemble your example more closely. I guess - I am > > > not sure - that this would imply making the listener also > > > serializable, and using a stateful web service instead of a plainly > > > standard one, so it would have to live in the GT4 container. Is what I > > > described you meant? > > > > When I said "service" I didn't necessarily mean "newtwork service". It > > can be a static hashtable. > > > > > > > > As for GT2 vs GT4, I am a bit lost with the GramJob classes, I am > > > reading mainly the javadoc at > > > http://www.cogkit.org/release/4_1_4/api/jglobus/index.html and also > > > some others I found in other sites like globus, but I am not really > > > sure which are pre-WS (GT2) and which are WS (GT4). > > > > Jglobus only deals with pre-WS gram. For the ws-gram GramJob see > > http://www-unix.globus.org/api/javadoc-4.0/globus_wsrf_gram_client_java/. > > > > > > > > I put some further thought into the problem and I think I would run > > > into some more problems with TaskGraphs (from Abstractions). I hoped I > > > could do a kind of full monitoring of the status so that if a user > > > submitted a TaskGraph I would be able to tell which task in the > > > taskgraph was executing and the progress corresponding to that > > > specific task, and present this to the user in the portal. Perhaps I > > > trying to squeeze more out of it than I should? It would be nice to be > > > able to submit complex (composed) tasks to the grid and provide that > > > kind of granular monitoring, so I would be able to tell the user 'Your > > > job has staged the input files, run, and is now staging the output > > > files back to your directory, and this last step is complete at 65%'. > > > > I'd say start with the simple things first. They seem to have a tendency > > to be more complicated than they appear. > > > > Mihael > > > > > > > > > > By the way, thanks for letting me know I should not be using the gt2 > > > fault tolerant provider, you saved me some time trying to reedit my > > > experiments with it. > > > > > > Regards, > > > > > > Matt > > > > > > > > > On Feb 12, 2008 4:10 PM, Mihael Hategan <[EMAIL PROTECTED]> wrote: > > > > So I think your best bet is to have some background services in the > > > > portal that you attach or detach to a portal session. In other words, > > > > you can have the CoG objects alive even when a user logs out. So > > > > basically a session of your own that is only loosely connected to the > > > > portal session. This way you get the benefits of both worlds. For > > > > example: > > > > > > > > -login happens > > > > register(username, statefulObject); > > > > -logout happens > > > > -stuff happens in the background > > > > -login happens > > > > statefulObject = retrieve(username) > > > > ... > > > > > > > > Some more specific answers inline. > > > > > > > > > > > > On Tue, 2008-02-12 at 14:38 -0200, Matias Alberto Gavinowich wrote: > > > > > Hi, > > > > > > > > > > I am writing to see if anyone can give me any pointers on monitoring > > > > > file transfer and job submission tasks on the Grid. What I would like > > > > > to do is to be able to present detailed information such as status of > > > > > the job (ideally even if sent to a PBS batch system) and progress > > > > > (e.g. in percentage) of a file transfer, even if the user logs out and > > > > > then back in into the portal (so registering a listener in the session > > > > > is not enough in my case). The monitoring that the batch job > > > > > submission portlet in OGCE provides is pretty much what I want for the > > > > > job case. > > > > > > > > > > Now the issue I am having with this: I am trying to apply the Java CoG > > > > > Kit abstractions. That is, I have studied the code for the batch job > > > > > submission portlet and I know it employs the GramJob class in the > > > > > (jGlobus?) part of the CoG API. I haven't tried to do this for file > > > > > transfers so I also don't know how it would be done in that case. > > > > > > > > > > Nevertheless, using the abstractions I was only able to register a > > > > > listener in the session, which constrained me mainly to synchronous or > > > > > almost synchronous executions. What I like about the abstractions is > > > > > their simplicity and implementation of object oriented concepts. If I > > > > > used e.g. the GramJob class, then as far as I know I need to build the > > > > > rsl. > > > > > > > > > > These are therefore the main items I am in doubt about: > > > > > > > > > > Is it possible to use the CoG Abstractions for the purpose I > > > > > described? Or at least is there a way to get the native job handle > > > > > from the abstractions and build a GramJob object just for the purpose > > > > > of monitoring the task, that is, combine both techniques? The only > > > > > handle I could get my hands on was a urn:cog handle that I didn't know > > > > > what to use for. > > > > > > > > The internals of the implementation are not much exposed by the > > > > abstractions API. So you'd probably have to use the implementation APIs > > > > directly. > > > > > > > > > > > > > > I am not sure the extent up to which GramJob works with GT4 (WS-based) > > > > > execution, > > > > > > > > It doesn't. But GT4 has a GramJob of its own. Though fairly different > > > > from the pre-ws gram one. > > > > > > > > > I have seen that a GramJobPreWS object is also used in the > > > > > portlet I studied, can I get this GramJob class to work with any > > > > > provider (at least any globus provider, namely gt2 and gt4)? > > > > > > > > > > How would this be done for file transfers? > > > > > > > > Same principles apply. > > > > > > > > > > > > > > I would greatly appreciate your comments on these topics. If there's a > > > > > way to combine approaches so as to get the advantages of using the > > > > > abstractions, that would be great. If anyone has further examples on > > > > > how to achieve such monitoring without the abstractions, that would > > > > > also be helpful. Even examples on monitoring using the abstractions > > > > > only in the same session - though I would really like to be able to > > > > > span sessions - would come in handy. > > > > > > > > > > I'd like to add I once tried to do this by using the gt2 fault > > > > > tolerant provider and trying to checkpoint and reconnect to tasks, but > > > > > I was never able to get this to work, even from the command line. > > > > > > > > Hmm. I just removed that one from SVN. Hasn't been maintained in a while > > > > and you're the only person I heard of to have tried to use it. > > > > > > > > Mihael > > > > > > > > > > > > > > > > > > Thank you a lot for your help, > > > > > > > > > > Matt > > > > > > > > > > > > > > > > > > > > >
