Daniel, all good points. Best, Andre.
On Mon, Sep 15, 2014 at 8:46 AM, Daniel Gruber <[email protected]> wrote: > As you indicated, Peter, it is not as easy as it sounds to add all job > template attributes > to the job info object. As mentioned before I think in most systems you can > change the > submission attributes after submission (in Grid Engine we have qalter, JSV, > job classes, > sge_request files, the -@ parameter, path aliasing) therefore we should be > conservative and not > specify that the submission parameters in job info needs to match 100% the > job template. > > Andre: You don’t need to keep the job template at this point in time anyhow. > There is a > get job template method specified on the job object itself. > > When it comes to the Univa Grid Engine implementation, I’m storing the job > template > in the library when a submission is done. When you leave the application and > do a > re-connect the behavior is unspecified (meaning there is no job template > available > anymore). While there is now a way in Univa Grid Engine 8.2 to get the > information > out of qstat (you can see the qsub request line in qstat and qacct) I would > not rely on > this to be available on all systems. Even when a 3rd party DRMAA2 > implementation > relay on parsing this out, this is highly error prone (as new parameters and > syntax > changes can come in, which makes an implementation unusable). > > Also, we need to consider that the job can be either submitted from the > drmaa2 app > (where we have the job template) and in a monitoring session the job can be > from any source (where we don’t have the job template). Having here a bunch of > attributes which finally are optional and not available makes the interface > not > attractive. The job info object is extensible hence each implementation can > decided > what to add and what not as the last way out to get something like a job name > into job info. > > Just one example to make it more clearer: > > You can submit a job with an certain output path in job template but on > the machine where it runs the output path could be translated in something > different, probably because of a different mount point. In UGE we have > something called path aliasing, which is done by the system transparently > to the user. Now which output path would be correct? > When the job template output path is right, then the question > is if it is catchable by all systems in a general way with job status calls? > When the real output path is right then the question is do we get that one > for all systems? I don’t think we can specify which one is the right on > DRMAA level hence we would need to make all of them optional or implementation > specific. Hence the implementation specific approach seems more > natural for me. > > Due to the fact that we have already a method for returning the job template, > and in order to have small consistent set of attributes in job info, I > would really not go the way to do such a major change. > > Why I added this *job name* request in the job info object is because > the job name (rather then other attributes) is probably the most > natural attribute of a job. In Univa Grid Engine you can do a lot of > useful things with it: defining job dependencies (or multi-job dependencies), > using is for a grouped qstat etc. > > Short: > - I just want to add a job name in job info since it is one of the most > basic job info attributes > - I do not want to specify whether the job name matches the job name > of the job template or not. I would only specify that this is the job name > the system considers as the job name - and as a fall back to have it > available in all implementations it could be also just the job id. > - I do not want to add all job template attributes to the job info object > since most of them we don’t need for filtering in real live and due to > the fact that they would be optional anyhow. Hence implementation > specific extensions is the way to go IMHO. > - I can also live with the fact that job name is implementation specific > but I really see the job name very basic and really worthy for job > filtering (especially when you want to group jobs like in UGE case). > > Sorry the delayed answer > > Cheers > > Daniel > > Am 14.09.2014 um 20:53 schrieb Peter Tröger <[email protected]>: > >> I agree to Bruno and Andre that the advantages are worth it. Performance >> considerations should be done in the language bindings and / or the >> implementations, the root spec is the wrong place for that. I therefore vote >> for the simple addition all job templates attributes to the JobInfo >> structure. >> >> An interesting side aspect are implementation-specific job template >> attributes. I would prefer to leave it to the implementors if they want to >> support them also in the JobInfo structure. >> >> If Daniel agrees to all of this, especially from his implementation >> viewpoint in Grid Engine 8.2, then I would update the root spec and the C >> binding accordingly. >> >> Best regards, >> Peter. >> >> >> Am 12.09.2014 um 22:18 schrieb Andre Merzky <[email protected]>: >> >>> I think the job template information are easy to add, and make app >>> writing easier (I don't need to keep the old submitted job template >>> around...). Only drawback I see is increase in memory footprint when >>> used to report on many jobs -- but when used as a filtering >>> instruction, this should not matter. >>> >>> >>> My $0,02, Andre. >>> >>> On Wed, Sep 10, 2014 at 7:48 PM, Peter Tröger <[email protected]> wrote: >>>> Dear all, >>>> >>>> at OGF-42, we stumbled over the issue that „JobInfo“ does not contain the >>>> original job name being used in the template: >>>> >>>> http://redmine.ogf.org/issues/102 >>>> >>>> This leads to the situation that you cannot filter for it when doing >>>> monitoring. >>>> >>>> Although we quickly decided to add it, I meanwhile figured out that this >>>> is a deeper problem. JobInfo intentionally contains nothing from the job >>>> template at the moment, which makes perfectly sense if you see it as a >>>> pure reporting data structure. You don’t need to fetch information that >>>> you handed in by yourself as the calling application. >>>> >>>> For filtering, however, you obviously want to have all of the template >>>> attributes being usable. This is not possible at the moment. >>>> >>>> Another case is the „change job template after submission“ magic in Grid >>>> Engine, which could lead to the fact that your job has different >>>> attributes from what you originally specified. You may also want to see >>>> that in the JobInfo structure. >>>> >>>> All in all, it seems like that „JobInfo“ needs all of attributes from >>>> „JobTemplate“ too, not only the job name. >>>> >>>> What do you think ? >>>> >>>> Best regards, >>>> Peter. >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> drmaa-wg mailing list >>>> [email protected] >>>> https://www.ogf.org/mailman/listinfo/drmaa-wg >>> >>> >>> >>> -- >>> It was a sad and disappointing day when I discovered that my Universal >>> Remote Control did not, in fact, control the Universe. (Not even >>> remotely.) >> > -- It was a sad and disappointing day when I discovered that my Universal Remote Control did not, in fact, control the Universe. (Not even remotely.) -- drmaa-wg mailing list [email protected] https://www.ogf.org/mailman/listinfo/drmaa-wg
