Am 23.02.2012 um 10:38 schrieb William Hay:
> On 23 February 2012 09:31, Reuti <[email protected]> wrote:
>> Am 23.02.2012 um 10:01 schrieb William Hay:
>>
>>> On 23 February 2012 00:36, Maes, Richard <[email protected]> wrote:
>>>> Reuti,
>>>> For the example below where you spec which PE to instantiate.
>>>>> $ qsub -pe ixia* 1 job.sh
>>>>
>>>> Can this accept something other than wildcards? Is there a way to make
>>>> it do REGEX? Or ranges?
>>>
>>>
>>>> For a case where I have Ixia1, Ixia2, and Ixia3, and lets says I want
>>>> one group of tests to use Ixia1 and Ixia2. I have a second group of
>>>> tests I want to run on Ixia3,
>>>> I would be cool to do
>>>> Qsub -pe ixia[12] 1 job.sh
>>> You can transform the pe name into that from a server side JSV and it
>>> works as you would want.
>>> However it will be rejected on job submission so you would need some
>>> other way to pass this in
>>> if you want to allow the user to set it possibly a context variable.
>>> This falls under the heading of undocumented (and I'm sure unsupported)
>>> feature.
>>
>> Why should this be undocumented? -ac is documented.
>>
> That the pe is interpreted as a full pattern (per sge_types) which can
> be set to ixia[12]
> from the server side JSV is the undocumented part. Sorry if I was unclear.
Argh, this was an extension beyond 6.2u5 and the pe_name can be any object_name
also on the command line IIRC. You can request ixia[12] to get one of them, but
not both (unless you transform it by the JSV). The sge_types list only
object_name for it.
For you it's working in the JSV but not on the command line - which version of
SGE are you using?
-- Reuti
> William
>
>
>>
>>
>>>> Or even qsub -pe {ixia1,ixia2} 1 job.sh,
>>> Alternatively you might just associate each PE with a separate queue
>>> and restrict the queues that
>>> the job is allowed to run in (if they're on the same hosts this might
>>> necessitate additional changes to
>>> ensure they don't get oversubscribed).
>>>>
>>>> But I don't think either of those works.
>>>> Rich
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Reuti [mailto:[email protected]]
>>>> Sent: Friday, February 17, 2012 5:34 PM
>>>> To: Maes, Richard
>>>> Cc: [email protected]
>>>> Subject: Re: [gridengine users] Tricky consumables problem
>>>>
>>>> Am 18.02.2012 um 02:24 schrieb Maes, Richard:
>>>>
>>>>> To answer your question about layout. All nodes can talk to the
>>>> single
>>>>> Ixia. There is a diagram below.
>>>>>
>>>>> To confirm what you are saying here
>>>>>> limit pes {ixia1,ixia2,ixia3} to jobs=1
>>>>>> ("jobs" complex setup as a JOB consumable from an arbitrary high
>>>>
>>>> JOB instead of consumable YES
>>>>
>>>>> value in the global configuration)
>>>>>
>>>>> Are you saying I need to create jobs in the cluster configuration ->
>>>>> global host? And set it to 999999 for instance?
>>>>
>>>> Exactly, this way you can limit "jobs" opposed to "slots" in case it's
>>>> necessary by RQS for users, pes, queues, hosts. As you used only one
>>>> slot in your former setup, I wasn't sure how you would run parallel jobs
>>>> then (besides using just a granted node completly for this one-slot job
>>>> then).
>>>>
>>>> -- Reuti
>>>>
>>>>> ___________
>>>>> [ WA-GRID ]
>>>>> [queuemaster] ___________________
>>>>> [wasim01 ] <-NETWORK-> | WA-IXIA-01 |
>>>>> [wasim02 ] |slot 1 - ports 1 -4| <-- Eth x 4 -->
>>>> DUT
>>>>> 1
>>>>> [wasim03 ] |slot 2 - ports 1 -4| <-- Eth x 4 -->
>>>> DUT
>>>>> 2
>>>>> [wasim04 ] |slot 3 - ports 1 -4| <-- Eth x 4 -->
>>>> DUT
>>>>> 3
>>>>> [wasim05 ] ____________________
>>>>> [wasim06 ]
>>>>> [wasim07 ]
>>>>> [wasim08 ]
>>>>> ____________
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Reuti [mailto:[email protected]]
>>>>> Sent: Friday, February 17, 2012 5:05 PM
>>>>> To: Maes, Richard
>>>>> Cc: [email protected]
>>>>> Subject: Re: [gridengine users] Tricky consumables problem
>>>>>
>>>>> Hi Richard,
>>>>>
>>>>> Am 18.02.2012 um 01:28 schrieb Maes, Richard:
>>>>>
>>>>> I am using gridengine to load tests on to an Ixia. Only one test can
>>>>>> run at time so I configured the queue to resource quota configuration
>>>>>> with a rule that says limit users * queues lt_np_13 to slots = 1
>>>>>
>>>>> as it's for all users, the "users *" can be left out here.
>>>>>
>>>>>
>>>>>> This has worked fine for years.
>>>>>>
>>>>>> Now, however I have many Ixia blades and I still want to use
>>>>> gridengine
>>>>>> to distribute jobs to the cluster nodes, but ultimately, there can
>>>>> only
>>>>>> be one test running per ixia blade.
>>>>>
>>>>> I don't understand the detailed layout of your cluster. There are
>>>>> dedicated nodes connected to each ixia blade?
>>>>>
>>>>> limit hosts @ixia1 to slots=1
>>>>> limit hosts @ixia2 to slots=1
>>>>> limit hosts @ixia3 to slots=1
>>>>>
>>>>> with three hostgroups would do? Or are all nodes connected with IXIA
>>>> and
>>>>> could use any, but only one as you wrote:
>>>>>
>>>>> Three PEs with a slot limit of one inside the PE could work, and this
>>>>> could be requested by a wildcard:
>>>>>
>>>>> $ qsub -pe ixia* 1 job.sh
>>>>>
>>>>> The PE name you get inside the job and can select the proper config
>>>>> file. If these are already parallel jobs, this could be put in an RQS
>>>>> then:
>>>>>
>>>>> limit pes {ixia1,ixia2,ixia3} to jobs=1
>>>>>
>>>>> ("jobs" complex setup as a JOB consumable from an arbitrary high value
>>>>> in the global configuration)
>>>>>
>>>>> -- Reuti
>>>>>
>>>>>
>>>>>> I think what I need to do is to use a consumable attribute. A
>>>> command
>>>>>> like qsub -l ixiablade1=1 myjob.sh, will take a consumable resource
>>>>> for
>>>>>> ixiablade1.
>>>>>> However, lets' say I have three ixia blades, ixiablade1, 2 and 3.
>>>>>> Somehow I need to have a way of saying you can take ixiablade1 or you
>>>>>> can take ixiablade2 or you can take ixiablade3, but you can only take
>>>>>> one. Presumably, tests will have to wait until one of the three
>>>>>> consumables is available.
>>>>>>
>>>>>>
>>>>>> Here is a diagram
>>>>>>
>>>>>> [queuemaster]
>>>>>> [wasim01] <-----NETWORK -----> IXIA[wa-ixia-01]
>>>>>> [wasim02] [slot 1 - ports 1 -4] <--
>>>>>> Direct Ethernet x 4 --> DUT 1
>>>>>> [wasim03] [slot 2 - ports 1 -4] <--
>>>>>> Direct Ethernet x 4 --> DUT 2
>>>>>> [wasim04] [slot 3 - ports 1 -4] <--
>>>>>> Direct Ethernet x 4 --> DUT 3
>>>>>> [wasim05]
>>>>>> [wasim06]
>>>>>> [wasim07]
>>>>>> [wasim08]
>>>>>>
>>>>>> For example, I may have 100 individual tests queued to run on the
>>>>> grid,
>>>>>> across 8 different machines, and because of resource quota or
>>>>>> consumables, only three jobs can run at a time. When a job moves
>>>> from
>>>>>> the waiting to running, it will be because one of the consumables was
>>>>>> available (and now taken again). When the test begins execution, the
>>>>>> test will examine the environment to determine which consumable it
>>>> was
>>>>>> given and use that to select the appropriate config file (For
>>>>> slot1.cfg,
>>>>>> slot2.cfg or slot3.cfg) which defines port connections and other DUT
>>>>>> related configuration information.
>>>>>>
>>>>>> Any thoughts would be appreciated.
>>>>>> Thanks
>>>>>> Rich
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> [email protected]
>>>>>> https://gridengine.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> [email protected]
>>>> https://gridengine.org/mailman/listinfo/users
>>>>
>>>>
>>>
>>
>>
>>
>
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
