> I would like to simplify even more problem description. Basically I have X agents. I need to call a "manager" that tells me "there is a free agent". Then I tell the "manager" to run a software on that agent. If there are no more free agents I wait.
This is exactly how GoCD works - Manager (the GoCD Server) has X agents configured, foreach job (pipeline / stage / job) that you trigger, the Manager waits until it finds a free agent and executes it there. What to execute is specified as tasks within the pipeline / stage / job spec. On Wed, 2 Mar 2022 at 22:27, Mario Giammarco <mgiamma...@gmail.com> wrote: > I would like to simplify even more problem description. Basically I have X > agents. I need to call a "manager" that tells me "there is a free agent". > Then I tell the "manager" to run a software on that agent. If there are no > more free agents I wait. > > Il giorno mercoledì 2 marzo 2022 alle 15:38:54 UTC+1 Mario Giammarco ha > scritto: > >> If I create several pipelines from a template can I run them in multiple >> agents? >> >> Il giorno mercoledì 2 marzo 2022 alle 15:28:15 UTC+1 Mario Giammarco ha >> scritto: >> >>> Il giorno martedì 1 marzo 2022 alle 16:26:54 UTC+1 Chad Wilson ha >>> scritto: >>> >>>> >>>> >>>> - separate *jobs* within the same stage of a single pipeline that >>>> override env vars differently at job scope, but execute the same tasks >>>> in >>>> each job >>>> - separate *pipelines* entirely, that have the same material >>>> inputs/triggers, but have different configuration for the >>>> stage/job/tasks >>>> >>>> >>>> >>> Regarding this, can you confirm that I can use same material with >>> different pipelines? I have tried ONLY with web gui and it seems it ignore >>> existing material configuration during creation of a new pipeline. >>> >>> >>>> On Tue, Mar 1, 2022 at 10:43 PM Mario Giammarco <mgiam...@gmail.com> >>>> wrote: >>>> >>>>> Sorry I need to change enviroment variables not parameters. >>>>> >>>>> Il giorno martedì 1 marzo 2022 alle 15:38:09 UTC+1 Mario Giammarco ha >>>>> scritto: >>>>> >>>>>> Hi, >>>>>> I resurrect this thread. >>>>>> I have one pipeline and several agents. I need to run the pipeline >>>>>> multiple times in parallel with different parameters. >>>>>> It seems to me a common and easy question. >>>>>> Reading this thread, bug description and other threads it seems it is >>>>>> not possible. >>>>>> I am really surprised about it. >>>>>> Is it really not possible? >>>>>> Is there a workaround? >>>>>> Thanks, >>>>>> Mario >>>>>> >>>>>> Il giorno lunedì 3 ottobre 2016 alle 08:00:08 UTC+2 Zabil C M ha >>>>>> scritto: >>>>>> >>>>>>> There's an issue logged for this here >>>>>>> https://github.com/gocd/gocd/issues/2329 >>>>>>> >>>>>>> In short, you can't do this at the moment but we are willing to help >>>>>>> out with and merge this feature if someone picks it up. >>>>>>> >>>>>>> On Wed, Sep 28, 2016 at 3:48 PM, <eran....@riskified.com> wrote: >>>>>>> >>>>>>>> One of the key features that made us decide to migrate to gocd >>>>>>>> several months ago was the idea that pipelines are able to run in >>>>>>>> parallel >>>>>>>> and that more than one instance of a pipeline can be started. >>>>>>>> Only now are we starting to realise that actually the same pipeline >>>>>>>> cannot be executed multiple times in parallel since each stage will >>>>>>>> always >>>>>>>> run sequentially even if it's the same stage in different pipeline >>>>>>>> instances. I'm stating this only based on this post >>>>>>>> <https://groups.google.com/d/msg/go-cd/E0goLJITLy8/8UN7KrjCCAAJ> which >>>>>>>> is the only mention of this issue that I've been able to find. >>>>>>>> >>>>>>>> So first off I'd like to verify if this is correct. Is it not >>>>>>>> possible to run several instances of the same pipeline concurrently (in >>>>>>>> parallel )without one instance being constrained by another (i.e. the >>>>>>>> second instance of the same pipeline is able to complete even if a >>>>>>>> previously initiated instance has not done so)? >>>>>>>> >>>>>>>> If this is the case, I'd really appreciate any help/ideas on >>>>>>>> overcoming this limitation in some way. >>>>>>>> The pipeline I'm working on has only one stage which simply runs a >>>>>>>> docker and then deletes the container and image. The docker does some >>>>>>>> work >>>>>>>> on our machine learning models, there is no problem to run several >>>>>>>> containers of this docker at the same time - and that's exactly what >>>>>>>> I'd >>>>>>>> like to do - i.e. run another docker each time that the pipeline is >>>>>>>> triggered (we're using the api to trigger it). >>>>>>>> I'd like to see the output created by each docker and, of course, >>>>>>>> see whether each pipeline has finished successfully or failed. The >>>>>>>> order of >>>>>>>> execution and even the material version is not relevant, each instance >>>>>>>> of >>>>>>>> the pipeline/docker has it's job to do. >>>>>>>> >>>>>>>> Finally, I'd like to ask if you guys feel that a feature that >>>>>>>> enables the same stage to run concurrently in different pipeline >>>>>>>> instances >>>>>>>> is feasible. And that a request for this is something which has a >>>>>>>> chance >>>>>>>> of being accepted (I will, of course, contribute anything I can within >>>>>>>> my >>>>>>>> technical skills) . >>>>>>>> I'm sure that the are many use cases that will benefit from such a >>>>>>>> feature. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "go-cd" group. >>>>>>>> >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to go-cd+un...@googlegroups.com. >>>>>>> >>>>>>> >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "go-cd" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to go-cd+un...@googlegroups.com. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/go-cd/b9f673a6-01ba-4950-9d0e-9cafc2151014n%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/go-cd/b9f673a6-01ba-4950-9d0e-9cafc2151014n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/c848aeb6-3fab-400a-8279-911a9eb1eca1n%40googlegroups.com > <https://groups.google.com/d/msgid/go-cd/c848aeb6-3fab-400a-8279-911a9eb1eca1n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Ashwanth Kumar / ashwanthkumar.in -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAD9m7CyTEtSOYS6H%3Dxhtf_jQbHP%3D27%3DUqkUkKUnbUU9OhC8%2BPg%40mail.gmail.com.
