Thank you Marques, I really appreciate your help and support. You've been very responsive. I'm looking for an alternative to Jenkins. Unfortunately we have really large repos so most existing CI tools don't work out of the box because of the clone issues. I really like GoCD and have enjoyed creating a POC for it. I don't want to drop it just yet. I'll continue to look for a workaround.
Thanks again On Mon, May 10, 2021 at 7:12 PM Marques Lee <[email protected]> wrote: > Ok, fair enough :). Let us know if we can help in any way. > > On Mon, May 10, 2021 at 7:06 PM HUSSEIN KADIRI <[email protected]> wrote: > >> Let me explore the local clone option some more >> $ git clone /local/path/to/bare/repo >> >> Thanks for the tip >> On Monday, 10 May 2021 at 19:01:34 UTC-7 HUSSEIN KADIRI wrote: >> >>> You're right the agent won't need read write. The volume is what needs >>> to be setup as read write. See doc >>> <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes> >>> >>> Write - would be the "cron" updating the repo >>> Read - would be the agents "cloning" >>> Reading and Writing can't happen simultaneously. >>> >>> On Monday, 10 May 2021 at 18:55:21 UTC-7 Marques Lee wrote: >>> >>>> Typo — “why would the agents need read/write” was what I meant >>>> >>>> On Mon, May 10, 2021 at 6:53 PM Marques Lee <[email protected]> >>>> wrote: >>>> >>>>> Pardon my lack of familiarity with GKE, my question may be extremely >>>>> naive - if there’s only process that needs to write to the mount (the cron >>>>> that performs the fetch to keep the code updated), then by would the >>>>> agents >>>>> need read/write on the volume? Isn’t read-only enough to clone for a >>>>> pipeline run? >>>>> >>>>> On Mon, May 10, 2021 at 6:48 PM HUSSEIN KADIRI <[email protected]> >>>>> wrote: >>>>> >>>>>> Hmm, we setup GoCD on kubernetes so the server is a kubernetes >>>>>> deployment. >>>>>> >>>>>> We are on GKE . Yes we can mount a volume but GKE PVC are only >>>>>> ReadWriteOnce (One pod mounted to read and write) or ReadOnlyMany >>>>>> (Multiple >>>>>> pods mounted and reading only from the volume). >>>>>> >>>>>> Your recommendation requires a ReadWriteMany setup which is not >>>>>> possible in GKE. >>>>>> >>>>>> On Mon, May 10, 2021, 6:43 PM Marques Lee <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hmm, unfortunately I don’t believe the git material in GoCD has >>>>>>> native support for reference repos. >>>>>>> >>>>>>> Could the server not run a cron to do git fetch —all on the bare >>>>>>> repo to keep it updated? >>>>>>> >>>>>>> Then all agents would pick up new changes so long as the volume >>>>>>> holding the bare repo is mounted to agents. >>>>>>> >>>>>>> Otherwise, yeah, GoCD may not be a good fit for you until we build >>>>>>> in support for —reference-repo if that’s a deal breaker. >>>>>>> >>>>>>> On Mon, May 10, 2021 at 6:37 PM HUSSEIN KADIRI <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I have elastic agents so the cron route would not be feasible. >>>>>>>> >>>>>>>> I mount a reference repo as a K8s PVC. >>>>>>>> >>>>>>>> I want to do a git clone <url> --reference-repo <path to my mounted >>>>>>>> reference repo>. >>>>>>>> >>>>>>>> Can the git material support or be modified to accept reference >>>>>>>> repo paths? >>>>>>>> >>>>>>>> This is kind of a deal breaker if it can't >>>>>>>> >>>>>>>> >>>>>>>> On Mon, May 10, 2021, 6:31 PM Marques Lee < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hmm. Well I suppose you could git clone —bare —mirror to a known >>>>>>>>> location and then: >>>>>>>>> >>>>>>>>> 1) create git materials with file system paths for the url instead >>>>>>>>> of http/ssh >>>>>>>>> 2) run a cron to keep the ref repo updated >>>>>>>>> >>>>>>>>> Would that work? >>>>>>>>> On Mon, May 10, 2021 at 6:25 PM HUSSEIN KADIRI <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> My repo is too big, it can't be cloned by itself. it needs a >>>>>>>>>> reference repo. >>>>>>>>>> >>>>>>>>>> Is there a way to configure a reference repo ? >>>>>>>>>> >>>>>>>>>> On Mon, May 10, 2021, 6:23 PM Marques Lee < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Git materials support shallow clone. I think you need to expand >>>>>>>>>>> the advance tab to see the option. It’s of course in >>>>>>>>>>> cruise-config.xml as >>>>>>>>>>> well as the various pipelines-as-config syntaxes. >>>>>>>>>>> >>>>>>>>>>> The workspace isn’t recloned every time either. If it exists on >>>>>>>>>>> disk, it gets updated via fetch. >>>>>>>>>>> >>>>>>>>>>> On Mon, May 10, 2021 at 4:29 PM HUSSEIN KADIRI < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> Sometimes a git repo is large and so it's not efficient/fast to >>>>>>>>>>>> do a git clone. One would have to use a reference repo. For all >>>>>>>>>>>> the parts >>>>>>>>>>>> that require git (config repository, git material, etc), is it >>>>>>>>>>>> possible to >>>>>>>>>>>> have more complex git workflows - utilizing a reference repo, >>>>>>>>>>>> shallow clone >>>>>>>>>>>> - since a simple git clone is not always possible? >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com >>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>> in the Google Groups "go-cd" group. >>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>> to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com >>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>> the Google Groups "go-cd" group. >>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>> to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "go-cd" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "go-cd" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> >>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "go-cd" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com >> <https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "go-cd" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- Hussein Kadiri -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3DLmXbHJHFNqULqQCYkMVWu0GLvoKJmuGgMTPUXQccU6w%40mail.gmail.com.
