Thank you Hussein for that, glad to help. You may have seen this article here: https://medium.com/@Sushil_Kumar/readwritemany-persistent-volumes-in-google-kubernetes-engine-a0b93e203180
I think this person works around limitations on readwritemany by creating separate PVs fir read and write backed by the same NFS disk. Not sure if that’s helpful or not or worth the extra infrastructure. On Mon, May 10, 2021 at 7:28 PM HUSSEIN KADIRI <[email protected]> wrote: > Thank you Marques, I really appreciate your help and support. You've been > very responsive. > I'm looking for an alternative to Jenkins. Unfortunately we have really > large repos so most existing CI tools don't work out of the box because of > the clone issues. > I really like GoCD and have enjoyed creating a POC for it. I don't want to > drop it just yet. I'll continue to look for a workaround. > > Thanks again > > > > On Mon, May 10, 2021 at 7:12 PM Marques Lee <[email protected]> > wrote: > >> Ok, fair enough :). Let us know if we can help in any way. >> >> On Mon, May 10, 2021 at 7:06 PM HUSSEIN KADIRI <[email protected]> >> wrote: >> >>> Let me explore the local clone option some more >>> $ git clone /local/path/to/bare/repo >>> >>> Thanks for the tip >>> On Monday, 10 May 2021 at 19:01:34 UTC-7 HUSSEIN KADIRI wrote: >>> >>>> You're right the agent won't need read write. The volume is what needs >>>> to be setup as read write. See doc >>>> <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes> >>>> >>>> Write - would be the "cron" updating the repo >>>> Read - would be the agents "cloning" >>>> Reading and Writing can't happen simultaneously. >>>> >>>> On Monday, 10 May 2021 at 18:55:21 UTC-7 Marques Lee wrote: >>>> >>>>> Typo — “why would the agents need read/write” was what I meant >>>>> >>>>> On Mon, May 10, 2021 at 6:53 PM Marques Lee <[email protected]> >>>>> wrote: >>>>> >>>>>> Pardon my lack of familiarity with GKE, my question may be extremely >>>>>> naive - if there’s only process that needs to write to the mount (the >>>>>> cron >>>>>> that performs the fetch to keep the code updated), then by would the >>>>>> agents >>>>>> need read/write on the volume? Isn’t read-only enough to clone for a >>>>>> pipeline run? >>>>>> >>>>>> On Mon, May 10, 2021 at 6:48 PM HUSSEIN KADIRI <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hmm, we setup GoCD on kubernetes so the server is a kubernetes >>>>>>> deployment. >>>>>>> >>>>>>> We are on GKE . Yes we can mount a volume but GKE PVC are only >>>>>>> ReadWriteOnce (One pod mounted to read and write) or ReadOnlyMany >>>>>>> (Multiple >>>>>>> pods mounted and reading only from the volume). >>>>>>> >>>>>>> Your recommendation requires a ReadWriteMany setup which is not >>>>>>> possible in GKE. >>>>>>> >>>>>>> On Mon, May 10, 2021, 6:43 PM Marques Lee <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hmm, unfortunately I don’t believe the git material in GoCD has >>>>>>>> native support for reference repos. >>>>>>>> >>>>>>>> Could the server not run a cron to do git fetch —all on the bare >>>>>>>> repo to keep it updated? >>>>>>>> >>>>>>>> Then all agents would pick up new changes so long as the volume >>>>>>>> holding the bare repo is mounted to agents. >>>>>>>> >>>>>>>> Otherwise, yeah, GoCD may not be a good fit for you until we build >>>>>>>> in support for —reference-repo if that’s a deal breaker. >>>>>>>> >>>>>>>> On Mon, May 10, 2021 at 6:37 PM HUSSEIN KADIRI <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I have elastic agents so the cron route would not be feasible. >>>>>>>>> >>>>>>>>> I mount a reference repo as a K8s PVC. >>>>>>>>> >>>>>>>>> I want to do a git clone <url> --reference-repo <path to my >>>>>>>>> mounted reference repo>. >>>>>>>>> >>>>>>>>> Can the git material support or be modified to accept reference >>>>>>>>> repo paths? >>>>>>>>> >>>>>>>>> This is kind of a deal breaker if it can't >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, May 10, 2021, 6:31 PM Marques Lee < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hmm. Well I suppose you could git clone —bare —mirror to a known >>>>>>>>>> location and then: >>>>>>>>>> >>>>>>>>>> 1) create git materials with file system paths for the url >>>>>>>>>> instead of http/ssh >>>>>>>>>> 2) run a cron to keep the ref repo updated >>>>>>>>>> >>>>>>>>>> Would that work? >>>>>>>>>> On Mon, May 10, 2021 at 6:25 PM HUSSEIN KADIRI <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> My repo is too big, it can't be cloned by itself. it needs a >>>>>>>>>>> reference repo. >>>>>>>>>>> >>>>>>>>>>> Is there a way to configure a reference repo ? >>>>>>>>>>> >>>>>>>>>>> On Mon, May 10, 2021, 6:23 PM Marques Lee < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Git materials support shallow clone. I think you need to expand >>>>>>>>>>>> the advance tab to see the option. It’s of course in >>>>>>>>>>>> cruise-config.xml as >>>>>>>>>>>> well as the various pipelines-as-config syntaxes. >>>>>>>>>>>> >>>>>>>>>>>> The workspace isn’t recloned every time either. If it exists on >>>>>>>>>>>> disk, it gets updated via fetch. >>>>>>>>>>>> >>>>>>>>>>>> On Mon, May 10, 2021 at 4:29 PM HUSSEIN KADIRI < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> Sometimes a git repo is large and so it's not efficient/fast >>>>>>>>>>>>> to do a git clone. One would have to use a reference repo. For >>>>>>>>>>>>> all the >>>>>>>>>>>>> parts that require git (config repository, git material, etc), is >>>>>>>>>>>>> it >>>>>>>>>>>>> possible to have more complex git workflows - utilizing a >>>>>>>>>>>>> reference repo, >>>>>>>>>>>>> shallow clone - since a simple git clone is not always possible? >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>>> in the Google Groups "go-cd" group. >>>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe >>>>>>>>>>>> . >>>>>>>>>>>> To unsubscribe from this group and all its topics, send an >>>>>>>>>>>> email to [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com >>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com >>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>> in the Google Groups "go-cd" group. >>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>> to [email protected]. >>>>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "go-cd" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "go-cd" group. >>>>>>>> To unsubscribe from this topic, visit >>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> [email protected]. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "go-cd" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "go-cd" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com >>> <https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "go-cd" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >> > To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Hussein Kadiri > > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3DLmXbHJHFNqULqQCYkMVWu0GLvoKJmuGgMTPUXQccU6w%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3DLmXbHJHFNqULqQCYkMVWu0GLvoKJmuGgMTPUXQccU6w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAPKX9jYF7ffnvezTWff0xg9VSj519ea0Pk4vx2AeyydnDVgkyA%40mail.gmail.com.
