Is creating a new PV and provisioning that a feasible option? That's what I do on my Illumos build zones for some use cases - I use a ZFS snapshot, mount it into a Zone and run specific build experiments. It saves me a lot of time since I can avoid cloning gigabytes of code even from the same baremetal server.
On Tue, May 11, 2021 at 10:36 AM Marques Lee <[email protected]> wrote: > Thank you Hussein for that, glad to help. > > You may have seen this article here: > > https://medium.com/@Sushil_Kumar/readwritemany-persistent-volumes-in-google-kubernetes-engine-a0b93e203180 > > I think this person works around limitations on readwritemany by creating > separate PVs fir read and write backed by the same NFS disk. Not sure if > that’s helpful or not or worth the extra infrastructure. > > On Mon, May 10, 2021 at 7:28 PM HUSSEIN KADIRI <[email protected]> wrote: > >> Thank you Marques, I really appreciate your help and support. You've been >> very responsive. >> I'm looking for an alternative to Jenkins. Unfortunately we have really >> large repos so most existing CI tools don't work out of the box because of >> the clone issues. >> I really like GoCD and have enjoyed creating a POC for it. I don't want >> to drop it just yet. I'll continue to look for a workaround. >> >> Thanks again >> >> >> >> On Mon, May 10, 2021 at 7:12 PM Marques Lee <[email protected]> >> wrote: >> >>> Ok, fair enough :). Let us know if we can help in any way. >>> >>> On Mon, May 10, 2021 at 7:06 PM HUSSEIN KADIRI <[email protected]> >>> wrote: >>> >>>> Let me explore the local clone option some more >>>> $ git clone /local/path/to/bare/repo >>>> >>>> Thanks for the tip >>>> On Monday, 10 May 2021 at 19:01:34 UTC-7 HUSSEIN KADIRI wrote: >>>> >>>>> You're right the agent won't need read write. The volume is what needs >>>>> to be setup as read write. See doc >>>>> <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes> >>>>> >>>>> Write - would be the "cron" updating the repo >>>>> Read - would be the agents "cloning" >>>>> Reading and Writing can't happen simultaneously. >>>>> >>>>> On Monday, 10 May 2021 at 18:55:21 UTC-7 Marques Lee wrote: >>>>> >>>>>> Typo — “why would the agents need read/write” was what I meant >>>>>> >>>>>> On Mon, May 10, 2021 at 6:53 PM Marques Lee < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Pardon my lack of familiarity with GKE, my question may be extremely >>>>>>> naive - if there’s only process that needs to write to the mount (the >>>>>>> cron >>>>>>> that performs the fetch to keep the code updated), then by would the >>>>>>> agents >>>>>>> need read/write on the volume? Isn’t read-only enough to clone for a >>>>>>> pipeline run? >>>>>>> >>>>>>> On Mon, May 10, 2021 at 6:48 PM HUSSEIN KADIRI <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hmm, we setup GoCD on kubernetes so the server is a kubernetes >>>>>>>> deployment. >>>>>>>> >>>>>>>> We are on GKE . Yes we can mount a volume but GKE PVC are only >>>>>>>> ReadWriteOnce (One pod mounted to read and write) or ReadOnlyMany >>>>>>>> (Multiple >>>>>>>> pods mounted and reading only from the volume). >>>>>>>> >>>>>>>> Your recommendation requires a ReadWriteMany setup which is not >>>>>>>> possible in GKE. >>>>>>>> >>>>>>>> On Mon, May 10, 2021, 6:43 PM Marques Lee < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hmm, unfortunately I don’t believe the git material in GoCD has >>>>>>>>> native support for reference repos. >>>>>>>>> >>>>>>>>> Could the server not run a cron to do git fetch —all on the bare >>>>>>>>> repo to keep it updated? >>>>>>>>> >>>>>>>>> Then all agents would pick up new changes so long as the volume >>>>>>>>> holding the bare repo is mounted to agents. >>>>>>>>> >>>>>>>>> Otherwise, yeah, GoCD may not be a good fit for you until we build >>>>>>>>> in support for —reference-repo if that’s a deal breaker. >>>>>>>>> >>>>>>>>> On Mon, May 10, 2021 at 6:37 PM HUSSEIN KADIRI <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I have elastic agents so the cron route would not be feasible. >>>>>>>>>> >>>>>>>>>> I mount a reference repo as a K8s PVC. >>>>>>>>>> >>>>>>>>>> I want to do a git clone <url> --reference-repo <path to my >>>>>>>>>> mounted reference repo>. >>>>>>>>>> >>>>>>>>>> Can the git material support or be modified to accept reference >>>>>>>>>> repo paths? >>>>>>>>>> >>>>>>>>>> This is kind of a deal breaker if it can't >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, May 10, 2021, 6:31 PM Marques Lee < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hmm. Well I suppose you could git clone —bare —mirror to a known >>>>>>>>>>> location and then: >>>>>>>>>>> >>>>>>>>>>> 1) create git materials with file system paths for the url >>>>>>>>>>> instead of http/ssh >>>>>>>>>>> 2) run a cron to keep the ref repo updated >>>>>>>>>>> >>>>>>>>>>> Would that work? >>>>>>>>>>> On Mon, May 10, 2021 at 6:25 PM HUSSEIN KADIRI < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> My repo is too big, it can't be cloned by itself. it needs a >>>>>>>>>>>> reference repo. >>>>>>>>>>>> >>>>>>>>>>>> Is there a way to configure a reference repo ? >>>>>>>>>>>> >>>>>>>>>>>> On Mon, May 10, 2021, 6:23 PM Marques Lee < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Git materials support shallow clone. I think you need to >>>>>>>>>>>>> expand the advance tab to see the option. It’s of course in >>>>>>>>>>>>> cruise-config.xml as well as the various pipelines-as-config >>>>>>>>>>>>> syntaxes. >>>>>>>>>>>>> >>>>>>>>>>>>> The workspace isn’t recloned every time either. If it exists >>>>>>>>>>>>> on disk, it gets updated via fetch. >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, May 10, 2021 at 4:29 PM HUSSEIN KADIRI < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> Sometimes a git repo is large and so it's not efficient/fast >>>>>>>>>>>>>> to do a git clone. One would have to use a reference repo. For >>>>>>>>>>>>>> all the >>>>>>>>>>>>>> parts that require git (config repository, git material, etc), >>>>>>>>>>>>>> is it >>>>>>>>>>>>>> possible to have more complex git workflows - utilizing a >>>>>>>>>>>>>> reference repo, >>>>>>>>>>>>>> shallow clone - since a simple git clone is not always possible? >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com >>>>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dm2Ah3rAMzkt8j9pM_LoAKOn%3DgK0KMqNW-7dm8FuQOpyA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>> . >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>>>> in the Google Groups "go-cd" group. >>>>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe >>>>>>>>>>>>> . >>>>>>>>>>>>> To unsubscribe from this group and all its topics, send an >>>>>>>>>>>>> email to [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jatvS8kBrhek9meEdOJpus6Hb%2BpLBnWjcxNoQK97FXn7A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com >>>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DnZo77QzutmB1SwUHw%2B2sGC5JNjXVkWWyXGFzTn7OQdzg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>> in the Google Groups "go-cd" group. >>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>> to [email protected]. >>>>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com >>>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZas%2BPcdOZ855o1ZgpRoi0J0X4%3DnWMXC%3DBg9GjzAwwb-A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "go-cd" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7Dn%2BDPphBOumaogyhuPSObpi4t1Ji2TgBFjTzEQ7M2qq6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>> the Google Groups "go-cd" group. >>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>> to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAPKX9jZNHv83JEWZbm9R5jNMR9_h1Gk365ou6w%3DfGe4e5uDXQg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "go-cd" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7DkReyQMze3UpoxKYOrvXn%2B1WC6ax2z8U%2Bs5YBrxjSU0%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com >>>> <https://groups.google.com/d/msgid/go-cd/30b62739-5858-491b-a4ef-6b353592265cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "go-cd" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/go-cd/wEK4-BYReb8/unsubscribe. >>> >> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/go-cd/CAPKX9ja1hJirt0E6XWKcxcqiJtwO%3DUyy1g%2BsiSYpKgfbjsgbBA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Hussein Kadiri >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3DLmXbHJHFNqULqQCYkMVWu0GLvoKJmuGgMTPUXQccU6w%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CAFD%2B7D%3DLmXbHJHFNqULqQCYkMVWu0GLvoKJmuGgMTPUXQccU6w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAPKX9jYF7ffnvezTWff0xg9VSj519ea0Pk4vx2AeyydnDVgkyA%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAPKX9jYF7ffnvezTWff0xg9VSj519ea0Pk4vx2AeyydnDVgkyA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CANiY96ZkMGtXjh63pfOiJ3ie6qb1XgnYHrG29-p-th2F6zSgjQ%40mail.gmail.com.
