Your material for the pipeline can be the repository where you've your Shell scripts required for the build. It's a good thing to keep those separate and use GoCD to just trigger them.
For triggering pipeline with S3 File Change, I see you've the following choices 1. You can write your own SCM Plugin <https://plugin-api.gocd.org/current/scm/>. (Shameless plug - Take a look at this repo <https://github.com/ashwanthkumar/gocd-build-github-pull-requests> for an example of SCM Plugin). 2. If your GoCD is on AWS, you can setup Event Notifications on S3 to trigger a Lambda function <https://aws.amazon.com/premiumsupport/knowledge-center/lambda-configure-s3-event-notification/> whenever a new file is added (using ObjectCreated Event) and use the Schedule Endpoint <https://api.gocd.org/current/#scheduling-pipelines> of the pipeline to trigger it from the Lambda function. You can choose either of the two approaches depending on how comfortable your organization / team is with either GoCD or AWS Stack. On Fri, Jan 10, 2020 at 11:20 AM J B <[email protected]> wrote: > We are trying to start a build in GoCD when a file in S3 is added or > updated. > > The pipelines require a material, which is tied to a source control > repository. These files are too large for source code and are using S3 as a > way to manage this. > > Is there a way to solve this? Is a material required for the pipeline to > trigger? A timer can be used to start the pipeline, but how can it be > configured to run a shell script to fetch the file from S3 as the first > step, because the rest of the build cannot continue without it? If the > pipeline requires a material, and that material must be in a repository, > will the pipeline run without it and continue to execute stages/jobs/tasks? > > Thank you! > > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/a5527548-8231-4311-9d73-8cff15d603fa%40googlegroups.com > <https://groups.google.com/d/msgid/go-cd/a5527548-8231-4311-9d73-8cff15d603fa%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Ashwanth Kumar / ashwanthkumar.in -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CAD9m7Cx8wr%2BG3NEOUcrOzZGQ5PMcCaVEWM-m%3DsHsJVyYch2nPw%40mail.gmail.com.
