How can I take a folder as an input in my go api and know the relative path
or absolute path of each file in each folder
On Fri, Feb 8, 2019 at 7:12 AM David Riley <fraveyd...@gmail.com> wrote:
> Yes, but we have somewhat different objectives than you might.
>
> We use the Docker golang (alpine) image to build our images, and it works
> wonderfully (and really makes it a lot easier to cope with differences in
> Jenkins build nodes). However, our apps generally eventually run on
> Kubernetes, which means we have a few things to consider for actual use
> that other folks might not.
>
> I should also note that for the most part, we use shell scripts to build,
> as make doesn't really bring us many advantages; it can't really track
> dependencies across the Docker barrier, and since they're not strictly
> speaking files, it can't track the predecessor images (we use a multi-stage
> build process) as dependencies, so shell scripts it is.
>
> By way of background, our project is a collection of microservices which
> we generally build at the same time for a variety of reasons (one of which
> is to maximize parallelism and cache reuse). We use `go build` and/or `go
> install` to achieve that, but we use `go run` when testing things.
>
> Our build process is as follows:
>
> - We make a "build" image for each supported distribution we build our
> apps for (currently Alpine and CentOS, which have to be built separately
> because Alpine's musl-libc makes the binaries incompatible with everything
> else). The build image is generally the base Go image (which is the
> official one for Alpine, and an equivalent cobbled-together one for CentOS)
> along with a basic set of packages installed that we know we need for our
> build (typically git, gcc, protobuf stuff and not much more). That image
> is stored locally and used as a base for the next step.
>
> - We make a "base" image by first copying in the bootstrap scripts
> (formerly to install dep, but now just to do a bit of prep work) and the
> go.{mod,sum} files. We warm up the build cache by running `go mod
> download`, then copy in some secondary scripts to run `go get -u` to pull
> in the required Go binaries for things like code generators needed for the
> build. Then we copy in the rest of the code (we defer this to maximize use
> of the Docker build cache so it doesn't fetch the dependencies every time),
> run the code generators with `go generate`, and then run `go install` on
> all the apps we want to build (rather than `go build` so they wind up in an
> easily-known location), running it on all of them at once to build them at
> the same time. After that, we copy a few of the static assets that get
> bundled with the apps over to other well-known locations in the image
> because none of our environment variables carry over to the next step.
>
> - We then use Docker's newer multi-stage build capability to start from a
> fresh runtime image for each app/distro combo, and copy the apps and their
> required static assets into the fresh image. That way, our Alpine images
> tend to be less than 15MB each (CentOS is a bit more, though in theory the
> base layers get reused, though this is not always true for distribution).
> Because they are microservices, each app container is built with the
> entrypoint simply launching the app itself so the container terminates when
> the app does (which is preferable for Kubernetes).
>
>
> Now, this obviously wouldn't be ideal for rapid deployment, which is where
> things start getting interesting. In theory, we could work directly from
> the "base" container (the middle piece), since that has all the source and
> runtime needed to run the containers, but not everyone on the team relishes
> using terminal-based text editors (there's no accounting for taste). We
> do, in fact, run the unit tests and benchmarks from that container and it
> works splendidly. But for an edit-compile-run cycle, it leaves a little
> bit to be desired, so we add a bit more on for local development.
>
> For the iterative cycle, we do two things. First, we bind-mount our
> source directory over the copied source directory in the container (yeah,
> it's a little wasteful, but the image is already there and set up, so). We
> then run an autorunner which watches the source files and re-runs `go run
> ./<appdir>/<app.go> <args>` (we use `reflex` since it worked best for our
> purposes, but there are others out there including `realize` which seems to
> be generally more popular). This works wonders for developing
> microservices in the container environment with an editor outside the
> container, though getting the regexes to hit all the right files can be
> interesting; you may need to manually poke things after a `go generate`,
> for example. This works extremely well even across the VM barrier of
> Docker for Mac, and I would assume also Docker for Windows, since they have
> taken great pains to ensure that filesystem notifications work across
> platforms.
>
> For specific Kubernetes work, because we want to be able to connect to
> in-cluster resources that aren't available outside the cluster as well as
> connect to the API and get all the environment variables we'd get when
> deployed in-cluster, it's a bit more complex. You don't want to be trying
> to negotiate bind-mounted volumes in a k8s cluster deployed by a Helm
> chart; it's just not worth the trouble (and just isn't possible if your
> cluster is remote). For this, we use Telepresence, which does a pretty
> great job of proxying a local container into the remote or local cluster in
> place of another deployment; as far as your container and the cluster are
> concerned, you might as well be running right inside. We use the
> container-proxying method Telepresence offers; the VPN option won't work
> with AnyConnect (which we need) and proxies the entire machine into the
> cluster anyway, and the libc shim functions don't work on Go because, well,
> libc.
>
> Coupled with the autorunner, this has massively improved our development
> experience. Before we had to move in-cluster (because we could use just
> the external interfaces), we could just `go run` things locally with
> environment variables set, but once we moved in cluster it became a
> laborious cycle of edit code->build container->kubectl edit/delete
> pod->kubectl logs, which took about 3 minutes per cycle worst case. Now it
> just rebuilds when we edit the source files, and the cycle time is
> something like 10s max.
>
> Anyway, I hope that helps. I'm happy to help with more specific queries,
> but that's a containerized setup that's worked quite well for us so far.
>
>
> - Dave
>
>
> > On Feb 2, 2019, at 12:02 PM, Keith Brown <keith6...@gmail.com> wrote:
> >
> > Thanks.
> >
> > At the moment I use a Makefile (different from the above :-) ) .
> >
> > I suppose I am asking because I want to integrate vim-go with the docker
> build ( docker run -v $PWD:/mnt -w /mnt bash -c "go build main.go" )
> > Was wondering if anyone has done that before.
> >
> >
> > On Friday, February 1, 2019 at 7:30:32 PM UTC-5, Space A. wrote:
> > You simply need
> > docker run <...>
> > which will invoke smth like
> > go build
> > at the end.
> >
> > PS: The above Makefile is garbage.
> >
> >
> > пятница, 1 февраля 2019 г., 21:59:53 UTC+3 пользователь Bojan Delić
> написал:
> > I have Makefile that supports build in docker image, part of it looks
> something like this:
> >
> > NAME := <name>
> > PACKAGE := github.com/username/repo
> >
> >
> > .PHONY: build
> > build: clean gobuild ## Build application
> >
> > .PHONY: gobuild
> > gobuild: LDFLAGS=$(shell ./scripts/gen-ldflags.sh $(VERSION))
> > gobuild:
> > @echo "Building '$(NAME)'"
> > @env go generate -mod=vendor
> > @CGO_ENABLED=0 go build -mod=vendor --ldflags "$(LDFLAGS)" -tags
> netgo .
> >
> > .PHONY: clean
> > clean: ## Remove build artifacts
> > @echo "==> Cleaning build artifacts..."
> > @rm -fv coverage.txt
> > @find . -name '*.test' | xargs rm -fv
> > @rm -fv $(NAME)
> >
> > .PHONY: build-in-docker
> > build-in-docker:
> > @docker run --rm \
> > -v $(shell pwd):/go/src/$(PACKAGE) \
> > -w /go/src/$(PACKAGE) \
> > -e GO111MODULE=on \
> > golang:latest \
> > make build
> >
> > As you can see, there are some external scripts called (like
> gen-ldflags.sh) and docker build is just invoking "make build" inside
> docker container. I do not use this for CI (GitLab CI is already setup to
> use docker images), so that is why I use latest tag (in CI I use explicit
> version of Go).
> >
> >
> > There are some leftovers from earlier times, like mounting working dir
> to GOPATH, which is not needed if GO111MODULE is set.
> >
> >
> > On Friday, February 1, 2019 at 4:48:01 AM UTC+1, Keith Brown wrote:
> > does anyone use docker golang image to compile? if so,how is your setup?
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
