And FYI the folder is in local system and API is on server and hence the
path thing will not work, we need to send the object to API
On Fri, Feb 8, 2019 at 11:09 AM akshita babel <akshitababel.1...@gmail.com>
wrote:
> How can I take a folder as an input in my go api and know the relative
> path or absolute path of each file in each folder
>
> On Fri, Feb 8, 2019 at 7:12 AM David Riley <fraveyd...@gmail.com> wrote:
>
>> Yes, but we have somewhat different objectives than you might.
>>
>> We use the Docker golang (alpine) image to build our images, and it works
>> wonderfully (and really makes it a lot easier to cope with differences in
>> Jenkins build nodes). However, our apps generally eventually run on
>> Kubernetes, which means we have a few things to consider for actual use
>> that other folks might not.
>>
>> I should also note that for the most part, we use shell scripts to build,
>> as make doesn't really bring us many advantages; it can't really track
>> dependencies across the Docker barrier, and since they're not strictly
>> speaking files, it can't track the predecessor images (we use a multi-stage
>> build process) as dependencies, so shell scripts it is.
>>
>> By way of background, our project is a collection of microservices which
>> we generally build at the same time for a variety of reasons (one of which
>> is to maximize parallelism and cache reuse). We use `go build` and/or `go
>> install` to achieve that, but we use `go run` when testing things.
>>
>> Our build process is as follows:
>>
>> - We make a "build" image for each supported distribution we build our
>> apps for (currently Alpine and CentOS, which have to be built separately
>> because Alpine's musl-libc makes the binaries incompatible with everything
>> else). The build image is generally the base Go image (which is the
>> official one for Alpine, and an equivalent cobbled-together one for CentOS)
>> along with a basic set of packages installed that we know we need for our
>> build (typically git, gcc, protobuf stuff and not much more). That image
>> is stored locally and used as a base for the next step.
>>
>> - We make a "base" image by first copying in the bootstrap scripts
>> (formerly to install dep, but now just to do a bit of prep work) and the
>> go.{mod,sum} files. We warm up the build cache by running `go mod
>> download`, then copy in some secondary scripts to run `go get -u` to pull
>> in the required Go binaries for things like code generators needed for the
>> build. Then we copy in the rest of the code (we defer this to maximize use
>> of the Docker build cache so it doesn't fetch the dependencies every time),
>> run the code generators with `go generate`, and then run `go install` on
>> all the apps we want to build (rather than `go build` so they wind up in an
>> easily-known location), running it on all of them at once to build them at
>> the same time. After that, we copy a few of the static assets that get
>> bundled with the apps over to other well-known locations in the image
>> because none of our environment variables carry over to the next step.
>>
>> - We then use Docker's newer multi-stage build capability to start from a
>> fresh runtime image for each app/distro combo, and copy the apps and their
>> required static assets into the fresh image. That way, our Alpine images
>> tend to be less than 15MB each (CentOS is a bit more, though in theory the
>> base layers get reused, though this is not always true for distribution).
>> Because they are microservices, each app container is built with the
>> entrypoint simply launching the app itself so the container terminates when
>> the app does (which is preferable for Kubernetes).
>>
>>
>> Now, this obviously wouldn't be ideal for rapid deployment, which is
>> where things start getting interesting. In theory, we could work directly
>> from the "base" container (the middle piece), since that has all the source
>> and runtime needed to run the containers, but not everyone on the team
>> relishes using terminal-based text editors (there's no accounting for
>> taste). We do, in fact, run the unit tests and benchmarks from that
>> container and it works splendidly. But for an edit-compile-run cycle, it
>> leaves a little bit to be desired, so we add a bit more on for local
>> development.
>>
>> For the iterative cycle, we do two things. First, we bind-mount our
>> source directory over the copied source directory in the container (yeah,
>> it's a little wasteful, but the image is already there and set up, so). We
>> then run an autorunner which watches the source files and re-runs `go run
>> ./<appdir>/<app.go> <args>` (we use `reflex` since it worked best for our
>> purposes, but there are others out there including `realize` which seems to
>> be generally more popular). This works wonders for developing
>> microservices in the container environment with an editor outside the
>> container, though getting the regexes to hit all the right files can be
>> interesting; you may need to manually poke things after a `go generate`,
>> for example. This works extremely well even across the VM barrier of
>> Docker for Mac, and I would assume also Docker for Windows, since they have
>> taken great pains to ensure that filesystem notifications work across
>> platforms.
>>
>> For specific Kubernetes work, because we want to be able to connect to
>> in-cluster resources that aren't available outside the cluster as well as
>> connect to the API and get all the environment variables we'd get when
>> deployed in-cluster, it's a bit more complex. You don't want to be trying
>> to negotiate bind-mounted volumes in a k8s cluster deployed by a Helm
>> chart; it's just not worth the trouble (and just isn't possible if your
>> cluster is remote). For this, we use Telepresence, which does a pretty
>> great job of proxying a local container into the remote or local cluster in
>> place of another deployment; as far as your container and the cluster are
>> concerned, you might as well be running right inside. We use the
>> container-proxying method Telepresence offers; the VPN option won't work
>> with AnyConnect (which we need) and proxies the entire machine into the
>> cluster anyway, and the libc shim functions don't work on Go because, well,
>> libc.
>>
>> Coupled with the autorunner, this has massively improved our development
>> experience. Before we had to move in-cluster (because we could use just
>> the external interfaces), we could just `go run` things locally with
>> environment variables set, but once we moved in cluster it became a
>> laborious cycle of edit code->build container->kubectl edit/delete
>> pod->kubectl logs, which took about 3 minutes per cycle worst case. Now it
>> just rebuilds when we edit the source files, and the cycle time is
>> something like 10s max.
>>
>> Anyway, I hope that helps. I'm happy to help with more specific queries,
>> but that's a containerized setup that's worked quite well for us so far.
>>
>>
>> - Dave
>>
>>
>> > On Feb 2, 2019, at 12:02 PM, Keith Brown <keith6...@gmail.com> wrote:
>> >
>> > Thanks.
>> >
>> > At the moment I use a Makefile (different from the above :-) ) .
>> >
>> > I suppose I am asking because I want to integrate vim-go with the
>> docker build ( docker run -v $PWD:/mnt -w /mnt bash -c "go build main.go" )
>> > Was wondering if anyone has done that before.
>> >
>> >
>> > On Friday, February 1, 2019 at 7:30:32 PM UTC-5, Space A. wrote:
>> > You simply need
>> > docker run <...>
>> > which will invoke smth like
>> > go build
>> > at the end.
>> >
>> > PS: The above Makefile is garbage.
>> >
>> >
>> > пятница, 1 февраля 2019 г., 21:59:53 UTC+3 пользователь Bojan Delić
>> написал:
>> > I have Makefile that supports build in docker image, part of it looks
>> something like this:
>> >
>> > NAME := <name>
>> > PACKAGE := github.com/username/repo
>> >
>> >
>> > .PHONY: build
>> > build: clean gobuild ## Build application
>> >
>> > .PHONY: gobuild
>> > gobuild: LDFLAGS=$(shell ./scripts/gen-ldflags.sh $(VERSION))
>> > gobuild:
>> > @echo "Building '$(NAME)'"
>> > @env go generate -mod=vendor
>> > @CGO_ENABLED=0 go build -mod=vendor --ldflags "$(LDFLAGS)" -tags
>> netgo .
>> >
>> > .PHONY: clean
>> > clean: ## Remove build artifacts
>> > @echo "==> Cleaning build artifacts..."
>> > @rm -fv coverage.txt
>> > @find . -name '*.test' | xargs rm -fv
>> > @rm -fv $(NAME)
>> >
>> > .PHONY: build-in-docker
>> > build-in-docker:
>> > @docker run --rm \
>> > -v $(shell pwd):/go/src/$(PACKAGE) \
>> > -w /go/src/$(PACKAGE) \
>> > -e GO111MODULE=on \
>> > golang:latest \
>> > make build
>> >
>> > As you can see, there are some external scripts called (like
>> gen-ldflags.sh) and docker build is just invoking "make build" inside
>> docker container. I do not use this for CI (GitLab CI is already setup to
>> use docker images), so that is why I use latest tag (in CI I use explicit
>> version of Go).
>> >
>> >
>> > There are some leftovers from earlier times, like mounting working dir
>> to GOPATH, which is not needed if GO111MODULE is set.
>> >
>> >
>> > On Friday, February 1, 2019 at 4:48:01 AM UTC+1, Keith Brown wrote:
>> > does anyone use docker golang image to compile? if so,how is your setup?
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to golang-nuts+unsubscr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
