Hello! A goal (that is up for anyone interested to work on) is to move the Beam Go SDK to use Go Modules[2] rather than the GoGradle[3] plugin's lock file, since this unifies the experience for both working on the Go SDK (as a Beam dev) and with the Go SDK (as a Beam user). The plugin hasn't been updated to deal with Go Modules, but it's possible to disable the locking and .implicit vendoring in favour of the standard Go Module cache. The gogradle lock file would need to be updated so that Gradle gets the avro package into it's cache (if it's not listed already).
The SDK itself is compatible with Go 1.13 but that's outside of any dependency module concerns. Resolving this would let the SDK adopt new depencies more easily, and better integrate with general Go tooling, rather than just with the demi-integration we have with Gradle. Robert B [1] https://github.com/golang/go/wiki/Modules [2] https://gogradle.github.io/gogradle/docs/getting-started.html On Fri, 29 Nov 2019 at 08:18, Amogh Tiwari <[email protected]> wrote: > Adding +Sameer Abhyankar <[email protected]> for visibility. > > On Fri, Nov 29, 2019 at 4:10 PM Amogh Tiwari <[email protected]> wrote: > >> Hi Elliotte, >> Thanks for your reply. >> Tried out the steps that you suggested, still not able to resolve this >> issue. >> Should I use a particular version of go for the build? >> What I have noticed is that when I ./gradlew build, it automatically >> downloads go 1.12 and sets the go env for it, basically ignoring whatever >> version of go that I already have installed. >> Please follow the last comment of this >> https://issues.apache.org/jira/browse/BEAM-8564. >> Regards >> Amogh >> >> >> On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold <[email protected]> >> wrote: >> >>> I did get through this one, and made the classic mistake of not >>> immediately committing the steps I took to writing. I believe it >>> involved some combination of setting go paths in environment >>> variables. I seem to have added this to the end of my .profile: >>> >>> export GOROOT=/usr/local/go >>> export GOPATH=$HOME/go/packages >>> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH >>> >>> According to my history, I also ran >>> >>> go get github.com/linkedin/goavro >>> >>> I really need to dedicate some time to cleaning up the build >>> documentation. See https://issues.apache.org/jira/browse/BEAM-8798 >>> >>> Right now we have overlapping and sometimes contradictory docs in four >>> different places: >>> >>> README.md >>> CONTRIBUTING.md >>> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ >>> https://beam.apache.org/contribute/ >>> >>> We should probably pick one as the source of truth and rewrite the >>> other three to simply point to it. I propose putting all checkout, >>> build, test, commit, and push instructions in CONTRIBUTING.md in the >>> repo. What do folks think? >>> >>> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari <[email protected]> wrote: >>> > >>> > Hi Elliotte, >>> > I am facing a similar goVet issue. It would be great if you can guide >>> me through the solution. Please let me know the steps that you followed. >>> > Regards, >>> > Amogh >>> > >>> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold < >>> [email protected]> wrote: >>> >> >>> >> Tentatively, the goVet issue does seem to have been an issue with my >>> >> Go install I have now cleaned up. The clickhouse issue remains, as do >>> >> several others I'm working through. >>> >> >>> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to >>> >> consolidate and update the instructions for getting to a working >>> >> build. >>> >> >>> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold >>> >> <[email protected]> wrote: >>> >> > >>> >> > I'm slowly working my way through getting the tests to run and pass. >>> >> > We have a lot of work to do on the contributing docs to explain how >>> to >>> >> > setup and run the build. There's clearly a lot of knowledge in >>> >> > developers' heads and workstations that hasn't yet made it into the >>> >> > docs. >>> >> > >>> >> > The latest is a problem finding "github.com/linkedin/goavro" when I >>> >> > run goVet. I'm not a go person. Is this something that requires an >>> >> > extra install? If so, how is it installed? Or is this some error in >>> >> > the build.gradles? Or perhaps my go config is borked and gradle is >>> >> > looking in the wrong directory? >>> >> > >>> >> > > Task :sdks:go:examples:resolveBuildDependencies >>> >> > Resolving ./ >>> github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go >>> >> > .gogradle/project_gopath/src/ >>> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 >>> : >>> >> > cannot find package "github.com/linkedin/goavro" in any of: >>> >> > >>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ >>> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro >>> >> > (vendor tree) >>> >> > /home/elharo/.gradle/go/binary/1.12/go/src/ >>> github.com/linkedin/goavro >>> >> > (from $GOROOT) >>> >> > >>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ >>> github.com/linkedin/goavro >>> >> > (from $GOPATH) >>> >> > >>> >> > > Task :sdks:go:examples:goVet FAILED >>> >> > >>> >> > I'm also seeing failures in ClickHouseIOTest: >>> >> > >>> >> > > Task :sdks:java:io:clickhouse:test >>> >> > >>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod >>> FAILED >>> >> > java.lang.IllegalStateException >>> >> > >>> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod >>> FAILED >>> >> > java.lang.NullPointerException >>> >> > >>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod >>> FAILED >>> >> > java.lang.IllegalStateException >>> >> > >>> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod >>> FAILED >>> >> > java.lang.NullPointerException >>> >> > >>> >> > -- >>> >> > Elliotte Rusty Harold >>> >> > [email protected] >>> >> >>> >> >>> >> >>> >> -- >>> >> Elliotte Rusty Harold >>> >> [email protected] >>> >>> >>> >>> -- >>> Elliotte Rusty Harold >>> [email protected] >>> >>
