Re: goVet and clickHouse tests failing
Hello! A goal (that is up for anyone interested to work on) is to move the Beam Go SDK to use Go Modules[2] rather than the GoGradle[3] plugin's lock file, since this unifies the experience for both working on the Go SDK (as a Beam dev) and with the Go SDK (as a Beam user). The plugin hasn't been updated to deal with Go Modules, but it's possible to disable the locking and .implicit vendoring in favour of the standard Go Module cache. The gogradle lock file would need to be updated so that Gradle gets the avro package into it's cache (if it's not listed already). The SDK itself is compatible with Go 1.13 but that's outside of any dependency module concerns. Resolving this would let the SDK adopt new depencies more easily, and better integrate with general Go tooling, rather than just with the demi-integration we have with Gradle. Robert B [1] https://github.com/golang/go/wiki/Modules [2] https://gogradle.github.io/gogradle/docs/getting-started.html On Fri, 29 Nov 2019 at 08:18, Amogh Tiwari wrote: > Adding +Sameer Abhyankar for visibility. > > On Fri, Nov 29, 2019 at 4:10 PM Amogh Tiwari wrote: > >> Hi Elliotte, >> Thanks for your reply. >> Tried out the steps that you suggested, still not able to resolve this >> issue. >> Should I use a particular version of go for the build? >> What I have noticed is that when I ./gradlew build, it automatically >> downloads go 1.12 and sets the go env for it, basically ignoring whatever >> version of go that I already have installed. >> Please follow the last comment of this >> https://issues.apache.org/jira/browse/BEAM-8564. >> Regards >> Amogh >> >> >> On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold >> wrote: >> >>> I did get through this one, and made the classic mistake of not >>> immediately committing the steps I took to writing. I believe it >>> involved some combination of setting go paths in environment >>> variables. I seem to have added this to the end of my .profile: >>> >>> export GOROOT=/usr/local/go >>> export GOPATH=$HOME/go/packages >>> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH >>> >>> According to my history, I also ran >>> >>> go get github.com/linkedin/goavro >>> >>> I really need to dedicate some time to cleaning up the build >>> documentation. See https://issues.apache.org/jira/browse/BEAM-8798 >>> >>> Right now we have overlapping and sometimes contradictory docs in four >>> different places: >>> >>> README.md >>> CONTRIBUTING.md >>> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ >>> https://beam.apache.org/contribute/ >>> >>> We should probably pick one as the source of truth and rewrite the >>> other three to simply point to it. I propose putting all checkout, >>> build, test, commit, and push instructions in CONTRIBUTING.md in the >>> repo. What do folks think? >>> >>> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari wrote: >>> > >>> > Hi Elliotte, >>> > I am facing a similar goVet issue. It would be great if you can guide >>> me through the solution. Please let me know the steps that you followed. >>> > Regards, >>> > Amogh >>> > >>> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold < >>> elh...@ibiblio.org> wrote: >>> >> >>> >> Tentatively, the goVet issue does seem to have been an issue with my >>> >> Go install I have now cleaned up. The clickhouse issue remains, as do >>> >> several others I'm working through. >>> >> >>> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to >>> >> consolidate and update the instructions for getting to a working >>> >> build. >>> >> >>> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold >>> >> wrote: >>> >> > >>> >> > I'm slowly working my way through getting the tests to run and pass. >>> >> > We have a lot of work to do on the contributing docs to explain how >>> to >>> >> > setup and run the build. There's clearly a lot of knowledge in >>> >> > developers' heads and workstations that hasn't yet made it into the >>> >> > docs. >>> >> > >>> >> > The latest is a problem finding "github.com/linkedin/goavro" when I >>> >> > run goVet. I'm not a go person. Is this something that requires an >>> >> > extra install? If so, how is it installed? Or is this some error in >>> >> > the build.gradles? Or perhaps my go config is borked and gradle is >>> >> > looking in the wrong directory? >>> >> > >>> >> > > Task :sdks:go:examples:resolveBuildDependencies >>> >> > Resolving ./ >>> github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go >>> >> > .gogradle/project_gopath/src/ >>> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 >>> : >>> >> > cannot find package "github.com/linkedin/goavro" in any of: >>> >> > >>> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ >>> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro >>> >> > (vendor tree) >>> >> > /home/elharo/.gradle/go/binary/1.12/go/src/ >>> github.com/linkedin/goavro >>> >> > (from $GOROOT) >>> >> > >>
Re: goVet and clickHouse tests failing
Adding +Sameer Abhyankar for visibility. On Fri, Nov 29, 2019 at 4:10 PM Amogh Tiwari wrote: > Hi Elliotte, > Thanks for your reply. > Tried out the steps that you suggested, still not able to resolve this > issue. > Should I use a particular version of go for the build? > What I have noticed is that when I ./gradlew build, it automatically > downloads go 1.12 and sets the go env for it, basically ignoring whatever > version of go that I already have installed. > Please follow the last comment of this > https://issues.apache.org/jira/browse/BEAM-8564. > Regards > Amogh > > > On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold > wrote: > >> I did get through this one, and made the classic mistake of not >> immediately committing the steps I took to writing. I believe it >> involved some combination of setting go paths in environment >> variables. I seem to have added this to the end of my .profile: >> >> export GOROOT=/usr/local/go >> export GOPATH=$HOME/go/packages >> export PATH=$GOPATH/bin:$GOROOT/bin:$PATH >> >> According to my history, I also ran >> >> go get github.com/linkedin/goavro >> >> I really need to dedicate some time to cleaning up the build >> documentation. See https://issues.apache.org/jira/browse/BEAM-8798 >> >> Right now we have overlapping and sometimes contradictory docs in four >> different places: >> >> README.md >> CONTRIBUTING.md >> https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ >> https://beam.apache.org/contribute/ >> >> We should probably pick one as the source of truth and rewrite the >> other three to simply point to it. I propose putting all checkout, >> build, test, commit, and push instructions in CONTRIBUTING.md in the >> repo. What do folks think? >> >> On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari wrote: >> > >> > Hi Elliotte, >> > I am facing a similar goVet issue. It would be great if you can guide >> me through the solution. Please let me know the steps that you followed. >> > Regards, >> > Amogh >> > >> > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold < >> elh...@ibiblio.org> wrote: >> >> >> >> Tentatively, the goVet issue does seem to have been an issue with my >> >> Go install I have now cleaned up. The clickhouse issue remains, as do >> >> several others I'm working through. >> >> >> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to >> >> consolidate and update the instructions for getting to a working >> >> build. >> >> >> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold >> >> wrote: >> >> > >> >> > I'm slowly working my way through getting the tests to run and pass. >> >> > We have a lot of work to do on the contributing docs to explain how >> to >> >> > setup and run the build. There's clearly a lot of knowledge in >> >> > developers' heads and workstations that hasn't yet made it into the >> >> > docs. >> >> > >> >> > The latest is a problem finding "github.com/linkedin/goavro" when I >> >> > run goVet. I'm not a go person. Is this something that requires an >> >> > extra install? If so, how is it installed? Or is this some error in >> >> > the build.gradles? Or perhaps my go config is borked and gradle is >> >> > looking in the wrong directory? >> >> > >> >> > > Task :sdks:go:examples:resolveBuildDependencies >> >> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go >> >> > .gogradle/project_gopath/src/ >> github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 >> : >> >> > cannot find package "github.com/linkedin/goavro" in any of: >> >> > >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ >> github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro >> >> > (vendor tree) >> >> > /home/elharo/.gradle/go/binary/1.12/go/src/ >> github.com/linkedin/goavro >> >> > (from $GOROOT) >> >> > >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ >> github.com/linkedin/goavro >> >> > (from $GOPATH) >> >> > >> >> > > Task :sdks:go:examples:goVet FAILED >> >> > >> >> > I'm also seeing failures in ClickHouseIOTest: >> >> > >> >> > > Task :sdks:java:io:clickhouse:test >> >> > >> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod >> FAILED >> >> > java.lang.IllegalStateException >> >> > >> >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod >> FAILED >> >> > java.lang.NullPointerException >> >> > >> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod >> FAILED >> >> > java.lang.IllegalStateException >> >> > >> >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod >> FAILED >> >> > java.lang.NullPointerException >> >> > >> >> > -- >> >> > Elliotte Rusty Harold >> >> > elh...@ibiblio.org >> >> >> >> >> >> >> >> -- >> >> Elliotte Rusty Harold >> >> elh...@ibiblio.org >> >> >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org >> >
Re: goVet and clickHouse tests failing
Hi Elliotte, Thanks for your reply. Tried out the steps that you suggested, still not able to resolve this issue. Should I use a particular version of go for the build? What I have noticed is that when I ./gradlew build, it automatically downloads go 1.12 and sets the go env for it, basically ignoring whatever version of go that I already have installed. Please follow the last comment of this https://issues.apache.org/jira/browse/BEAM-8564. Regards Amogh On Wed, Nov 27, 2019 at 5:46 PM Elliotte Rusty Harold wrote: > I did get through this one, and made the classic mistake of not > immediately committing the steps I took to writing. I believe it > involved some combination of setting go paths in environment > variables. I seem to have added this to the end of my .profile: > > export GOROOT=/usr/local/go > export GOPATH=$HOME/go/packages > export PATH=$GOPATH/bin:$GOROOT/bin:$PATH > > According to my history, I also ran > > go get github.com/linkedin/goavro > > I really need to dedicate some time to cleaning up the build > documentation. See https://issues.apache.org/jira/browse/BEAM-8798 > > Right now we have overlapping and sometimes contradictory docs in four > different places: > > README.md > CONTRIBUTING.md > https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ > https://beam.apache.org/contribute/ > > We should probably pick one as the source of truth and rewrite the > other three to simply point to it. I propose putting all checkout, > build, test, commit, and push instructions in CONTRIBUTING.md in the > repo. What do folks think? > > On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari wrote: > > > > Hi Elliotte, > > I am facing a similar goVet issue. It would be great if you can guide me > through the solution. Please let me know the steps that you followed. > > Regards, > > Amogh > > > > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold < > elh...@ibiblio.org> wrote: > >> > >> Tentatively, the goVet issue does seem to have been an issue with my > >> Go install I have now cleaned up. The clickhouse issue remains, as do > >> several others I'm working through. > >> > >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to > >> consolidate and update the instructions for getting to a working > >> build. > >> > >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold > >> wrote: > >> > > >> > I'm slowly working my way through getting the tests to run and pass. > >> > We have a lot of work to do on the contributing docs to explain how to > >> > setup and run the build. There's clearly a lot of knowledge in > >> > developers' heads and workstations that hasn't yet made it into the > >> > docs. > >> > > >> > The latest is a problem finding "github.com/linkedin/goavro" when I > >> > run goVet. I'm not a go person. Is this something that requires an > >> > extra install? If so, how is it installed? Or is this some error in > >> > the build.gradles? Or perhaps my go config is borked and gradle is > >> > looking in the wrong directory? > >> > > >> > > Task :sdks:go:examples:resolveBuildDependencies > >> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go > >> > .gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 > : > >> > cannot find package "github.com/linkedin/goavro" in any of: > >> > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro > >> > (vendor tree) > >> > /home/elharo/.gradle/go/binary/1.12/go/src/ > github.com/linkedin/goavro > >> > (from $GOROOT) > >> > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/linkedin/goavro > >> > (from $GOPATH) > >> > > >> > > Task :sdks:go:examples:goVet FAILED > >> > > >> > I'm also seeing failures in ClickHouseIOTest: > >> > > >> > > Task :sdks:java:io:clickhouse:test > >> > > >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod > FAILED > >> > java.lang.IllegalStateException > >> > > >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod > FAILED > >> > java.lang.NullPointerException > >> > > >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod > FAILED > >> > java.lang.IllegalStateException > >> > > >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod > FAILED > >> > java.lang.NullPointerException > >> > > >> > -- > >> > Elliotte Rusty Harold > >> > elh...@ibiblio.org > >> > >> > >> > >> -- > >> Elliotte Rusty Harold > >> elh...@ibiblio.org > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
Re: goVet and clickHouse tests failing
I did get through this one, and made the classic mistake of not immediately committing the steps I took to writing. I believe it involved some combination of setting go paths in environment variables. I seem to have added this to the end of my .profile: export GOROOT=/usr/local/go export GOPATH=$HOME/go/packages export PATH=$GOPATH/bin:$GOROOT/bin:$PATH According to my history, I also ran go get github.com/linkedin/goavro I really need to dedicate some time to cleaning up the build documentation. See https://issues.apache.org/jira/browse/BEAM-8798 Right now we have overlapping and sometimes contradictory docs in four different places: README.md CONTRIBUTING.md https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ https://beam.apache.org/contribute/ We should probably pick one as the source of truth and rewrite the other three to simply point to it. I propose putting all checkout, build, test, commit, and push instructions in CONTRIBUTING.md in the repo. What do folks think? On Wed, Nov 27, 2019 at 3:51 AM Amogh Tiwari wrote: > > Hi Elliotte, > I am facing a similar goVet issue. It would be great if you can guide me > through the solution. Please let me know the steps that you followed. > Regards, > Amogh > > On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold > wrote: >> >> Tentatively, the goVet issue does seem to have been an issue with my >> Go install I have now cleaned up. The clickhouse issue remains, as do >> several others I'm working through. >> >> I've filed https://issues.apache.org/jira/browse/BEAM-8798 to >> consolidate and update the instructions for getting to a working >> build. >> >> On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold >> wrote: >> > >> > I'm slowly working my way through getting the tests to run and pass. >> > We have a lot of work to do on the contributing docs to explain how to >> > setup and run the build. There's clearly a lot of knowledge in >> > developers' heads and workstations that hasn't yet made it into the >> > docs. >> > >> > The latest is a problem finding "github.com/linkedin/goavro" when I >> > run goVet. I'm not a go person. Is this something that requires an >> > extra install? If so, how is it installed? Or is this some error in >> > the build.gradles? Or perhaps my go config is borked and gradle is >> > looking in the wrong directory? >> > >> > > Task :sdks:go:examples:resolveBuildDependencies >> > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go >> > .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2: >> > cannot find package "github.com/linkedin/goavro" in any of: >> > >> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro >> > (vendor tree) >> > >> > /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro >> > (from $GOROOT) >> > >> > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro >> > (from $GOPATH) >> > >> > > Task :sdks:go:examples:goVet FAILED >> > >> > I'm also seeing failures in ClickHouseIOTest: >> > >> > > Task :sdks:java:io:clickhouse:test >> > >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> > java.lang.IllegalStateException >> > >> > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> > java.lang.NullPointerException >> > >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> > java.lang.IllegalStateException >> > >> > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> > java.lang.NullPointerException >> > >> > -- >> > Elliotte Rusty Harold >> > elh...@ibiblio.org >> >> >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org -- Elliotte Rusty Harold elh...@ibiblio.org
Re: goVet and clickHouse tests failing
Hi Elliotte, I am facing a similar goVet issue. It would be great if you can guide me through the solution. Please let me know the steps that you followed. Regards, Amogh On Thu, Nov 21, 2019 at 6:06 PM Elliotte Rusty Harold wrote: > Tentatively, the goVet issue does seem to have been an issue with my > Go install I have now cleaned up. The clickhouse issue remains, as do > several others I'm working through. > > I've filed https://issues.apache.org/jira/browse/BEAM-8798 to > consolidate and update the instructions for getting to a working > build. > > On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold > wrote: > > > > I'm slowly working my way through getting the tests to run and pass. > > We have a lot of work to do on the contributing docs to explain how to > > setup and run the build. There's clearly a lot of knowledge in > > developers' heads and workstations that hasn't yet made it into the > > docs. > > > > The latest is a problem finding "github.com/linkedin/goavro" when I > > run goVet. I'm not a go person. Is this something that requires an > > extra install? If so, how is it installed? Or is this some error in > > the build.gradles? Or perhaps my go config is borked and gradle is > > looking in the wrong directory? > > > > > Task :sdks:go:examples:resolveBuildDependencies > > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go > > .gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 > : > > cannot find package "github.com/linkedin/goavro" in any of: > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro > > (vendor tree) > > /home/elharo/.gradle/go/binary/1.12/go/src/ > github.com/linkedin/goavro > > (from $GOROOT) > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/linkedin/goavro > > (from $GOPATH) > > > > > Task :sdks:go:examples:goVet FAILED > > > > I'm also seeing failures in ClickHouseIOTest: > > > > > Task :sdks:java:io:clickhouse:test > > > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > > java.lang.IllegalStateException > > > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > > java.lang.NullPointerException > > > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > > java.lang.IllegalStateException > > > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > > java.lang.NullPointerException > > > > -- > > Elliotte Rusty Harold > > elh...@ibiblio.org > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
Re: goVet and clickHouse tests failing
It would be nice if this failed gracefully, indicating what dependency is not installed. Kenn On Thu, Nov 21, 2019 at 4:38 AM Elliotte Rusty Harold wrote: > That would explain it. I didn't have Docker installed. I do now, so > let's see if it passes. Ultimately the root cause for a lot of this > seems to be incomplete and conflicting docs. I hope we can invest some > effort into improving this. > > On Thu, Nov 21, 2019 at 6:27 AM Gleb Kanterov wrote: > > > > :sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is > a Java library that supports JUnit tests, providing lightweight, throwaway > instances of common databases, Selenium web browsers, or anything else that > can run in a Docker container. Therefore, ClickHouse tests require a local > Docker environment. It might make sense to move tests using testcontainers > into a separate gradle task if it's a common issue. > > > > On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold < > elh...@ibiblio.org> wrote: > >> > >> I'm slowly working my way through getting the tests to run and pass. > >> We have a lot of work to do on the contributing docs to explain how to > >> setup and run the build. There's clearly a lot of knowledge in > >> developers' heads and workstations that hasn't yet made it into the > >> docs. > >> > >> The latest is a problem finding "github.com/linkedin/goavro" when I > >> run goVet. I'm not a go person. Is this something that requires an > >> extra install? If so, how is it installed? Or is this some error in > >> the build.gradles? Or perhaps my go config is borked and gradle is > >> looking in the wrong directory? > >> > >> > Task :sdks:go:examples:resolveBuildDependencies > >> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go > >> .gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 > : > >> cannot find package "github.com/linkedin/goavro" in any of: > >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro > >> (vendor tree) > >> /home/elharo/.gradle/go/binary/1.12/go/src/ > github.com/linkedin/goavro > >> (from $GOROOT) > >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/linkedin/goavro > >> (from $GOPATH) > >> > >> > Task :sdks:go:examples:goVet FAILED > >> > >> I'm also seeing failures in ClickHouseIOTest: > >> > >> > Task :sdks:java:io:clickhouse:test > >> > >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > >> java.lang.IllegalStateException > >> > >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > >> java.lang.NullPointerException > >> > >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > >> java.lang.IllegalStateException > >> > >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > >> java.lang.NullPointerException > >> > >> -- > >> Elliotte Rusty Harold > >> elh...@ibiblio.org > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
Re: goVet and clickHouse tests failing
That would explain it. I didn't have Docker installed. I do now, so let's see if it passes. Ultimately the root cause for a lot of this seems to be incomplete and conflicting docs. I hope we can invest some effort into improving this. On Thu, Nov 21, 2019 at 6:27 AM Gleb Kanterov wrote: > > :sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is a > Java library that supports JUnit tests, providing lightweight, throwaway > instances of common databases, Selenium web browsers, or anything else that > can run in a Docker container. Therefore, ClickHouse tests require a local > Docker environment. It might make sense to move tests using testcontainers > into a separate gradle task if it's a common issue. > > On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold > wrote: >> >> I'm slowly working my way through getting the tests to run and pass. >> We have a lot of work to do on the contributing docs to explain how to >> setup and run the build. There's clearly a lot of knowledge in >> developers' heads and workstations that hasn't yet made it into the >> docs. >> >> The latest is a problem finding "github.com/linkedin/goavro" when I >> run goVet. I'm not a go person. Is this something that requires an >> extra install? If so, how is it installed? Or is this some error in >> the build.gradles? Or perhaps my go config is borked and gradle is >> looking in the wrong directory? >> >> > Task :sdks:go:examples:resolveBuildDependencies >> Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go >> .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2: >> cannot find package "github.com/linkedin/goavro" in any of: >> >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro >> (vendor tree) >> /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro >> (from $GOROOT) >> >> /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro >> (from $GOPATH) >> >> > Task :sdks:go:examples:goVet FAILED >> >> I'm also seeing failures in ClickHouseIOTest: >> >> > Task :sdks:java:io:clickhouse:test >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.IllegalStateException >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.NullPointerException >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.IllegalStateException >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.NullPointerException >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org -- Elliotte Rusty Harold elh...@ibiblio.org
Re: goVet and clickHouse tests failing
Tentatively, the goVet issue does seem to have been an issue with my Go install I have now cleaned up. The clickhouse issue remains, as do several others I'm working through. I've filed https://issues.apache.org/jira/browse/BEAM-8798 to consolidate and update the instructions for getting to a working build. On Thu, Nov 21, 2019 at 6:10 AM Elliotte Rusty Harold wrote: > > I'm slowly working my way through getting the tests to run and pass. > We have a lot of work to do on the contributing docs to explain how to > setup and run the build. There's clearly a lot of knowledge in > developers' heads and workstations that hasn't yet made it into the > docs. > > The latest is a problem finding "github.com/linkedin/goavro" when I > run goVet. I'm not a go person. Is this something that requires an > extra install? If so, how is it installed? Or is this some error in > the build.gradles? Or perhaps my go config is borked and gradle is > looking in the wrong directory? > > > Task :sdks:go:examples:resolveBuildDependencies > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go > .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2: > cannot find package "github.com/linkedin/goavro" in any of: > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro > (vendor tree) > /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro > (from $GOROOT) > > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro > (from $GOPATH) > > > Task :sdks:go:examples:goVet FAILED > > I'm also seeing failures in ClickHouseIOTest: > > > Task :sdks:java:io:clickhouse:test > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.NullPointerException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.NullPointerException > > -- > Elliotte Rusty Harold > elh...@ibiblio.org -- Elliotte Rusty Harold elh...@ibiblio.org
Re: goVet and clickHouse tests failing
:sdks:java:io:clickhouse:test is using testcontainers. Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. Therefore, ClickHouse tests require a local Docker environment. It might make sense to move tests using testcontainers into a separate gradle task if it's a common issue. On Thu, Nov 21, 2019 at 12:10 PM Elliotte Rusty Harold wrote: > I'm slowly working my way through getting the tests to run and pass. > We have a lot of work to do on the contributing docs to explain how to > setup and run the build. There's clearly a lot of knowledge in > developers' heads and workstations that hasn't yet made it into the > docs. > > The latest is a problem finding "github.com/linkedin/goavro" when I > run goVet. I'm not a go person. Is this something that requires an > extra install? If so, how is it installed? Or is this some error in > the build.gradles? Or perhaps my go config is borked and gradle is > looking in the wrong directory? > > > Task :sdks:go:examples:resolveBuildDependencies > Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go > .gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2 > : > cannot find package "github.com/linkedin/goavro" in any of: > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro > (vendor tree) > /home/elharo/.gradle/go/binary/1.12/go/src/ > github.com/linkedin/goavro > (from $GOROOT) > /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/ > github.com/linkedin/goavro > (from $GOPATH) > > > Task :sdks:go:examples:goVet FAILED > > I'm also seeing failures in ClickHouseIOTest: > > > Task :sdks:java:io:clickhouse:test > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.NullPointerException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.NullPointerException > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
goVet and clickHouse tests failing
I'm slowly working my way through getting the tests to run and pass. We have a lot of work to do on the contributing docs to explain how to setup and run the build. There's clearly a lot of knowledge in developers' heads and workstations that hasn't yet made it into the docs. The latest is a problem finding "github.com/linkedin/goavro" when I run goVet. I'm not a go person. Is this something that requires an extra install? If so, how is it installed? Or is this some error in the build.gradles? Or perhaps my go config is borked and gradle is looking in the wrong directory? > Task :sdks:go:examples:resolveBuildDependencies Resolving ./github.com/apache/beam/sdks/go@/home/elharo/beam/sdks/go .gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/apache/beam/sdks/go/pkg/beam/io/avroio/avroio.go:28:2: cannot find package "github.com/linkedin/goavro" in any of: /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/apache/beam/sdks/go/examples/vendor/github.com/linkedin/goavro (vendor tree) /home/elharo/.gradle/go/binary/1.12/go/src/github.com/linkedin/goavro (from $GOROOT) /home/elharo/beam/sdks/go/examples/.gogradle/project_gopath/src/github.com/linkedin/goavro (from $GOPATH) > Task :sdks:go:examples:goVet FAILED I'm also seeing failures in ClickHouseIOTest: > Task :sdks:java:io:clickhouse:test org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED java.lang.IllegalStateException org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED java.lang.NullPointerException org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED java.lang.IllegalStateException org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED java.lang.NullPointerException -- Elliotte Rusty Harold elh...@ibiblio.org
Re: clickhouse tests failing
These tests are using testcontainers and assume that you have Docker environment locally. On Sun, Sep 8, 2019 at 5:14 PM Lukasz Cwik wrote: > Is passing at head on Jenkins: > https://builds.apache.org/job/beam_PreCommit_Java_Cron/1771/testReport/org.apache.beam.sdk.io.clickhouse/ > > What are the failures your seeing at initialization? (the tests do rely on > setting up zookeeper and other stuff that could fail) > > On Fri, Sep 6, 2019 at 12:36 PM Elliotte Rusty Harold > wrote: > >> At head I noticed the following: >> >> >> $ ./gradlew -p sdks/java/io/ check >> Configuration on demand is an incubating feature. >> >> > Task :sdks:java:io:clickhouse:test >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.IllegalStateException >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.NullPointerException >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.IllegalStateException >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.NullPointerException >> >> 29 tests completed, 4 failed >> >> > Task :sdks:java:io:clickhouse:test FAILED >> >> FAILURE: Build failed with an exception. >> >> >> Is anyone else seeing this? Are the tests expected to pass, or is >> there some requirement (e.g. Java 11) that I might be missing? >> >> -- >> Elliotte Rusty Harold >> elh...@ibiblio.org >> > -- Cheers, Gleb
Re: clickhouse tests failing
Is passing at head on Jenkins: https://builds.apache.org/job/beam_PreCommit_Java_Cron/1771/testReport/org.apache.beam.sdk.io.clickhouse/ What are the failures your seeing at initialization? (the tests do rely on setting up zookeeper and other stuff that could fail) On Fri, Sep 6, 2019 at 12:36 PM Elliotte Rusty Harold wrote: > At head I noticed the following: > > > $ ./gradlew -p sdks/java/io/ check > Configuration on demand is an incubating feature. > > > Task :sdks:java:io:clickhouse:test > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED > java.lang.NullPointerException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.IllegalStateException > > org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED > java.lang.NullPointerException > > 29 tests completed, 4 failed > > > Task :sdks:java:io:clickhouse:test FAILED > > FAILURE: Build failed with an exception. > > > Is anyone else seeing this? Are the tests expected to pass, or is > there some requirement (e.g. Java 11) that I might be missing? > > -- > Elliotte Rusty Harold > elh...@ibiblio.org >
clickhouse tests failing
At head I noticed the following: $ ./gradlew -p sdks/java/io/ check Configuration on demand is an incubating feature. > Task :sdks:java:io:clickhouse:test org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED java.lang.IllegalStateException org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED java.lang.NullPointerException org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED java.lang.IllegalStateException org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED java.lang.NullPointerException 29 tests completed, 4 failed > Task :sdks:java:io:clickhouse:test FAILED FAILURE: Build failed with an exception. Is anyone else seeing this? Are the tests expected to pass, or is there some requirement (e.g. Java 11) that I might be missing? -- Elliotte Rusty Harold elh...@ibiblio.org