Looking at the current version of the `verify-release-candidate.sh` script,
it looks like it does set GOROOT and GOPATH to be subdirs of
$ARROW_TMPDIR/go. It installs go to ${ARROW_TMPDIR}/go etc. But it looks
like it checks $USE_CONDA and if that is >0 it uses Conda to install Go but
doesn't set GOPATH/GOROOT/etc. env vars and doesn't run `go install` at all.
In the case where you're using Conda, `go install` would install the
binaries to something under the conda_prefix. I'll open a PR to run `go
install ./...` in test_go() and if you see anything wrong with the
description above as far as the logic for USE_CONDA vs installing and using
it entirely under ARROW_TMPDIR, please add on to the PR. I'll tag you in it
Kou.
That said, as before I still don't believe this should be a blocker at all
for the v10.0.0 release.
--Matt
On Mon, Oct 24, 2022 at 11:08 PM Sutou Kouhei <k...@clear-code.com> wrote:
> Hi,
>
> > What version of Go is your system Go?
>
> $ go version
> go version go1.19 linux/amd64
>
> > Also, can you go to the `go` subdirectory of the repo and
> > run `go install ./...` then try again?
>
> Yes. "go install ./..." created
> ~/go/bin/arrow-flight-integration-{client,server}.
>
> Should we run "go install ./..." in test_go() in
> verify-release-candidate.sh?
>
> BTW, can we set GOPATH to under $ARROW_TMPDIR such as
> $ARROW_TMPDIR/go with system Go? I hope that
> verify-release-candidate.sh doesn't output any files outside
> $ARROW_TMPDIR.
>
>
> Thanks,
> --
> kou
>
> In <CAH4123YeLvjw3Zb2Ca0frkk2=nbg7l-arosxgj9yh13oidt...@mail.gmail.com>
> "Re: [VOTE] Release Apache Arrow 10.0.0 - RC0" on Mon, 24 Oct 2022
> 19:58:40 -0400,
> Matt Topol <zotthewiz...@gmail.com> wrote:
>
> > @kou What version of Go is your system Go? Also, can you go to the `go`
> > subdirectory of the repo and run `go install ./...` then try again?
> Running
> > that `go install ./...` command should build the binaries that were
> > missing. Unless you have the environment variable `GOBIN` set, it will
> > place the built binaries into `$(go env GOPATH)/bin`. It's possible that
> > the issue is needing GOPATH to be set since the archery scripts default
> to
> > using $HOME/go if there's no GOPATH set.
> >
> > I agree that it shouldn't be a blocker, but I'll take a look at the
> verify
> > scripts to see what can be changed to improve this.
> >
> > --Matt
> >
> > On Mon, Oct 24, 2022 at 5:14 PM Sutou Kouhei <k...@clear-code.com> wrote:
> >
> >> +1
> >>
> >> I ran the followings on Debian GNU/Linux sid:
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_SOURCE=1 \
> >> TEST_INTEGRATION_GO=0 \
> >> LANG=C \
> >> TZ=UTC \
> >> CUDAToolkit_ROOT=/usr \
> >> ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
> >> -Dxsimd_SOURCE=BUNDLED" \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_APT=1 \
> >> LANG=C \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_BINARY=1 \
> >> LANG=C \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_JARS=1 \
> >> LANG=C \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_WHEELS=1 \
> >> LANG=C \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> * TEST_DEFAULT=0 \
> >> TEST_YUM=1 \
> >> LANG=C \
> >> dev/release/verify-release-candidate.sh 10.0.0 0
> >>
> >> with:
> >>
> >> * .NET SDK (6.0.400)
> >> * Python 3.10.6
> >> * Python 3.9.13
> >> * gcc (Debian 12.2.0-1) 12.2.0
> >> * nvidia-cuda-dev 11.4.3-4+b1
> >> * openjdk version "18.0.2-ea" 2022-07-19
> >> * ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
> >>
> >> Notes:
> >>
> >> Integration test was failed because Go's
> >> arrow-flight-integration-client and
> >> arrow-flight-integration-server weren't found.
> >>
> >> For example:
> >>
> >> ##########################################################
> >> Flight: C++ serving, Go requesting
> >> ##########################################################
> >> ==========================================================
> >> Testing file duplicate_fieldnames
> >> ==========================================================
> >> -- Skipping test because consumer Go does not support
> >> ==========================================================
> >> Testing file flight_sql:extension
> >> ==========================================================
> >> -- Skipping test because consumer Go does not support
> >> ==========================================================
> >> Testing file datetime
> >> ==========================================================
> >> Traceback (most recent call last):
> >> File
> >>
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/runner.py",
> >> line 376, in _run_flight_test_case
> >> consumer.flight_request(port, **client_args)
> >> File
> >>
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/tester_go.py",
> >> line 121, in flight_request
> >> run_cmd(cmd)
> >> File
> >>
> "/tmp/arrow-10.0.0.pp1DF/apache-arrow-10.0.0/dev/archery/archery/integration/util.py",
> >> line 133, in run_cmd
> >> output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
> >> File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
> >> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
> >> File "/usr/lib/python3.10/subprocess.py", line 501, in run
> >> with Popen(*popenargs, **kwargs) as process:
> >> File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
> >> self._execute_child(args, executable, preexec_fn, close_fds,
> >> File "/usr/lib/python3.10/subprocess.py", line 1845, in _execute_child
> >> raise child_exception_type(errno_num, err_msg, err_filename)
> >> FileNotFoundError: [Errno 2] No such file or directory:
> >> '/home/kou/go/bin/arrow-flight-integration-client'
> >>
> >> I skipped integration test with Go with
> >> TEST_INTEGRATION_GO=0. I don't think that this is a blocker
> >> because it passed on GitHub Actions:
> >>
> >>
> https://github.com/ursacomputing/crossbow/actions/runs/3293278249/jobs/5429607084#step:5:74963
> >>
> >> I think that this was caused because I used system Go. (The
> >> above verification on GitHub Actions installs Go from
> >> https://dl.google.com/go/ .) We may need to improve our
> >> verification script for system Go. (Should we set GOPATH or
> >> something?)
> >>
> >>
> >> Thanks,
> >> --
> >> kou
> >>
> >>
> >> In <20221021.150638.710084994470067293....@clear-code.com>
> >> "[VOTE] Release Apache Arrow 10.0.0 - RC0" on Fri, 21 Oct 2022
> 15:06:38
> >> +0900 (JST),
> >> Sutou Kouhei <k...@clear-code.com> wrote:
> >>
> >> > Hi,
> >> >
> >> > I would like to propose the following release candidate (RC0) of
> Apache
> >> > Arrow version 10.0.0. This is a release consisting of 470
> >> > resolved JIRA issues[1].
> >> >
> >> > This release candidate is based on commit:
> >> > 89f9a0948961f6e94f1ef5e4f310b707d22a3c11 [2]
> >> >
> >> > The source release rc0 is hosted at [3].
> >> > The binary artifacts are hosted at [4][5][6][7][8][9][10][11].
> >> > The changelog is located at [12].
> >> >
> >> > Please download, verify checksums and signatures, run the unit tests,
> >> > and vote on the release. See [13] for how to validate a release
> >> candidate.
> >> >
> >> > See also a verification result on GitHub pull request [14].
> >> >
> >> > The vote will be open for at least 72 hours.
> >> >
> >> > [ ] +1 Release this as Apache Arrow 10.0.0
> >> > [ ] +0
> >> > [ ] -1 Do not release this as Apache Arrow 10.0.0 because...
> >> >
> >> > [1]:
> >>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20%28Resolved%2C%20Closed%29%20AND%20fixVersion%20%3D%2010.0.0
> >> > [2]:
> >>
> https://github.com/apache/arrow/tree/89f9a0948961f6e94f1ef5e4f310b707d22a3c11
> >> > [3]:
> >> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-10.0.0-rc0
> >> > [4]: https://apache.jfrog.io/artifactory/arrow/almalinux-rc/
> >> > [5]: https://apache.jfrog.io/artifactory/arrow/amazon-linux-rc/
> >> > [6]: https://apache.jfrog.io/artifactory/arrow/centos-rc/
> >> > [7]: https://apache.jfrog.io/artifactory/arrow/debian-rc/
> >> > [8]: https://apache.jfrog.io/artifactory/arrow/java-rc/10.0.0-rc0
> >> > [9]: https://apache.jfrog.io/artifactory/arrow/nuget-rc/10.0.0-rc0
> >> > [10]: https://apache.jfrog.io/artifactory/arrow/python-rc/10.0.0-rc0
> >> > [11]: https://apache.jfrog.io/artifactory/arrow/ubuntu-rc/
> >> > [12]:
> >>
> https://github.com/apache/arrow/blob/89f9a0948961f6e94f1ef5e4f310b707d22a3c11/CHANGELOG.md
> >> > [13]:
> >>
> https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates
> >> > [14]: https://github.com/apache/arrow/pull/14466
> >>
>
