Re: [gentoo-dev] dev-lang/go
On Fri, 14 Feb 2014 17:13:27 -0600
William Hubbs willi...@gentoo.org wrote:
On Fri, Feb 14, 2014 at 11:02:49AM -0500, Emery Hemingway wrote:
On Fri, 14 Feb 2014 13:30:10 +0100
Jan Matejka y...@gentoo.org wrote:
On Thu, 13 Feb 2014 11:59:16 -0600
William Hubbs willi...@gentoo.org wrote:
Hi all,
I responded to this a while back, but I guess my email didn't
go out for some reason.
As the primary go maintainer, I do want to be involved in
this. :-)
On Tue, Feb 11, 2014 at 01:38:44AM +0100, yac wrote:
On Mon, 30 Dec 2013 15:48:17 -0500
Emery Hemingway em...@vfemail.net wrote:
I really like working with Go, and would like to see a
means of merging Go packages with Portage. In short I am
asking if anyone else is interested in a Go project.
I might be. I have packaged something for private use but it
just a bunch of hacks. Anyway, I have some production go code.
For those who aren't familiar with Go, I will sumarise why
Portage and Go do not play well together.
Go is static linked by default.
The Go compiler creates static libraries and binaries.
Libraries compilied with different versions of Go (1.1/1.2)
may not be linked into the same binary.
Haskell is staticaly linked as well (by default) and you can
see the gentoo haskell project. I don't see this as a
problem, we just will have all dependencies in DEPEND and
will have to scope on the go compiler version under something
like /usr/lib/go-1.{1,2}/...
That could be done easily enough, but what about the tools
in /usr/bin (there aren't many, but there are a couple), and
these do not change name with each version of go.
Please see what python does for different python versions (which
you omitted from my previous email).
I omitted it, because thinking about it, we don't need to worry about
this. There isn't a reason you would want go 1.1 and go 1.2 on your
system. Source level compatibility is guaranteed for all go1 programs
[1].
I've modified the go-1.2 ebuild to install to usr/lib/go1.2 and I'm
working on an eselect module to manage the symlink to
usr/bin/[go,gofmt]
I would just install to /usr/lib/go1 and not worry about the eselect
module; there should not be a need to keep several versions of go1
around, again, because go1.x releases will be source compatible.
We could even just leave this as /usr/lib/go, because upstream doesn't
even know if a go-2 specification will happen.
The default GOROOT that go looks at for base libraries seems to be
compiled in so this should be pretty easy, like python but simplier.
It looks for standard libraries in GOROOT_FINAL which is set in the
ebuild and compiled into the binaries.
Third party libraries are interesting in this case, because, all of
the third party libraries we install will not be usable once the user
upgrades from say go-1.2 to go-1.3. However, rebuilding those
libraries from source will work.
William
[1] http://golang.org/doc/go1compat
The reason I thought go should be slotting was that all compliled
libraries would break when go was replaced.
Mabye the only the library source could be installed, and a cache of
compiled libraries could be overlayed over that...
Re: [gentoo-dev] dev-lang/go
On Fri, 14 Feb 2014 13:30:10 +0100
Jan Matejka y...@gentoo.org wrote:
On Thu, 13 Feb 2014 11:59:16 -0600
William Hubbs willi...@gentoo.org wrote:
Hi all,
I responded to this a while back, but I guess my email didn't go out
for some reason.
As the primary go maintainer, I do want to be involved in this. :-)
On Tue, Feb 11, 2014 at 01:38:44AM +0100, yac wrote:
On Mon, 30 Dec 2013 15:48:17 -0500
Emery Hemingway em...@vfemail.net wrote:
I really like working with Go, and would like to see a means of
merging Go packages with Portage. In short I am asking if anyone
else is interested in a Go project.
I might be. I have packaged something for private use but it just
a bunch of hacks. Anyway, I have some production go code.
For those who aren't familiar with Go, I will sumarise why
Portage and Go do not play well together.
Go is static linked by default.
The Go compiler creates static libraries and binaries. Libraries
compilied with different versions of Go (1.1/1.2) may not be
linked into the same binary.
Haskell is staticaly linked as well (by default) and you can see
the gentoo haskell project. I don't see this as a problem, we
just will have all dependencies in DEPEND and will have to scope
on the go compiler version under something
like /usr/lib/go-1.{1,2}/...
That could be done easily enough, but what about the tools
in /usr/bin (there aren't many, but there are a couple), and these
do not change name with each version of go.
Please see what python does for different python versions (which you
omitted from my previous email).
I've modified the go-1.2 ebuild to install to usr/lib/go1.2 and I'm
working on an eselect module to manage the symlink to usr/bin/[go,gofmt]
The default GOROOT that go looks at for base libraries seems to be
compiled in so this should be pretty easy, like python but simplier.
An eclass could look at a GO_MINIMUM variable and install for each
version go that is present and matches. Dropping old versions of go
will be easy because linking wont break, and new releases should be
forwards compatible. Maybe 3rd party library sources could be stored in
a version agnostic directory and symlinked in to usr/lib/goX.X/gentoo
to deduplicate the files?
Go libraries are usually unversioned.
Libraries outside the system library are resolved with an import
statement that specifies a source code repository, such as a git
or mecurial repository. Most often Go libraries are installed
using the 'go get' tool that clones a repository, and simply
assumes HEAD/tip is the best revision to build against. There is
some support for using git tags but it is not well documented.
Often these libraries are very small for the sake of reuse and
to keep APIs simple.
My understanding is that a library repo will have branches based on
the version of go, so for example, it might have a branch called
go-1 which will be where go get will look to find the latest
version of the code that works with go-1.x.
In this case we just have to require upstream to make releases or
publish either live ebuilds, or ebuilds versioned something like
0_pre-MM-DD.ebuild [1]
I don't think we are going to be able to require upstream to make
releases, so that leaves us with:
1) using live ebuilds, which will never be allowed to have keywords
by gentoo policy, or
2) publishing snapshots, which also means we have to create tarballs
to match them. This will be a lot of work for us as maintainers.
Also, the only way we will know when a new version of the library
is released is to closely monitor the upstream git repository.
As I said in previous email, I think at least part of go community
sees this as an issue and this is something we can not solve right
now but rather need to work on this on case-by-case basis.
Some upstreams may be willing to do releases / follow semver.org or
something like that. But there will also be upstream which won't and
that's fine, we should be able to handle both cases.
Anyway, asking the upstream to do a release is simple enough and you
won't know until you ask.
The other concern, which I believe zero was talking about is, once a
library is installed in GOPATH, I don't think the go build system
rebuilds it. In other words, go get will see that it is already
there and I don't think it goes back out to the net to check for any
changes.
I think when doing a `go build` it will check if newer version is
available and print a warning.
We may have to make some sort of utitilty to parse sources and check
for updates, and roll tarballs to mirror.
William
--
Jan Matějka| Gentoo Developer
https://gentoo.org | Gentoo Linux
GPG: A33E F5BC A9F6 DAFD 2021 6FB6 3EBF D45B EEB6 CA8B
Re: [gentoo-dev] dev-lang/go
I think it would be good idea to start a separate gentoo-golang repository (github?) and treat it more (to keep it aligned with the way gentoo works) or less (to speed up the development) as if it were gx86. In the organization part, I think we could inspire ourself in the way gentoo-haskell works. Jan Matějka| Gentoo Developer https://gentoo.org | Gentoo Linux GPG: A33E F5BC A9F6 DAFD 2021 6FB6 3EBF D45B EEB6 CA8B Thanks the comments, I'm not familar with what portage does with haskel so I'll take a look. I'll collect the notes that have been made here and start a project on github or gitorious in the next few days and post the details here. Emery
Re: [gentoo-dev] dev-lang/go
I made an overlay for Go eclasses and packages: https://github.com/gentoo-golang/overlay If anyone is interested ping 'emery' in #gentoo-dev-help and I'll add you to the github organization. There is an overlay skeleton at master, and a first draft eclass and ebuilds for btcd on another branch. I'll start adding what was discussed here as issues on the repo. Emery
[gentoo-dev] dev-lang/go
I really like working with Go, and would like to see a means of merging Go packages with Portage. In short I am asking if anyone else is interested in a Go project. For those who aren't familiar with Go, I will sumarise why Portage and Go do not play well together. Go is static linked by default. The Go compiler creates static libraries and binaries. Libraries compilied with different versions of Go (1.1/1.2) may not be linked into the same binary. It is possible to compile dynamicly and that may involve using the GCC frontend, which is probably less tested and less optimized. Go libraries are usually unversioned. Libraries outside the system library are resolved with an import statement that specifies a source code repository, such as a git or mecurial repository. Most often Go libraries are installed using the 'go get' tool that clones a repository, and simply assumes HEAD/tip is the best revision to build against. There is some support for using git tags but it is not well documented. Often these libraries are very small for the sake of reuse and to keep APIs simple. If all that sounds bad, thats because it is. Is it worth versioning many tiny libraries or do we simply cache the repositiories and blame upstream when things stop compiling? A have made an eclass for Go and an ebuild for the bitcoin node written in pure Go to atleast prove that all this is possible. These are in the 'emery' overlay: http://git.overlays.gentoo.org/gitweb/?p=user/emery.git;a=tree;f=eclass http://git.overlays.gentoo.org/gitweb/?p=user/emery.git;a=tree;f=dev-go http://git.overlays.gentoo.org/gitweb/?p=user/emery.git;a=tree;f=net-p2p/btcd The eclass it a bit of a mess but it works, having done that, I would say that making ebuilds for every go library is tedious, but can be done almost entirely with boilerplate, almost every time. The eclasss installs go source and static libraries to /usr/lib/go/gentoo (source code and .a library are required to link). The problem is when Go is updated, this folder may get wiped out, and if it isn't, those libraries will be incompatable with the new release anyway. The other solution I see is to make a Go directory in /var/cache or something like it and just manage it as a cache. Libraries may come and go but that is fine. Bare repositories may be cached in DISTDIR just like the git and mercurial eclasses do. Doing things this way may require a specific utility for Portage that wraps the Go toolchain, which I would be willing to create. This utility could probably automatically resolve and fetch the libraries that are required as opposed to making an ebuild for each library, but that raises the problem of assuming the developers of each library maintain consistant quality and security. The problem is Go makes it trivial to build from source, but it does that in a very different and less precise way than Gentoo. There is always the option of build bots and installing binaries to /opt... Emery
