Re: lang/go failes to build with poudriere, since 2018-04-05
Ah, you'll note: fork/exec implicated here. Looks like this guy strikes again: https://github.com/golang/go/issues/15658 It pains me to say but Go on FreeBSD is (and has always been) broken. Should be fine if you don't exec. Something that might help, is setting GOMAXPROCS=1. Derek On 18-04-25 07:45 AM, Steven Hartland wrote: Builds fine on 11.1-RELEASE-p6 here: [00:04:02] Committing packages to repository [00:04:02] Removing old packages [00:04:02] Built ports: lang/go [ports11-1-multiplay] [2018-04-25_11h37m16s] [committing:] Queued: 1 Built: 1 Failed: 0 Skipped: 0 Ignored: 0 Tobuild: 0 Time: 00:04:01 [00:04:02] Logs: /usr/local/poudriere/data/logs/bulk/ports11-1-multiplay/2018-04-25_11h37m16s [00:04:02] Cleaning up svn info Path: . Working Copy Root Path: /usr/local/poudriere/ports/multiplay URL: https://svn.freebsd.org/ports/head Relative URL: ^/head Repository Root: https://svn.freebsd.org/ports Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5 Revision: 468275 Node Kind: directory Schedule: normal Last Changed Author: tobik Last Changed Rev: 468275 Last Changed Date: 2018-04-25 11:08:41 + (Wed, 25 Apr 2018) Regards Steve On 25/04/2018 12:07, Bjarne wrote: I got a job to rebuild all packages every night, but since 2018-04-05 building /usr/ports/lang/go is failing. Apparently 2018-04-05 some major changed was introduced, since 331 pakackes was rebuilt. Not sure what it was, I saw nothing in UPATING. Top of logfile: Building lang/go build started at Thu Apr 5 03:04:07 UTC 2018 port directory: /usr/ports/lang/go package name: go-1.10.1,1 building for: FreeBSD freebsd_11-1-HEAD-job-03 11.1-RELEASE-p9 FreeBSD 11.1-RELEASE-p9 amd64 maintained by: jlaff...@freebsd.org Makefile ident: $FreeBSD: head/lang/go/Makefile 466249 2018-04-02 18:58:11Z jlaffaye $ Poudriere version: 3.2.6 Host OSVERSION: 1101001 Jail OSVERSION: 1101001 actually build fails both on freebsd 10.3 and 11.1 Running poudriere testport -j freebsd_11-1 -p HEAD -i -o lang/go : ===> === =>> Recording filesystem state for prebuild... done ===> ===> go-1.10.1,1 depends on package: go14>=1.4 - found ===> Configuring for go-1.10.1,1 === === ===> Building for go-1.10.1,1 cd /wrkdirs/usr/ports/lang/go/work/go/src && GOROOT=/wrkdirs/usr/ports/lang/go/work/go GOROOT_FINAL=/usr/local/go GOROOT_BOOTSTRAP=/usr/local/go14 GOBIN= GOARCH=amd64 GOOS=freebsd GO386= /bin/sh make.bash -ap: not found go: not found Building Go cmd/dist using /usr/local/go14. Building Go toolchain1 using /usr/local/go14. Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1. Building Go toolchain2 using go_bootstrap and Go toolchain1. runtime: gp: gp=0xc420499380, goid=38, gp->atomicstatus=2 runtime: g: g=0xc42c00, goid=0, g->atomicstatus=0 fatal error: bad g->status in ready runtime stack: runtime.throw(0x6f776b, 0x16) /usr/local/go/src/runtime/panic.go:616 +0x81 runtime.ready(0xc420499380, 0x5, 0xc420499201) /usr/local/go/src/runtime/proc.go:601 +0x27b runtime.goready.func1() /usr/local/go/src/runtime/proc.go:302 +0x38 runtime.systemstack(0x0) /usr/local/go/src/runtime/asm_amd64.s:409 +0x79 runtime.mstart() /usr/local/go/src/runtime/proc.go:1175 goroutine 37 [running]: runtime.systemstack_switch() /usr/local/go/src/runtime/asm_amd64.s:363 fp=0xc4204a06c0 sp=0xc4204a06b8 pc=0x450c00 runtime.goready(0xc420499380, 0x5) /usr/local/go/src/runtime/proc.go:301 +0x4b fp=0xc4204a06f0 sp=0xc4204a06c0 pc=0x42a74b runtime.readyWithTime(0xc4201a4c00, 0x5) /usr/local/go/src/runtime/sema.go:83 +0x41 fp=0xc4204a0710 sp=0xc4204a06f0 pc=0x43a071 runtime.semrelease1(0x8885e4, 0x0) /usr/local/go/src/runtime/sema.go:194 +0x12d fp=0xc4204a0768 sp=0xc4204a0710 pc=0x43a54d sync.runtime_Semrelease(0x8885e4, 0xc4204a0700) /usr/local/go/src/runtime/sema.go:66 +0x34 fp=0xc4204a0788 sp=0xc4204a0768 pc=0x439f84 sync.(*Mutex).Unlock(0x8885e0) /usr/local/go/src/sync/mutex.go:201 +0x75 fp=0xc4204a07b0 sp=0xc4204a0788 pc=0x4674d5 sync.(*RWMutex).Unlock(0x8885e0) /usr/local/go/src/sync/rwmutex.go:132 +0x7e fp=0xc4204a07e0 sp=0xc4204a07b0 pc=0x46824e syscall.forkExec(0xc4202f0440, 0x3d, 0xc4202aa1e0, 0x2, 0x2, 0xc4204a09d8, 0x20, 0x0, 0xc4200aa5a0) /usr/local/go/src/syscall/exec_unix.go:199 +0x3e1 fp=0xc4204a0900 sp=0xc4204a07e0 pc=0x46c031 syscall.StartProcess(0xc4202f0440, 0x3d, 0xc4202aa1e0, 0x2, 0x2, 0xc4204a09d8, 0x2, 0x4, 0x0, 0x0) /usr/local/go/src/syscall/exec_unix.go:241 +0x64 fp=0xc4204a0958 sp=0xc4204a0900 pc=0x46c484 os.startProcess(0xc4202f0440, 0x3d, 0xc4202aa1e0, 0x2, 0x2, 0xc4204a0b80
Re: Packaging Go Libs
Agree with previous sentiments, and: On 17-04-18 10:59 PM, Christopher Hall wrote: You should use built in golang vendoring to ensure these dependencies, as their is no guarantee that someone won't update the library port and your app would break, so doing that is very fragile Currently the GH_TUPLE method is working as it specifies exact dependency versions or specific git hashes. but we made several attempts at submodules in vendor dir but have had problems building and go get -u breaks things. I am wondering if you might suggest a tool or do any other programs in ports use such a dependency tool. Last time I searched ports tree I only saw GH_TUPLE used so I just followed that method. From my point of view, the only thing that should be in the vendor directory on checkout is the version-lock file. This is different for different tools. I have been using gb, as it makes the most sense to me: https://getgb.io/ sysutils/hfm uses this godep is also popular, from what I understand: https://godoc.org/github.com/tools/godep Vendoring changed internally in go with a GO15VENDOREXPERIMENT build environment variable (and then default in 1.6), although I have not yet played with it: https://docs.google.com/document/d/1Bz5-UB7g2uPBdOx-rw5t9MxJwkfpx90cqG9AFL0JAYo/edit ... from the docs, it sounds like it would be compatible with gb from a build standpoint - and you simply could use gb to track, fetch and lock versions in development - the same thing the ports GH_TUPLE covers. What you do when it's not hosted at github, well Derek ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Recording TIMESTAMPs in distinfo for reproducible builds work
On 16-05-12 02:08 PM, Ed Maste wrote: Baptiste and I have been looking at reproducible builds in the FreeBSD ports tree, and one thing we'll need is a consistent timestamp that doesn't change when a port is rebuilt without changes. Just wondering if any work has gone into consuming these timestamps in the tools? Specifically, looking at tar(1) from -head, I see that it doesn't current support the -mtime flag that GNU tar does, to be able to set the timestamp on all of the files being archived. Any thoughts on what the canonical way will be: e.g. a pass in advance with find/touch or if BSD tar is planned to support this internally? Thanks! Derek ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Pulling from github as a vendorized dependency in poudriere
On 16-02-19 08:42 PM, Derek (freebsd lists) wrote: On 16-02-19 02:16 PM, Brooks Davis wrote: On Fri, Feb 19, 2016 at 07:13:04AM -0500, Derek (freebsd lists) wrote: I can provide more detail, but would like to know if I'm doing something horribly wrong first (i.e. trying to access the network with gb as a make target, versus some other way to do this). When you run "gb vendor", what stage does that happen in? IIRC poudriere only configures network access during the fetch stage so you must find a way to run it as part of the fetch process or capture and emulate its result. Indeed, I was doing it in the post-extract stage, as there is some patching that happens to the pulled-in sources from there, and the vendor information is extracted from the tarball. I see now, poudriere wants "no network", as you mention: https://fossil.etoilebsd.net/poudriere/doc/trunk/doc/design.mkd I've submitted a patch to doc: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207416 Additionally, I was able to pull in my depends via: 5.4.3.1. Fetching Multiple Files from GitHub https://www.freebsd.org/doc/en/books/porters-handbook/makefile-distfiles.html#idp63641808 So it wasn't so bad. Dunno what it would look like if my depends were in a few different places though. Thanks all! Derek ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: Pulling from github as a vendorized dependency in poudriere
On 16-02-19 02:16 PM, Brooks Davis wrote: On Fri, Feb 19, 2016 at 07:13:04AM -0500, Derek (freebsd lists) wrote: I can provide more detail, but would like to know if I'm doing something horribly wrong first (i.e. trying to access the network with gb as a make target, versus some other way to do this). When you run "gb vendor", what stage does that happen in? IIRC poudriere only configures network access during the fetch stage so you must find a way to run it as part of the fetch process or capture and emulate its result. Thanks for the quick response - much appreciated. Indeed, I was doing it in the post-extract stage, as there is some patching that happens to the pulled-in sources from there, and the vendor information is extracted from the tarball. I see now, poudriere wants "no network", as you mention: https://fossil.etoilebsd.net/poudriere/doc/trunk/doc/design.mkd I wonder, with the rise of vendorizing tools, ala composer, gb, npm, virtualenv (possibly?), etc, etc... - are the assumptions about the build steps of ports still valid? i.e. - is there a make vendor target somewhere between either extract, and patch, or patch and build (that doesn't yet exist), where network access is still useful? Rhetorical - I'm sure it's come up before (and will do more digging to this effect). Better yet - how do other ports deal with this? Would love to hear from some maintainers who are happy with what they've done to handle this - if anyone is available? Even just the port name would be great. My current plan is to shoehorn a custom fetch/extract/patch into the port Makefile, based on Brooks' valued information, but it doesn't feel great. Thanks again - most helpful! Derek ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Pulling from github as a vendorized dependency in poudriere
Hi, I'm attempting to port a project over, and it is dependent on a "vendorizing" program, which then pulls in the source of the dependencies to build. (The final artifact is statically linked, so there shouldn't be a problem as far as installing unversioned libraries outside of the package/ports framework, for example.) My port works fine in my local ports tree, and also I can build the project by hand without the port no problem. When I try to use poudriere testport, and it's time to pull in the dependencies (as part of a build step), I get: fatal: unable to access 'https://github.com/foo/bar': Couldn't connect to server Is there something in poudriere that makes the network special? i.e. do build scripts not have network access in that context? I can provide more specifics on my project: - The vendorizing program is "gb vendor" - I have PATCH_DEPENDS on gb, and git - The pre-patch step runs `make deps`, a target which ultimately runs `gb vendor restore`, the step that gives the error message, and the build fails I can provide more detail, but would like to know if I'm doing something horribly wrong first (i.e. trying to access the network with gb as a make target, versus some other way to do this). Thanks! Derek ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
