On 05/13/15 07:41, Marc Espie wrote:
> I've fixed quite a few bugs recently, and I think it's now stable.
Thanks.
> If you've tried in the past, here's a list of significant changes.
>
> [ ... snip ... ]
>
> - you should set up a build user on every host that wants to build things.
> That build user does NOT need sudo rights. It does NOT need network access.
The build_user needs more than default limits (I saw "unable to fork" in the
build logs). I use a class (w/ probably overly generous limits) for this user,
ie:
dpb:\
:datasize-cur=7600M:\
:datasize-max=infinity:\
:stacksize-max=64M:\
:stacksize-cur=16M:\
:maxproc-max=1024:\
:maxproc-cur=512:\
:openfiles-max=2048:\
:openfiles-cur=1088:\
:ignorenologin:\
:requirehome@:\
:tc=default:
> [... skip ...]
>
> dpb itself creates some shared directories. It does so as root and changes
> ownership accordingly. In case of NFS setups where root does not have all
> access, set DROPPRIV=1 in your hosts file.
I didn't see this in dpb(1) so haven't been using it. I believe root has full
access to the shared NFS resource, ie.:
x6v64:build/packages 86>sudo su -l root
x6v64:/root 11#mount | grep nas3/work
nas3:/work on /nas3/work type nfs (nodev, nosuid, v3, udp, timeo=100,
retrans=101)
x6v64:/root 12#ls -l /nas3/work/OpenBSD/packages/amd64
total 1560
drwxrwxr-x 6 rd dpb 4096 Jan 31 2014 Old
drwxrwxr-x 2 rd dpb 507904 May 15 07:23 all
drwxrwxr-x 2 rd dpb 4096 Jan 23 2013 cache
drwxrwxrwx 2 rd dpb 487424 May 15 07:23 cdrom
drwxrwxrwx 2 rd dpb 487424 May 15 07:23 ftp
drwxrwxrwx 2 rd dpb 86016 May 15 06:55 no-arch
drwxrwxr-x 2 rd dpb 4096 May 15 07:23 tmp
x6v64:/root 13#touch /nas3/work/OpenBSD/packages/amd64/aaa /tmp/aaa
x6v64:/root 14#ls -lt /nas3/work/OpenBSD/packages/amd64/aaa /tmp/aaa
-rw-r--r-- 1 root wheel 0 May 15 07:39 /tmp/aaa
-rw-r--r-- 1 root wheel 0 May 15 07:39 /nas3/work/OpenBSD/packages/amd64/aaa
> In case of "shared directory" situations, there are also now knobs in
> bsd.port.mk, which was switched to using install -d to
> "create" common directories.
>
> WRKOBJDIR_MODE ?=
> DISTDIR_MODE ?= ${WRKOBJDIR_MODE}
> PACKAGE_REPOSITORY_MODE ?= ${WRKOBJDIR_MODE}
> LOCKDIR_MODE ?= ${WRKOBJDIR_MODE}
> PLISTDIR_MODE ?= ${WRKOBJDIR_MODE}
>
>
> You could e.g., set up WRKOBJDIR_MODE = -g wports -m 775
> to have group writable directories.
>
> A "quirk" of install -d is that it will complain when it cannot change
> modes/ownership, but not error out if the given directory was indeed
> created. So you may even want to
> WKROBJDIR_MODE = -g wports -m 775 2>/dev/null
> to shut it up, since make will still exit properly with an error code
> in case of a fatal error.
Is this "quirk" a temporary issue? "Shut it up" doesn't seem like your
preferred approach. ;-}
Is this the reason for the (new to me) message "Operation not permitted" in the
package stage?
archiving
install: /nas3/work/OpenBSD/ports/plist/amd64: Operation not permitted
/nas3/work/OpenBSD/ports/plist/amd64/libgpod-0.8.0p5 was updated
devel/gettext:gettext->=0.10.38:gettext-0.19.4 ->
devel/gettext:gettext->=0.10.38:gettext-0.19.4p0
devel/glib2:glib2-*:glib2-2.44.0p0 -> devel/glib2:glib2-*:glib2-2.44.1
Link to /nas3/work/OpenBSD/packages/amd64/ftp/libgpod-0.8.0p5.tgz
install: /nas3/work/OpenBSD/packages/amd64/ftp: Operation not permitted
Link to /nas3/work/OpenBSD/packages/amd64/cdrom/libgpod-0.8.0p5.tgz
install: /nas3/work/OpenBSD/packages/amd64/cdrom: Operation not permitted
>>> Running clean in audio/libgpod,,-main at 1431690021
===> audio/libgpod,,-main
===> Cleaning for libgpod-0.8.0p5
> If you want to setup a chroot, things are slightly more complicated.
> There will probably be a script to do that in the not so distant future.
> One goal is to do each distinct build in a separate chroot.
> (we're about 3/4 of the way there).
I've been using chroot soon after you first made it available since I don't
have dedicated build boxes but do have spare cycles. I also use vm's without
chroot since they are easily recreated but terribly slow compared to bare
metal. So, I have a mix of {non-,}chroot hosts. Since you initially
introduced the chroot improvements on May 1, I've been mostly using a single
vm(localhost) w/ two remote chroot hosts.
Since your updates of yesterday, I had some problems w/ the distributed build.
So, I tried to narrow the problem and have finished a localhost only build. I
noticed it finished w/ no outstanding errors in the term-report.log; however,
11 items were on the L= line. Looking at those, I see the lockfile
databases.pkglocatedb had needed=ccache-3.2.2 portslist-4.3. ccache was
installed;however, portslist-4.3 had just been packaged. I'm probably getting
more confused trying to interpret the logs, so I tarred up all of the dpb logs
(term-report,engine,locks,...)[1] as well the pkg_info for the single host and
a listing of the $PKG_PATH directory. Perhaps those logs/listings will help
understand the problem.
[1]<http://arp.thrush.com/openbsd/dpb/today.201505150842/>
This is the contents of the hosts file for the above single host dpb run:
x6v64:build/packages 131>cat dpb_hosts.amd64
DEFAULT build_user=dpb memory=200M stuck=1000 arch=amd64
DIRMODE=0770
FETCH_USER=dpb_fetch
LOG_USER=rd
STARTUP=/home/rd/OpenBSD/build/packages/dpb_start
UNPRIV_USER=dpb_unpriv
dpb@localhost jobs=1 memory=1G sf=1 stuck=2000 arch=amd64
I've appended the dmesg for that single host.
> Many thanks to people who gave me feedback, among others:
> RD Thrush, Mark Patruck, Stuart Henderson, Antoine Jacoutot.
Thanks for the ack.
OpenBSD 5.7-current (GENERIC) #910: Wed May 13 17:54:36 MDT 2015
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 3082747904 (2939MB)
avail mem = 2985586688 (2847MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) II X6 1055T Processor, 2815.02 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,MWAIT,NXE,FFXSR,LONG,3DNOW2,3DNOW,LAHF,AMCR8
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu0: AMD erratum 721 detected and fixed
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpibat0 at acpi0: BAT0 not present
acpiac0 at acpi0: AC unit online
acpivideo0 at acpi0: GFX0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
vga1 at pci0 dev 2 function 0 "InnoTek VirtualBox Graphics Adapter" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x02: apic 1 int 19, address
08:00:27:d6:8b:12
"InnoTek VirtualBox Guest Service" rev 0x00 at pci0 dev 4 function 0 not
configured
auich0 at pci0 dev 5 function 0 "Intel 82801AA AC97" rev 0x01: apic 1 int 21,
ICH AC97
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
ohci0 at pci0 dev 6 function 0 "Apple Intrepid USB" rev 0x00: apic 1 int 22,
version 1.0
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: SMBus disabled
ehci0 at pci0 dev 11 function 0 "Intel 82801FB USB" rev 0x00: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ahci0 at pci0 dev 13 function 0 "Intel 82801HBM AHCI" rev 0x02: apic 1 int 21,
AHCI 1.1
ahci0: device on port 0 didn't come ready, TFD: 0x171<ERR>
ahci0: port 0: 3.0Gb/s
ahci0: device on port 1 didn't come ready, TFD: 0x171<ERR>
ahci0: port 1: 3.0Gb/s
ahci0: device on port 2 didn't come ready, TFD: 0x131<ERR>
ahci0: port 2: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VBbcc99a08-744eefa6_
sd0: 12288MB, 512 bytes/sector, 25165824 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VB9da9db2b-7414c638_
sd1: 20480MB, 512 bytes/sector, 41943040 sectors
cd0 at scsibus1 targ 2 lun 0: <VBOX, CD-ROM, 1.0> ATAPI 5/cdrom removable
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot): using irq 1
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot): using irq 12
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "Apple OHCI root hub" rev 1.00/1.00 addr 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (81831238603d6adb.a) swap on sd0b dump on sd0b
