Hi, Over the past two weeks, I spent quite a bit of time looking into using etcupdate(8) to bootstrap /var/db/etcupdate for use with pkg(8), and here are some of the issues I have run into. The most current diff tested is attached.
1) A new 'create-packages-files' target was added, which invokes the etcupdate(8) 'extract' command using WSTAGEDIR for the DESTDIR. Despite passing 'TAGS=package=etc' and 'PACKAGE=etc' to etcupdate(8) '-M' flag, in addition to '-DNO_ROOT', some files still fail to be tagged in WSTAGEDIR/var/db/etcupdate/current/METALOG. To work around this, a hack was added to release/scripts/mtree-to-plist.awk to trap and tag files when '-v etcupdate=yes' is passed to awk(1) in this target. 2) Some files do not automatically get created by etcupdate(8), as noted in the manual page. For example, pwd_mkdb(8), cap_mkdb(1), newaliases(1), tzsetup(8), and services_mkdb(8) are invoked after merging files with the newly-created tree. This causes, for example, files like /etc/passwd to be nonexistent when creating a package. To work around this, another hack was added to the mtree-to-plist.awk script to skip nonexistent files in WSTAGEDIR/var/db/etcupdate. This is still non-ideal, I think. 3) Setting 'prefix = /var/db/etcupdate' in the release/packages/etc.ucl file appears to be ignored. Files in the 'etc' package conflict with files installed by other packages, such as runtime, autofs, and so on, with conflicting files being things like /etc/autofs/include_ldap or other similarly-installed files within /etc. To work around this when packaging files, a hack was added to the mtree-to-plist.awk script (again) to prepend '/var/db/etcupdate' to the file path in order to get the correct files included in the package. (This is somewhat in parallel with (2) above.) 4) When running etcupdate(8) after hacking around some of the issues above, I discovered that /var/db/etcupdate/current in the root of the system in which it is being run disappears. Additionally, I had to restore files within /etc (such as master.passwd(5), passwd(5), etc.) in order to make my test system usable again. In addition to replacing files that should have been merged, 'pkg check' against the 'FreeBSD-etc' package reports all of the files from the 'current' directory have vanished. In addition to the above items, I also did some (albeit minimal) testing with mergemaster(8), as it creates /var/db/mergemaster.mtree, however in my brief testing it seems the mtree(8) file created is not compatible with pkg(8) or the mtree-to-plist.awk script. (Note: the mergemaster(8) bits from my testing are not included in the diff, as I had to revert a number of changes to ensure the correct diff was included in this email. I am at the point now where I am out of ideas on how to proceed, and could use some help on figuring this out. Any help on the above issues, or anything in the Wiki would be greatly appreciated. https://wiki.freebsd.org/PkgBase Thanks, Glen
Index: Makefile
===================================================================
--- Makefile (revision 324062)
+++ Makefile (working copy)
@@ -132,6 +132,7 @@
builddtb xdev xdev-build xdev-install \
xdev-links native-xtools stageworld stagekernel stage-packages \
create-packages-world create-packages-kernel create-packages \
+ create-files-packages \
packages installconfig real-packages sign-packages package-pkg \
print-dir test-system-compiler
Index: Makefile.inc1
===================================================================
--- Makefile.inc1 (revision 324063)
+++ Makefile.inc1 (working copy)
@@ -1555,6 +1555,7 @@
.ORDER: stage-packages create-packages
.ORDER: create-packages create-world-packages
+.ORDER: create-packages create-files-packages
.ORDER: create-packages create-kernel-packages
.ORDER: create-packages sign-packages
@@ -1596,6 +1597,12 @@
DESTDIR=${WSTAGEDIR} \
PKG_VERSION=${PKG_VERSION} create-world-packages
+create-packages-files: _pkgbootstrap _repodir .PHONY
+ ${_+_}@cd ${.CURDIR}; \
+ ${MAKE} -f Makefile.inc1 \
+ DESTDIR=${WSTAGEDIR} \
+ PKG_VERSION=${PKG_VERSION} create-files-packages
+
create-packages-kernel: _pkgbootstrap _repodir .PHONY
${_+_}@cd ${.CURDIR}; \
${MAKE} -f Makefile.inc1 \
@@ -1603,8 +1610,25 @@
PKG_VERSION=${PKG_VERSION} DISTDIR=kernel \
create-kernel-packages
-create-packages: .PHONY create-packages-world create-packages-kernel
+create-packages: .PHONY create-packages-world create-packages-files
create-packages-kernel
+create-files-packages: _pkgbootstrap .PHONY
+ mkdir -p ${WSTAGEDIR}/var/db/etcupdate/current
+ cd ${WSTAGEDIR} ; \
+ ${WSTAGEDIR}/usr/sbin/etcupdate extract -B -M \
+ "TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} -DNO_ROOT
TAGS=package=etc PACKAGE=etc" \
+ -s ${SRCDIR} -d ${WSTAGEDIR}/var/db/etcupdate
+ awk -f ${SRCDIR}/release/scripts/mtree-to-plist.awk \
+ -v etcupdate=yes -v ws=${WSTAGEDIR} \
+ ${WSTAGEDIR}/var/db/etcupdate/current/METALOG
+ sh ${SRCDIR}/release/packages/generate-ucl.sh -o etc \
+ -s ${SRCDIR} -u ${WSTAGEDIR}/etc.ucl
+ ${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh -o ALLOW_BASE_SHLIBS=yes \
+ create -M ${WSTAGEDIR}/etc.ucl \
+ -p ${WSTAGEDIR}/etc.plist \
+ -r ${WSTAGEDIR} \
+ -o ${REPODIR}/$$(${PKG_CMD} -o ABI_FILE=${WSTAGEDIR}/bin/sh
config ABI)/${PKG_VERSION}
+
create-world-packages: _pkgbootstrap .PHONY
@rm -f ${WSTAGEDIR}/*.plist 2>/dev/null || :
@cd ${WSTAGEDIR} ; \
Index: release/packages/etc.ucl
===================================================================
--- release/packages/etc.ucl (nonexistent)
+++ release/packages/etc.ucl (working copy)
@@ -0,0 +1,22 @@
+#
+# $FreeBSD: head/release/packages/runtime.ucl 322327 2017-08-09 19:16:54Z gjb $
+#
+
+name = "FreeBSD-etc"
+origin = "base"
+version = "%VERSION%"
+comment = "etc package"
+categories = [ base ]
+maintainer = "[email protected]"
+www = "https://www.FreeBSD.org";
+prefix = "/var/db/etcupdate"
+licenselogic = "single"
+licenses = [ BSD2CLAUSE ]
+desc = <<EOD
+etc package
+EOD
+scripts: {
+ post-install = <<EOD
+ echo "Please use etcupdate(8) or mergemaster(8) to update"
+ echo "files under /etc."
+EOD
Index: release/scripts/mtree-to-plist.awk
===================================================================
--- release/scripts/mtree-to-plist.awk (revision 324062)
+++ release/scripts/mtree-to-plist.awk (working copy)
@@ -28,6 +28,13 @@
tags=tags""_kernconf
}
}
+ if (etcupdate != "") {
+ if (length(gsub(/tags=/, "", tags) == 0))
+ tags="package=etc"
+ if ((length(type) != 0) && (type == "dir")) {
+ next
+ }
+ }
if (length(tags) == 0)
next
if (tags ~ /package=/) {
@@ -76,5 +83,17 @@
output=pkgname".plist"
}
+ if (etcupdate != "") {
+ pfx="/var/db/etcupdate/current"
+ filename=ws""pfx""$1
+ if (system("test -e " filename) == 0) {
+ print "@"type"("uname","gname","mode","flags") "
pfx""$1 > output
+ next
+ } else {
+ next
+ }
+ next
+ }
+
print "@"type"("uname","gname","mode","flags") " $1 > output
}
signature.asc
Description: PGP signature
