Hi Filip, thanks for your recent work, great to have more people help
* On 2016-09-20 at 15:02 BST, Filip Chabik wrote:
> The way I understand it, one should pick the version against which package
> will be build. For example, should you want to target 2015Q4 (to provide some
> package for 15.4.1 LTS images), one should pick 15.4.1 pkgbuild and should you
> target 2016Q2 (or trunk) you should grab latest one available (at current
> it would be 16.2.0).
Correct. In theory a newer pkgbuild image should be able to build any
of the previous releases too, but first the bootstrap kits need to be
populated from https://pkgsrc.joyent.com/packages/SmartOS/bootstrap-pbulk/
What I'll probably do from pkgbuild-16.3.0 onwards is just populate
them all by default (or at least the most recent + LTS) to make things
> There are few places that cover preparation of the environment on the
> pkgbuild -
> some are on Joyent's images documentation, some are on GitHub, some are on
> Joyent's pkgsrc documentation and some bits are available only on some Gist.
> * https://github.com/joyent/pkgbuild/
> * https://pkgsrc.joyent.com/
> * https://gist.github.com/drscream/c45419950d8af648e2c6
> The last is there as I want to sign packages I'm building and it's the only
> place that explains how to add your key to be recognized by SmartOS and to
> me to install self-built stuff. (It doesn't work BTW., but I will get back to
Yeh, so firstly apologies for the scattered documentation. I'm
working on a centralised site for all of this in the background as and
when I get time. Hopefully once the 2016Q3 release is done I can
return to this.
> I tried to combine all of these sources and still failed to get it right in my
> first two attempts (even though Jonathan picked the PR I made on GitHub, he
> to introduce quite a few changes in order to push them further upstream --
> you BTW. also for the awesome help in the IRC channel). For one thing -- I
> always failed to properly add my public key to the pkgsrc keyring, like so:
> gpg --primary-keyring /opt/local/etc/gnupg/pkgsrc.gpg --import
> Command itself is not complaining, but pkg_install is still complaining when
> trying to install package built and signed with my GPG key :/
Things to verify with this are that the package is actually signed (it
will be an ar(1) archive rather than a compressed tarball, and inside
it will contain the key used for signing for you to verify against).
> Secondly, I never used to run-sandbox, which is necessary for proper chroot
> build environment (or at least that's how I understand it) -- simply because I
> missed it on pkgsrc.joyent.com and docs.joyent.com (noticed it later on on
Yeh, run-sandbox is quite critical as it not only gives you a clean
sandbox in which to build, it also sets up the build environment and
without it a bunch of things won't be set correctly.
> OK, enough. I would now like to clarify what is step-by-step to get this whole
> environment right -- with proper GPG signing, proper sandboxing etc. etc.
> 1. Spawn new zone based on the pkgbuild image (for example
> 4183fce6-49b2-11e6-a1ca-4f007e77f9d5 for 16.2.0).
> a) First question: which user should I use to build packages? There's
> admin user with 'sudo' superpowers, there's also pbulk user described as
> Package Builder (but, by default, the dude has no home [directory]), but
> code checkout under /data is owned by root... So, admin? pbulk? root?
For development I tend to just use root. You can use 'pbulk' without
a shell (just use 'su pbulk') but you will first need to install any
required dependencies as root before switching to pbulk for the build.
Unfortunately the 'depends' target will create an initial work
directory which you then won't be able to write to as pbulk, so the
full steps required to build a package as non-root are something like:
$ cd pkgsrc/foo/bar
$ bmake depends
$ chown -R pbulk /home/pbulk/build
$ su pbulk
This is why I just use root for working on packages. The official
packages we build and anything produced by our infrastructure always
correctly uses non-root however.
> 2. The way I understand it, ordinary fellow like me should not push stuff
> directly to joyent-pkgsrc. Preferred way here is definitely to fork the
> on GH, provide keys to clone it onto your pkgbuild zone to the user that
> be handling the builds (look it up above in point 1a) in place of the
> existing /data/pkgsrc. If you plan to maintain this repo for some time it
> might not be a terrible idea to add joyent-pkgsrc as a upstream source to
> sync it every now and then.
Yeh, we accept pull requests, patches, whatever, as long as we get the
contribution we don't mind. While we don't allow people to push
directly to our repository, there is nothing stopping people getting
involved with the main pkgsrc project and committing code directly
upstream (and this is certainly encouraged!)
> 3. User is picked, keys are in place, code has been forked & cloned. There's
> gcc installed by default, so I will now follow instructions for Building
> Packages from pkgsrc.joyent.com.
> a) pkgin -y in gcc49 gnupg2
There should be no need to do this, the run-sandbox script will set up
everything required inside the chroot, and it's advisable not to
install anything outside of it except gnupg2 for the agent.
> In theory exiting the sandbox should clean everything up (or are there any
> manual steps necessary like bmake clean clean-depends for example?).
There's no need to clean manually before exiting the sandbox. All
build artefacts are held under /home/pbulk/build and that is destroyed
> I'm really hoping to clarify couple of the things I mentioned above to have
> better overview what is the preferred way of providing the packages against
The PRs you've sent so far have been great, the only thing missing was
that PKG_DEVELOPER=yes wasn't set due to not running inside
run-sandbox. With that you'd have seen the same errors as me and
would have been able to fix them.
> Is there any linting mechanism I should also use?
There is pkglint installed by default in the sandbox, it will catch
most static errors. The rest are caught by PKG_DEVELOPER=yes with a
bunch of checks being ran on the built package.
> With best wishes
Jonathan Perkin - Joyent, Inc. - www.joyent.com
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription:
Powered by Listbox: http://www.listbox.com