Hi Filip, thanks for your recent work, great to have more people help out! * On 2016-09-20 at 15:02 BST, Filip Chabik wrote:
> The way I understand it, one should pick the version against which package > will be build. For example, should you want to target 2015Q4 (to provide some > package for 15.4.1 LTS images), one should pick 15.4.1 pkgbuild and should you > target 2016Q2 (or trunk) you should grab latest one available (at current > moment > it would be 16.2.0). Correct. In theory a newer pkgbuild image should be able to build any of the previous releases too, but first the bootstrap kits need to be populated from https://pkgsrc.joyent.com/packages/SmartOS/bootstrap-pbulk/ What I'll probably do from pkgbuild-16.3.0 onwards is just populate them all by default (or at least the most recent + LTS) to make things simpler. > There are few places that cover preparation of the environment on the > pkgbuild - > some are on Joyent's images documentation, some are on GitHub, some are on > Joyent's pkgsrc documentation and some bits are available only on some Gist. > The > list: > > * > https://docs.joyent.com/public-cloud/instances/infrastructure/images/smartos/pkgbuild > * https://github.com/joyent/pkgbuild/ > * https://pkgsrc.joyent.com/ > * https://gist.github.com/drscream/c45419950d8af648e2c6 > > The last is there as I want to sign packages I'm building and it's the only > place that explains how to add your key to be recognized by SmartOS and to > allow > me to install self-built stuff. (It doesn't work BTW., but I will get back to > it > later). Yeh, so firstly apologies for the scattered documentation. I'm working on a centralised site for all of this in the background as and when I get time. Hopefully once the 2016Q3 release is done I can return to this. > I tried to combine all of these sources and still failed to get it right in my > first two attempts (even though Jonathan picked the PR I made on GitHub, he > had > to introduce quite a few changes in order to push them further upstream -- > thank > you BTW. also for the awesome help in the IRC channel). For one thing -- I > always failed to properly add my public key to the pkgsrc keyring, like so: > > gpg --primary-keyring /opt/local/etc/gnupg/pkgsrc.gpg --import > pkgsrc/pkgsrc_pkg_sig.pub > > Command itself is not complaining, but pkg_install is still complaining when > trying to install package built and signed with my GPG key :/ Things to verify with this are that the package is actually signed (it will be an ar(1) archive rather than a compressed tarball, and inside it will contain the key used for signing for you to verify against). > Secondly, I never used to run-sandbox, which is necessary for proper chroot > build environment (or at least that's how I understand it) -- simply because I > missed it on pkgsrc.joyent.com and docs.joyent.com (noticed it later on on > GitHub). Yeh, run-sandbox is quite critical as it not only gives you a clean sandbox in which to build, it also sets up the build environment and without it a bunch of things won't be set correctly. > OK, enough. I would now like to clarify what is step-by-step to get this whole > environment right -- with proper GPG signing, proper sandboxing etc. etc. > > 1. Spawn new zone based on the pkgbuild image (for example > 4183fce6-49b2-11e6-a1ca-4f007e77f9d5 for 16.2.0). > a) First question: which user should I use to build packages? There's > default > admin user with 'sudo' superpowers, there's also pbulk user described as > Package Builder (but, by default, the dude has no home [directory]), but > the > code checkout under /data is owned by root... So, admin? pbulk? root? For development I tend to just use root. You can use 'pbulk' without a shell (just use 'su pbulk') but you will first need to install any required dependencies as root before switching to pbulk for the build. Unfortunately the 'depends' target will create an initial work directory which you then won't be able to write to as pbulk, so the full steps required to build a package as non-root are something like: $ cd pkgsrc/foo/bar $ bmake depends $ chown -R pbulk /home/pbulk/build $ su pbulk $ bmake This is why I just use root for working on packages. The official packages we build and anything produced by our infrastructure always correctly uses non-root however. > 2. The way I understand it, ordinary fellow like me should not push stuff > directly to joyent-pkgsrc. Preferred way here is definitely to fork the > repo > on GH, provide keys to clone it onto your pkgbuild zone to the user that > will > be handling the builds (look it up above in point 1a) in place of the > existing /data/pkgsrc. If you plan to maintain this repo for some time it > might not be a terrible idea to add joyent-pkgsrc as a upstream source to > sync it every now and then. Yeh, we accept pull requests, patches, whatever, as long as we get the contribution we don't mind. While we don't allow people to push directly to our repository, there is nothing stopping people getting involved with the main pkgsrc project and committing code directly upstream (and this is certainly encouraged!) > 3. User is picked, keys are in place, code has been forked & cloned. There's > no > gcc installed by default, so I will now follow instructions for Building > Packages from pkgsrc.joyent.com. > a) pkgin -y in gcc49 gnupg2 There should be no need to do this, the run-sandbox script will set up everything required inside the chroot, and it's advisable not to install anything outside of it except gnupg2 for the agent. > In theory exiting the sandbox should clean everything up (or are there any > manual steps necessary like bmake clean clean-depends for example?). There's no need to clean manually before exiting the sandbox. All build artefacts are held under /home/pbulk/build and that is destroyed on logout. > I'm really hoping to clarify couple of the things I mentioned above to have > better overview what is the preferred way of providing the packages against > joyent-pkgsrc. The PRs you've sent so far have been great, the only thing missing was that PKG_DEVELOPER=yes wasn't set due to not running inside run-sandbox. With that you'd have seen the same errors as me and would have been able to fix them. > Is there any linting mechanism I should also use? There is pkglint installed by default in the sandbox, it will catch most static errors. The rest are caught by PKG_DEVELOPER=yes with a bunch of checks being ran on the built package. > With best wishes Thank you! -- Jonathan Perkin - Joyent, Inc. - www.joyent.com ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
