Re: CURRENT, usr/src on git, howto "mergemaster"?

[Matthew Seaman]

On 04/01/2021 12:29, David Wolfskill wrote:


Caveat: Since the switch, I have yet to encounter a case where I needed
to merge a change in (e.g., because of a newly-created user, or there
was a commit to /etc/crontab or /etc/newsyslog.conf).  I may find things
rather "more interesting" when that happens; we shall see.


The process of merging changes in etcupdate(1) is essentially identical 
to merging in mergemaster(1) -- the difference being that typically 
etcupdate(1) will run to completion without any user intervention 
needed, or else it will flag up that there are unresolved differences to 
merge and flag to the user to run `etcupdate resolve` as a separate command.


This is much more time efficient than the typical mergemaster(1) 
procedure, and I find it lends itself much more effectively to 
automation through eg. ansible.  (ie. you can just run the first 
`etcupdate` through automation across all of your server inventory, and 
then go round and manually resolve anything that needs it.)


Cheers,

Matthew

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: what 3rd party boot mgr is required to boot multiple freebsd versions?

[Matthew Seaman]

On 17/03/2020 12:58, Florian Limberger wrote:
> On 16.03.20 23:33, Chris wrote:
> 
>> For the record. I'm *only* using FreeBSD in this situation. I
>> only mentioned Windows above, for the use of it's boot manager.
> 
> If you only use FreeBSD, and also use ZFS, you might find beadm[1]
> interesting.
> 
> [1]: https://www.freshports.org/sysutils/beadm
> 

Did you know that the system now comes with bectl(8) which is very
similar to beadm?  As far as I can tell, the biggest difference is that
if you have more than one ZFS in your boot environment then:

beadm create FOO

is actually equivalent to

bectl create -r FOO

ie. turning on the recursive functionality in bectl.

However, this is not really what the OP was asking about.  As I
understand it, they were looking for something that would allow choosing
between several independent installations of different versions of
FreeBSD, rather than having multiple environments in the same
installation.  You can achieve pretty much the same effect though --
there is a boot menu option to switch between BEs.  You would have to
manage any ported software between the different OS versions, perhaps by
including /usr/local and /var/db/pkg are both parts of your BEs.

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: Pkg repository is broken...

[Matthew Seaman]

On 07/03/2020 22:38, Greg 'groggy' Lehey wrote:
>> This was only an issue on the "latest" branch. If you don't alter
>> "/etc/pkg/FreeBSD.conf", you'll get packages from the "quarterly"
>> branch, which fortunately wasn't affected.

> No, this isn't necessarily correct.  I have never modified this file,
> but I ended up with a copy of /usr/src/usr.bin/pkg/FreeBSD.conf.latest
> with this revision string:
> 
>   # $FreeBSD: stable/11/etc/pkg/FreeBSD.conf 263937 2014-03-30 15:24:17Z 
> bdrewery $
> 
> Despite the age, this appears to identical to the current version,
> according to svn blame.  Arguably this should be the default anyway.
> 

Best practice is *not* to modify /etc/pkg/FreeBSD.conf, but to create a
supplemental file /usr/local/etc/pkg/repos/FreeBSD.conf to override any
of the default settings.

So, for example, this in /usr/local/etc/pkg/repos/FreeBSD.conf will
switch to the 'latest' package set:

```
FreeBSD: { url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest; }
```

and this will disable the FreeBSD pkg repos entirely:

```
FreeBSD: { enabled: no }
```

(presumably with an additional .../pkg/repos/foo.conf file to enable a
custom local repository)

The best way to confirm exactly what the resultant pkg(8) configuration
comes out as is by:

   pkg -vv

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: pkg: Cannot open /dev/null:No such file or directory

[Matthew Seaman]

On 04/06/2019 06:32, O. Hartmann wrote:
> As far as I know,, the package installation is performed via "chroot'ed"
> environment and somehow /dev/null is out of a sudden not accessible anymore
> while pkg tries to delegate some output to /dev/null.

Assuming you're chroot'd to /chroot, then:

mount -t devfs devfs /chroot/dev

should allow pkg(8) to work as usual.

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: New vm-image size is much smaller than previos

[Matthew Seaman]

On 03/05/2019 17:10, David Boyd wrote:

The vm-image for 13.0-CURRENT

 FreeBSD-13.0-CURRENT-amd64-20190503-r347033.vmdk

is only 4.0 GB in size.  Previous images were about 31.0 GB.

This smaller image doesn't leave much room to add packages and other
customizations.



Yes, the VM images are smaller, and deliberately so.  The idea is that 
the image is basically shrunk to the minimum size -- just big enough to 
hold a standard install.  Then when you deploy one of these images, you 
allocate a virtual drive of whatever size you need, and growfs(8) your 
filesystems (if using UFS) or allow your vdevs to expand to fill the 
space available (if using ZFS).  This allows you to build a system of 
pretty much any feasible size and parallels the behaviour of eg. the 
AMIs available for AWS.


Cheers,

Matthew
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Sharing compiled builds between multiple 12-CURRENT boxes.

[Matthew Seaman]

On 19/08/2018 01:55, Shane Ambler wrote:
>> I run 12-CURRENT on few machines, some more powerful that other (all
>> of them x86_64, march varies). 

> You can use freebsd-update by setting up your own update server
> https://www.freebsd.org/doc/en/articles/freebsd-update-server

freebsd-upgrade(8) only works for releases, and not 12-CURRENT as the OP
specified.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Base FreeBSD-lld package dependency problems...

[Matthew Seaman]

Here's a little glitch with packaged base I stumbled upon.

On a freshly pkg upgrade'd 12-current machine, I see:

codling:/usr/home/matthew:# pkg info -dr FreeBSD-lld-12.0.s20180609030436
FreeBSD-lld-12.0.s20180609030436
Depends on :
FreeBSD-runtime-12.0.s20180609030436

So, no dependencies except the runtime.

But look at this:

codling:/usr/home/matthew:# pkg autoremove
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 2 packages:

Installed packages to be REMOVED:
FreeBSD-libefivar-12.0.s20180609030436
FreeBSD-libregex-12.0.s20180609030436

Number of packages to be removed: 2

Proceed with deinstalling packages? [y/N]: y
[1/2] Deinstalling FreeBSD-libefivar-12.0.s20180609030436...
[1/2] Deleting files for FreeBSD-libefivar-12.0.s20180609030436: 100%
[2/2] Deinstalling FreeBSD-libregex-12.0.s20180609030436...
[2/2] Deleting files for FreeBSD-libregex-12.0.s20180609030436: 100%


codling:/usr/home/matthew:# pkg install -f FreeBSD-lld
Updating base repository catalogue...
base repository is up to date.
Updating ports repository catalogue...
ports repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 3 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
FreeBSD-libefivar: 12.0.s20180609030436 [base]
FreeBSD-libregex: 12.0.s20180609030436 [base]

Installed packages to be REINSTALLED:
FreeBSD-lld-12.0.s20180609030436 [base]

Number of packages to be installed: 2
Number of packages to be reinstalled: 1

Proceed with this action? [y/N]: y
[1/3] Reinstalling FreeBSD-lld-12.0.s20180609030436...
[1/3] Extracting FreeBSD-lld-12.0.s20180609030436: 100%
[2/3] Installing FreeBSD-libefivar-12.0.s20180609030436...
[2/3] Extracting FreeBSD-libefivar-12.0.s20180609030436: 100%
[3/3] Installing FreeBSD-libregex-12.0.s20180609030436...
[3/3] Extracting FreeBSD-libregex-12.0.s20180609030436: 100%

Looks like FreeBSD-lld might need to register dependencies on
FreeBSD-libefivar and FreeBSD-libregex?  Or perhaps its some other
package that needs libefivar and libregex?

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: Newbie q [repost]

[Matthew Seaman]

On 01/10/2017 13:22, Jeffrey Bouquet wrote:
> I've a uname-a : STABLE-11 that svn's to 12-HEAD  and will not buildworld 
> despite
> 'make cleanworld ' and '/bin/rm -rf /usr/obj/usr'  both...  and src.conf and 
> make.conf removed from /etc. 
> ...
> libmap.conf problem?
> unpack base.txz etc how?
> pkg of base ready somewhat and a how to ?
> wait out v12 and v13 is HEAD ?
> remove disk, new install and copy over .rc [etc] from old system?
> ..
> No urgency but has been ongoing for over a year maybe even over two years.
> ..

So, is your question "how can I fix the build problems" or "how can I
upgrade to 12-CURRENT _without_ having to build world"?

If the former, then you'll have to give us at least something to go on.
Saying "it doesn't work" may be completely factual, but it doesn't help
at all in working out why not.

If the latter then there are current snapshots available from eg.

http://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/12.0-CURRENT/

or as installer images from

http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/12.0/

I'd advise going for a new 12.0 install and porting over any
configuration files etc.  -- ideally leaving your existing 11-STABLE
system still available so you can boot back into that in case of
difficulties.

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: Critique this plan?

[Matthew Seaman]

On 2017/07/20 11:22, Jeffrey Bouquet wrote:
>   It seems I've a brick wall [too many ports to use pkg effectively ] -- [ 
> 3551 ] 
> and too many need ' pkg lock ' in ' v11 ' for ino64 fixes, 12.0-CURRENT, and 
> quite a few
> others fail to build from ports, either compiler, so are also 'pkg lock ' or 
> in a few
> instances binaries/trees copied from other installs, so that my DESKTOP can 
> continue
> running a if it were 2003 Microsoft based, vs 2004 Freebsd January based, 
> where a reinstall
> seems in order OR, I should just sit and wait til 13.0-CURRENT and proceed 
> that way.

You're proposing to make a backup of your system in spare space on your
hard drive, and then install a pristine system and backport your various
changes to it in order to bring your system up to date?

Waiting for 13.0-CURRENT probably won't solve all your compilation and
package management problems, or at least, not all of them.  You'ld be
better off updating now, but trying to clean up all your local changes
as far as possible so that future upgrades are less traumatic.

> ..
>   Meantime, how is the following as a workaround
>   mv /usr/src /src-2017
>   mv /usr/obj /obj-2017
>   mkdir -p /usr/src
>   mkdir -p /usr/obj
>   cd /usr/src
>   bw, etc
> 
> or
> .
>  [ clean install ]
>  mount -t ufs /dev/gpt/2004root /mnt-root
>  mount -t ufs /dev/gpt/2004var  /mnt-var
>  mount -t ufs /dev/gpt/2004tmp  /mnt-tmp
>   mount -t ufs /dev/gpt/2004usr /mnt-usr
>  into which I surmise an installworld would fail as multiple DESTDIRS are 
> included. 

You can do:

mount -t ufs /dev/gpt/2004root /mnt
mount -t ufs /dev/gpt/2004var  /mnt/var
mount -t ufs /dev/gpt/2004tmp  /mnt/tmp
mount -t ufs /dev/gpt/2004usr  /mnt/usr

so your copy of your 2004 system is laid out below /mnt as it would be
when live.  If you also do:

mount -t devfs devfs /mnt/dev

then you can chroot into /mnt, although I'm not sure quite how useful
that would be to you.

> .
>   nullfs ?
> ...
>  Revert to all-in-one / system, no /var /tmp /usr?
> .
>  or some new install 
> .
>   None of these are plans as of yet, save proceeding without any upgrade 
> whatsoever.  I recall
> unpacking base.txz [etc] to fix a failing installworld in the recent past, so 
> any foolproof
> method of that would also be welcome.  But I suspect much would remain 
> undone, 
> initial *proper* setup of /etc/mail, /etc/groups, as well as the loss of 
> fine-tunings I've
> done over the past 13 years or so, if it were done preplanned as a 
> new-then-rsync-the-old
> system-over-it sort of reinstall  [ not looking forward to undoing years of 
> week-by-week
> this-rc  that-rc fixups...  newbie in so many areas who just copy-pasted 
> the
> work of others into this system, to excellent, usually effect.] 
> .. 

Yeah.  You've a lot of work ahead reviewing each of your changes and
porting what you need to the new version.

As a matter of routine system maintenance, it is good practice to try
and revert local changes and track updates to the default system when
possible -- ie. to adopt any upstream fixes as they become available.

>   Apologies for the email that went on three times longer [ more verbose  ] 
> than I planned, sort
> of making its Subject:a  misstatement of the body of the email.
> ..
> ...

If you're planning on working from a new install, my advice would be to
summarize what functionality you want from your system as a series of
bullet points and then only port whichever of your changes you need in a
directed fashion to achieve each of those points.  Do this as cleanly as
possible, so you can achieve your required functionality with the
minimum amount of configuration work / local patches.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: Ports not building due to release not supported

[Matthew Seaman]

On 13/04/2017 21:50, mmu...@xtra.co.nz wrote:
> I am unable to build ports as I am getting the error message –
> Ports Collection support for your FreeBSD version has ended, and no ports are
> guaranteed to build on this system. Please upgrade to a supported release.
> 
> But my version is 
> FreeBSD rpi2 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r293851: Fri Jan 22 
> 22:09:32 NZDT 2016 
> mark@raspberryPI:/usr/home/mark/crochet-master/work/obj/arm.armv6/usr/src/sys/RPI2
>   arm

Yes, 11.0-CURRENT is no longer supported.  11.0-RELEASE is what is now
supported.

Your freebsd-update command is failing because the freebsd update
servers don't understand '11.0-CURRENT' as a known release -- this was a
development version before 11.0-RELEASE came out.  Your best choices are
either to grab some install media from the FTP sites and install a new
11.0-RELEASE system, or to use svn or similar to grab up-to-date sources
and build yourself a newer system.  Almost certainly the former given
you have a rpi2.

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: Log spam: Limiting * response from 1 to 200 packets/sec

[Matthew Seaman]

On 2016/12/13 15:43, Michael Butler wrote:
> On 12/13/16 10:29, Dimitry Andric wrote:
> 
>> Somebody is most likely port scanning your machines.  I see this all the
>> time on boxes connected to the internet.
> 
> As are mine. I wouldn't mind so much if the message contained sufficient
> useful information that could be acted on, e.g. originating IP address
> and, when appropriate, destination port.

If you want that sort of information, you can use pf(4) with a default
rule to log and reject connections to your system. (Plus rules to permit
traffic to legitimate services, obviously.)  You can also just 'drop'
the denied connections rather than the default response of sending back
an ICMP unreachable or reset response, which will save you sending out a
lot of itty-bitty packets that the port scanners wouldn't pay attention
to anyhow.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: CURRENT: pkg broken on r309320

[Matthew Seaman]

On 2016/11/30 12:14, Baptiste Daroussin wrote:
> On Wed, Nov 30, 2016 at 12:10:31PM +0000, Matthew Seaman wrote:
>> On 2016/11/30 10:14, Baptiste Daroussin wrote:
>>> revert r309314 the issue is there:
>>>
>>> The ports tree defines PKG_CMD as pkg register but now bsd.own.mk 
>>> predefines it
>>> to simply pkg.
>>>
>>> There is a collision there.
>>>
>>
>> Pointy hat to me.  I'll revert that commit.
>>
>>  Matthew
>>
> 
> I have proposed https://reviews.freebsd.org/D8677 instead which make sense and
> does not require your to modify bsd.own.mk

OK, I'll wait and see what the outcome of D8677 is.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: CURRENT: pkg broken on r309320

[Matthew Seaman]

On 2016/11/30 10:14, Baptiste Daroussin wrote:
> revert r309314 the issue is there:
> 
> The ports tree defines PKG_CMD as pkg register but now bsd.own.mk predefines 
> it
> to simply pkg.
> 
> There is a collision there.
> 

Pointy hat to me.  I'll revert that commit.

Matthew




signature.asc
Description: OpenPGP digital signature

Re: [FreeBSD-Announce] HEADS-UP: OpenSSH DSA keys are deprecated in 12.0 and 11.0

[Matthew Seaman]

On 09/08/2016 03:23, Jeffrey Bouquet wrote:
> Will/could there be some kind of UPDATING announcement re which files
> explicitly to switch out/remove/replace/checkfor etc the deprecated
> lines and precisely the steps to replace with new or some other
> suitable action? Action required for both the sshd and client?
> Subdirectories involved? etc...  Unclear here, but I don't use SSH
> hardly yet... despite having bought the book.

As far as managing sshd on your own systems, you should not need to make
massive changes to the /etc/ssh/sshd_config when upgrading to 11.0 or
12.0 -- the normal mergemaster or etcmerge procedures will probably
cover things.  On an upgraded system, you will have still have
/etc/ssh/ssh_host_dsa_key{,.pub} but these will be ignored by sshd and
would not be generated on a new machine.

Optionally, you may choose to replace /etc/ssh/ssh_host_rsa_key{,.pub}
if that key has a short bit-length.

You may find that you get 'Key mismatch' warnings -- ssh may use a
different type of host key on connection to a machine after this update,
and it will alert you if this does not match what it has in
~/ssh/known-hosts from previous connections.  If you're satisfied that
the warning is explained by this configuration change, then you can edit
known-hosts to eliminate the warning message.

As a ssh user, you will need to review the ssh keys you are using, and
what is listed in the ~/.ssh/authorized_keys files of any machines you
want to login to.  You can add a new key of and alternate type in
parallel to your existing keys, and load multiple keys into ssh-agent --
this allows you to phase in a new key with minimal risk that you will
lock yourself out of a remote machine.  Doing this *before* you upgrade
any systems is just common sense.

The default configuration of sshd provided with FreeBSD provides good
security and a good level of interoperability with other ssh
implementations, and you can use it with confidence.  Depending on local
requirements you may want to impose a stricter policy.  In that case,
the following references will be interesting to you:

https://wiki.mozilla.org/Security/Guidelines/OpenSSH
https://stribika.github.io/2015/01/04/secure-secure-shell.html

These are, however, rather more than most people will really find necessary.

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: HEADS-UP: OpenSSH DSA keys are deprecated in 12.0 and 11.0

[Matthew Seaman]

On 08/05/16 03:09, Glen Barber wrote:
> On Fri, Aug 05, 2016 at 01:59:18AM +, Glen Barber wrote:
>> This is a heads-up that OpenSSH keys are deprecated upstream by OpenSSH,
>> and will be deprecated effective 11.0-RELEASE (and preceeding RCs).
>>
> 
> Stupid editor mistake.  OpenSSH DSA keys are deprecated upstream.  Sorry
> for any confusion.
> 
>> Please see r303716 for details on the relevant commit, but upstream no
>> longer considers them secure.  Please replace DSA keys with ECDSA or RSA

I believe ED25519 keys are also a preferred type.

>> keys as soon as possible, otherwise there will be issues when upgrading
>> from 11.0-BETA4 to the subsequent 11.0 build, but most definitely the
>> 11.0-RELEASE build.
>>
> 
> Glen
> On behalf of: re@ and secteam@
> 

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: Oversight in /etc/defaults/rc.conf

[Matthew Seaman]

On 07/12/16 13:27, Glen Barber wrote:
> On Tue, Jul 12, 2016 at 07:17:19AM +0100, Matthew Seaman wrote:
>> I just upgraded my main machine to 11-STABLE.  Things are mostly working
>> fine -- however I did notice that the new iovctl rc script is apparently
>> enabled by default.  That seems like a trivial omission:
>>
>> Index: etc/defaults/rc.conf
>> ===
>> --- etc/defaults/rc.conf (revision 302482)
>> +++ etc/defaults/rc.conf (working copy)
>> @@ -695,6 +695,7 @@
>>  rctl_enable="YES"   # Load rctl(8) rules on boot
>>  rctl_rules="/etc/rctl.conf" # rctl(8) ruleset. See rctl.conf(5).
>>
>> +iovctl_enable="NO"
>>  iovctl_files="" # Config files for iovctl(8)
>>
>>  ##
>>
> 
> I'm not sure I understand.  Is there a functional and/or performance
> impact with it enabled by default?  (Note, I don't disable it in my
> rc.conf, and there is no /dev/iov/* on my system.)

I'm not religious about it being turned off per se.  More that it should
have a clearly defined on/off state shown in the defaults.

I went for 'off' following the general principle that rc.conf items
should mostly be off by default and require specific action to enable.
Yes, there are exceptions to this rule, but I see no particular reason
that iovctl should be one.  What's the advantage to turning it on by
default on every FreeBSD installation?

However, even if it's felt it should be enabled everywhere, then
shouldn't /etc/defaults/rc.conf have:

iovctl_enable="YES"

instead?

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: GOST in OPENSSL_BASE

[Matthew Seaman]

On 07/12/16 06:48, Kevin Oberman wrote:
> In case people are not aware of it, Russian law now requires ALL encrypted
> traffic must either be accessible by the FSB or that the private keys must
> be available to the FSB. I have always assumed that GOST has a hidden
> vulnerability/backdoor that the FSB is already using, but this makes it
> mandatory. Putin gave the FSB 2 weeks to implement the law, which is
> clearly impossible, but I suspect that there will be a huge effort to pick
> all low-hanging fruit. As a result, I suspect no one outside of Russia will
> touch GOST. (Not that they do now, either.) I'd hate to see its support
> required for any protocol except in Russia as someone will be silly enough
> to use it.

Agreed that it should be possible to use GOST crypto readily on FreeBSD,
but I dislike the idea of shipping with 'known vulnerable' ciphers
enabled by default.  It should take a positive act to enable them, given
the circumstances.  Whether that should entail installing something from
ports, or recompiling the system with specific settings in src.conf or
it could just be down to tweaking a config file somewhere I wouldn't
care to venture an opinion though.

I'm also curious as to how far these regulations are supposed to extend.
 Presumably traffic which is merely transiting Russian territory isn't
covered, at least in a practical sense.  How about people from Russia
accessing foreign websites?  I can't see any of the big Internet players
implementing GOST in any locations outside Russia any time soon.
Neither would I as a non-Russian have GOST capabilities client-side, so
what happens if I go and look at say a YandX website over HTTPS?  Putin
and his advisors aren't stupid, and they'd already have considered all
this; plus, as you say, the timetable is clearly impossible; so there
must be something else going on here.

Of course, now there's fairly good evidence that there's some sort of
backdoor in the GOST ciphers, all bets are off on how long it will be
until they get broken in a very public manner.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Oversight in /etc/defaults/rc.conf

[Matthew Seaman]

I just upgraded my main machine to 11-STABLE.  Things are mostly working
fine -- however I did notice that the new iovctl rc script is apparently
enabled by default.  That seems like a trivial omission:

Index: etc/defaults/rc.conf
===
--- etc/defaults/rc.conf(revision 302482)
+++ etc/defaults/rc.conf(working copy)
@@ -695,6 +695,7 @@
 rctl_enable="YES"  # Load rctl(8) rules on boot
 rctl_rules="/etc/rctl.conf"# rctl(8) ruleset. See rctl.conf(5).

+iovctl_enable="NO"
 iovctl_files=""# Config files for iovctl(8)

 ##

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: [CFT] ypldap testing against OpenLDAP and Microsoft Active Directory

[Matthew Seaman]

On 09/06/2016 18:34, Craig Rodrigues wrote:
> There is still value to ypldap as it is now, and getting feedback from
> users (especially Active Directory) would be very useful.
> If someone could document a configuration which uses IPSEC or OpenSSH
> forwarding, that would be nice.
> 
> In future, maybe someone in OpenBSD or FreeBSD will implement things like
> LDAP over SSL.

What advantages does ypldap offer over nss-pam-ldapd (in ports) ?
nss-pam-ldapd can use both ldap+STARTTLS or ldaps to encrypt data in
transit, and I find it works very well for using OpenLDAP as a central
account database.  I believe it works with AD, but haven't tried that
myself.

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: why 100 packages are evil

[Matthew Seaman]

On 23/04/2016 17:19, Poul-Henning Kamp wrote:
> 
> In message , Lyndon 
> Neren
> berg writes:
> 
>> With freebsd-update, an announcement comes out that says 'update'!.  So we 
>> do.  Move from 10.2-p11 to 10.2-p12.  There is a very clear track record 
>> of why and how this happened.
> 
> Is freebsd-update going away as result of the new packaging ?

Yes.  It will be replaced by 'pkg upgrade' -- as far as I know, that's
the plan for 11.0-RELEASE.  I have no idea if anyone intends to run
maintain freebsd-update in parallel for a transition period, but
freebsd-update will ultimately be superceded.

Cheers,

Matthew






signature.asc
Description: OpenPGP digital signature

Re: [CFT] packaging the base system with pkg(8)

[Matthew Seaman]

On 04/20/16 10:48, Slawa Olhovchenkov wrote:

> While number of packages don't see outside internal -- this is
> irrelevant.
> After possibility of update individual package -- nuber of packages is
> impotant.
> Take fresh 11.0. Before 11.1 update only kernel. What you system have?
> 11.0? 11.1-RC3? How you name it? How identify it for take support on
> forum or mail list?
> 
> How name system, updated all w/o compiler? or only some services?
> Currently we have simple naming:
> 
> 10.3-RC1, 10.3-RELEASE, 10.3-p7, 10.3-STABLE r123456.
> This is shortly and clearly identify system to anyone.
> 
> How do this with many packages?

Hmmm Good point.

At release time, a set of packages will be generated.  These will all
have the version number of the release eg. 11.0.  That's simple and
unambiguous.

Subsequently adding patches will add a patch level to the version, so
11.0-p1, 11.0-p2 exactly as now.  Only patches that affect the kernel
will cause the output of uname(1) to show the new patch level, but
updates to user land should be reflected in the output of
freebsd-version(1).  That's exactly the same as now, and assuming you,
as an end user, install the default set of FreeBSD packages and apply
all the patches as they come out, then there should be no problem.

This implies that /every/ patchset will include an update to the package
containing freebsd-version.

What packaging base does do is allow you to be selective in the patches
you apply.  So, for instance if patch -p1 was not relevant to you, you
might not install it.  So you could end up with a system where you
hadn't installed patches -p1 -- -p9 but you did install patch -p10.  The
freebsd-version(1) output would presumably show the system as 11.0-p10
-- but that's certainly not the same as a system with all of those
patches -p1 to -p9 applied.  Or you could just install the updated
freebsd-version(1) package and have your system blatantly lie about its
patching status.

So, yes, this does change the meaning of the version number string.
It's morally equivalent to tracking the releng/11.0 branch in SVN but
compiling your system with various WITH_FOO or WITHOUT_BAR flags, and
possible local modifications to the code base to back out certain
commits.  It's still 11.0-SOMETHING but it's not clear exactly what that
something should  be.  Hopefully people that do such things will be
sufficiently technically sophisticated to be able to characterize their
problems based on the versions of any relevant FreeBSD packages.

On the release of 11.1 there would be a complete new set of system
packages generated, and the upgrade process would install the new
versions of those packages all round, even if the content of an
individual package was identical to the one in 11.0.  There's probably a
handy optimization where we compare the before and after checksums for
package files and don't overwrite on disk what is identical between
package versions, but do update all of the bookkeeping in the pkgdb.

Cheers,

Matthew








signature.asc
Description: OpenPGP digital signature

Re: [CFT] packaging the base system with pkg(8)

[Matthew Seaman]

On 20/04/2016 06:12, Daniel Eischen wrote:
> [And it really bothers me that FreeBSD 'pkg list' behaves
>  like 'pkg files' or similar should.  It seems intuitive
>  that 'pkg list' should list the packages, not all the files
>  in all the packages.]

'pkg list' is one of the aliases defined in the default pkg.conf -- if
you want to rename it to 'pkg files' that's trivially easy.  If there's
any sort of consensus that 'pkg files' would be preferable in general
then I'm sure that the sample pkg.conf can be fixed in git.

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: zfsboot patch for /usr

[Matthew Seaman]

On 10/03/2016 01:02, Freddie Cash wrote:
> Set mountpoint=none if you just want to create the parent dataset without
> actually using it for storage. Then you can set properties on it, and child
> datasets will inherit then. Like pool/usr/local

Usually, you want the mountpoint to be one of the inherited properties
-- so set canmount=off instead.

This pattern is used by default already in the installer.  The root
filesystem is created as a BE under zroot/ROOT/default/ but there is
also a zroot/var which has canmount=off to act as a parent to
zroot/var/logs, zroot/var/mail etc. which are not part of the boot
environment but are mounted in the expected locations under /var

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: [CFT] packaging the base system with pkg(8)

[Matthew Seaman]

On 03/02/16 23:54, Glen Barber wrote:
> Also note (as repeated below), running 'pkg delete -a' will implicitly
> remove base system packages after they are installed.

This has the potential for many feet to be shot, given that up to now,
'pkg delete -a' would always leave you with a viable system.

We already make an exception for pkg itself -- you need 'pkg delete -fa'
to actually remove pkg(8) as well.  (Note to self: this needs to be
documented in the pkg-delete(8) man page.)

We should have similar exceptions for the essential bits of the base
system -- at minimum everything you need to boot the system up and
install stuff from a package repository.

We should also have a command line that will remove all ported software
but leave the base intact.   Maybe by adding '-r reponame' functionality
to 'pkg delete'?

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: "libssl.so.8" not found

[Matthew Seaman]

On 14/12/2015 07:18, Matthias Apitz wrote:
> I don't know why pkg 1.6.2 was produced with this recent libssl.so.8; it
> should have been done more conservative, IMHO

pkg(8) for HEAD just gets built against whatever is in recent HEAD.
Even though it can lead to problems like this when a shlib ABI version
gets bumped, I can't really see how else you'ld manage package building
for the bleeding edge version of the OS.

You will have similar problems for any port that links against libssl.so
from the base system.  You might be able to install openssl from ports
and make a sym-link to the libssl.so.8 that comes with that.

I believe bapt wants to make the base system libraries private to the
base system as far as possible and default to using versions from the
ports in cases like this, but it hasn't happened yet.  pkg(8) will
always be a special case though -- there's a bit of a chicken and egg
problem that means pkg(8) cannot itself have any external dependencies,
so pkg(8) may well end up importing some crypto code into its sources.

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: pkg does not update the repo catalogue

[Matthew Seaman]

On 12/07/15 08:50, Matthias Apitz wrote:
> 
> Hello,
> 
> This is with 11-CURRENT and ports from July this year; I have the
> packages which I build with poudriere on some other host in a dir
> /usr/PKGDIR.20150726 and added 8 new packages there, the total number is
> now 1691:
> 
> # ls *.txz | egrep -v 'packagesite.txz|meta.txz|digests.txz' | wc -l 
> 1691
> 
> My repo definition is:
> 
> # cat /usr/local/etc/pkg/repos/myrepo.conf
>FreeBSD: {
>url: "file:/usr/PKGDIR.20150726",
>enabled: true,
>}

There's no need to label your custom repo as 'FreeBSD' -- in fact, it's
probably better for you to use a distinct name, as the repo.conf files
accumulate for the same repo tag.  In this case you've possibly
inadvertently got pkg checking the pkg signatures against the default
FreeBSD repository keys, which isn't going to work for locally built
packages.

Just change the tag in the repo.conf to 'myrepo' and then check what
pkg(8) sees overall by running 'pkg -vv'.  You'll need to do a pkg
upgrade -f after that.

If you don't want to use the standard FreeBSD repo at all then you can
add a /usr/local/etc/repos/FreeBSD.conf containing

FreeBSD: { enabled: no }

> When I now want to update the 8 new packages to the repo catalogue, they
> are not added (i.e. the number stays with 1683 and I also can not
> install them with 'pkg instal ...'):
> 
> # pkg -v
> 1.5.5
> # pkg -R /usr/local/etc/pkg/repos/ update -f
> Updating FreeBSD repository catalogue...
> Fetching meta.txz: 100%260 B   0.3kB/s00:01
> Fetching packagesite.txz: 100%  382 KiB 391.6kB/s00:01
> Processing entries: 100%
> FreeBSD repository update completed. 1683 packages processed.
> 
> What I'm missing here?

If changing the repo tag doesn't fix the problem, try turning on some
debugging output:

   env DEBUG_LEVEL=4 pkg update -f

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: pkg does not update the repo catalogue

[Matthew Seaman]

On 12/07/15 11:04, Matthias Apitz wrote:
> Now 'pkg -vv' shows only myrepo; the 'pkg upgrade -f' ended up with
> reinstallation of all ~1000 packages;

Ooops.  'pkg upgrade -f' says 'reinstall everything' -- which was an
error on my part. I meant to type 'pkg update -f' which only refreshes
the repo catalogues.  Apologies.

> but all this did not solved the problem;
> 
>> > If changing the repo tag doesn't fix the problem, try turning on some
>> > debugging output:
>> > 
>> >env DEBUG_LEVEL=4 pkg update -f
> The output of STDERR is here: http://www.unixarea.de/pkg-stderr.txt
> (4 MByte, 100.000 lines)

H  it seems fairly clear to me that the 8 new packages aren't in
the repo catalogue that you're downloading.  You built the repo using
poudriere?  Poudriere does some fun'n'games with symbolic links to
achieve an atomic repo update, which means there is actually a history
of previous versions of the repo kept.  You'll see a directory structure
like this:

% ls -la
total 159
drwxr-xr-x  102 root  wheel  108 Dec  7 00:19 ./
drwxr-xr-x9 root  wheel   15 Aug  4 11:39 ../
lrwxr-xr-x1 root  wheel   16 Dec  7 00:19 .latest@ -> .real_1449447572
[...]
drwxr-xr-x4 root  wheel8 Dec  5 01:27 .real_1449278821/
drwxr-xr-x4 root  wheel8 Dec  6 00:55 .real_1449363354/
drwxr-xr-x4 root  wheel8 Dec  7 00:19 .real_1449447572/
lrwxr-xr-x1 root  wheel   11 Dec 19  2014 All@ -> .latest/All
lrwxr-xr-x1 root  wheel   14 Dec 19  2014 Latest@ -> .latest/Latest
lrwxr-xr-x1 root  wheel   19 Dec 19  2014 digests.txz@ ->
.latest/digests.txz
lrwxr-xr-x1 root  wheel   16 Dec 19  2014 meta.txz@ -> .latest/meta.txz
lrwxr-xr-x1 root  wheel   23 Dec 19  2014 packagesite.txz@ ->
.latest/packagesite.txz

Does the .latest symlink point at the .real_$TIMESTAMP you're expecting?

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: pkg does not update the repo catalogue

[Matthew Seaman]

On 12/07/15 11:42, Matthias Apitz wrote:
> Maybe I wasn't clear enough. I copied only the packages to the new
> system (my netbook) some weeks ago, and created the catalogue with
> 
> # pkg repo /usr/PKGDIR.20150726
> 
> and installed the packages; Today I built 8 packages more, copied them
> over too and was (may be in error) expecting to add these 8 new packages to
> the catalogue with 'pkg update -f'
> 
> Seems that I just have to reuse 'pkg repo '

Yeah.  That would explain what you were seeing.  Yes, you do have to
rebuild the repo catalogue on your repo server when you change the repo
contents.

Running 'pkg update -f' or similar won't cause pkg(8) to be run on the
repo side -- the repo is a read-only source of static content[*] as far
as the client pkg(8) can interact with it.

Cheers,

Matthew

[*] Well, except possibly for the repo.conf HTTP mirroring scheme which
is intended to allow load balancing amongst a set of repo servers by --
possibly dynamically -- generating a list of the available servers in
preference order.




signature.asc
Description: OpenPGP digital signature

Re: Segmentation fault running ntpd

[Matthew Seaman]

On 10/30/15 09:32, Franco Fichtner wrote:
> Well, it’s on stable/10 since September 16 and somebody reported that
> this particular branch would not trigger the crash along with HEAD,
> but any 10.x would.  Can’t find the reference right now though.

That was me, amongst others.  There are threads on security@ and questions@.

>> On 30 Oct 2015, at 10:24 am, NGie Cooper  wrote:
>>
>>
>>> On Oct 30, 2015, at 02:18, Franco Fichtner  wrote:
>>>
>>> Hi all,
>>>
>>> I did a quick test build and this seems to solve the ntpd crash issue
>>> on top of releng/10.1.
>>
>> Makes sense … looking through my email r287591 was never MFCed back to 
>> stable/9 or stable/10 :/ .
>> HTH,
>> -NGie

There were two problems reported:

1) ntpdc and ntpq would crash -- this was reported for 9.3-STABLE -- I
don't think it affected other releases, and was diagnosed as due to a
pthreads linking issue.  Solved for 9.x in r290044 and r290046

2) ntpd SEGV's on startup on 10.2-RELEASE-p6 (possibly others).
Curiously, so does net/ntp from ports, but only on the second startup.
Exactly the same ntp package seems to run and restart just fine on
recent 10-STABLE though.  As does the base system ntpd.

Cheers,

Matthew







signature.asc
Description: OpenPGP digital signature

Re: r286615: /usr/libexec/ftpd broken!

[Matthew Seaman]

On 08/14/15 12:45, O. Hartmann wrote:
 Man page ftpusers(5) states, that an entry username allow will allow 
 access
 to ftpd. But every user listed in /etc/ftpusers is denied access, no matter
 whether there is allow appended to the entry or not! This is strange.
 Whenever I delete a user's name from that file I wish to have access to the
 ftpd service, that user can login - but addig the users even as username
 allow (no * in the file, nothing else but the initial users names) access is
 denied.

If you've got a ftpusers(5) that presumably comes from some ported
software -- doesn't exist in the base system.  There is pam_ftpusers(8)
in base, although that doesn't seem to be in use by default.

Traditionally 'ftpusers' was just a plain list of usernames or groups
(indicated by a leading '@' character).  According to ftpd(8) it lists
the people *not* allowed access via FTP.

However, other implementations of FTP servers have adopted the ftpusers
file and expanded its capabilities in various ways, by adding some
additional flag fields for each username.  It depends on what ftpd
you're using exactly what syntax is used there.  Properly ported
software should really be using /usr/local/etc/ftpusers though.

Cheers,

Matthew






signature.asc
Description: OpenPGP digital signature

Re: Proposal: make portsnap generate INDEX-${OSREL:R} only by default

[Matthew Seaman]

On 08/07/15 09:40, Matthew Seaman wrote:
 Of course, if you're using custom options, then the ports tree you

the ports *INDEX*

D'Oh!

Matthew



signature.asc
Description: OpenPGP digital signature

Re: Proposal: make portsnap generate INDEX-${OSREL:R} only by default

[Matthew Seaman]

On 08/07/15 05:11, Kevin Oberman wrote:
 Isn't rebuilding the index useful for people running STABLE? I assume that
 I need a current index to get useful output from pkg version -vL=. I am
 probably a bit unusual in that I keep a current ports tre on a STABLE
 system, but there are a couple of ports that I need to build due to custom
 options and I find poudriere overkill for this case. I suspect many people
 running STABLE may use portsnap and build everything from ports. (This use
 to be common fairly recently and likely still is.)

Actually 'pkg version -vL=' uses one of three different methods to get
information about available port/pkg versions:

* by reading the INDEX (if it exists).
* failing that, by running 'make -V PKGNAME' (or similar) but only
  if there's a ports tree on the system.  This is horribly slow.
* failing that, by using the repository catalogue.

So it will cope without an INDEX file if it has to -- that's unless you
use any of the -I, -P or -R flags to tell it exactly what to do.

Of course, if you're using custom options, then the ports tree you
download from portsnap won't necessarily be accurate for your setup
anyhow.  The good news is that it really doesn't have to be.  Pretty
much everything I've run across in dealing with building software out of
the ports will work fine without an index or with an incorrect index.
Maybe a bit slower than otherwise, but frequently it makes no difference.

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: broken symbolic links in /usr/lib

[Matthew Seaman]

On 29/07/2015 05:48, Jamie Landeg-Jones wrote:
 Gary Palmer gpal...@freebsd.org wrote:
 
 As best that I can recall, the permissions of the directory underneath
 the mount point has been causing problems like this for as long as I've been
 using FreeBSD, which is over 20 years at this point.  It's certainly
 bit me in the distant past.
 
 I concur. I always make mount point directories 0111,noschg,nodump - it makes
 them stand out when not mounted, and also stops accidental directory deletion
 potentially stopping a reboot from working.
 
 But yeah, for 20+ years. I've also experienced problems if a mount-point
 directory doesn't have +x access.

A long time ago -- before the millenium -- NeXT machines did away with
the need for a mount-point directory to exist.  So, if you wanted to
mount /foo/bar, only the /foo directory needed to exist prior to the
mount.  Since NeXT was subsumed by Apple, and NeXTStep reborn as MacOSX,
the same is presumably true today all Macs.  (Although I haven't tested
this personally.)

I do wonder why the rest of the world didn't do likewise.  It would make
this sort of problem a non-event.

Cheers,

Matthew





signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] pkg 1.3.0 out!

[Matthew Seaman]

On 23/07/2014 19:38, Kevin Oberman wrote:
 I think one bullet was a bit mangled in French-English translation,
 though. What does The unicity of a package is not anymore origin mean? I
 have a couple of guesses, but I am not really sure. Ithink the best
 translations would be The unicity of a package is no longer the origin,
 but I am unsure of unicity. Uniqueness? That would make sense, but I am

The unique key in the main 'packages' table in local.sqlite has been
changed from just the package origin to a combination of the package
origin and the package name.

In essence, this means we can generate and handle several different
packages from the same origin in the ports.  Or in other words:
*sub-packages*.

While unicity is a legitimate English word, and it actually does mean
pretty much exactly what Bapt wanted to express here, it isn't the way a
native speaker would describe the concept.  However I feel disinclined
to criticize because I'd not have a hope of getting anywhere near the
right way of saying that in French.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: 11.0-CURRENT #1 r267422: OpenLDAP fails to startup out of the blue after buildworld

[Matthew Seaman]

On 12/06/2014 22:37, Steven Hartland wrote:
 According to that useless suggestion to restore from backup, I
 restored the configuration and the users from backups. slapadd works
 fine. But then starting the server fails again.

What user ID did you use to run slapadd as?  It's a common mistake to do
that as root, and then end up with stuff in /var/db/openldap owned by
root, rather than ldap.  Or the contents of
${LOCALBASE}/etc/openldap/slapd.d

Fix is just to chown those two directories to ldap:ldap

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: portmaster/pkg still populating /var/db/pkg/

[Matthew Seaman]

On 04/05/2014 09:46, Marc UBM wrote:
 After my last port rebuild (libxml, libxcb, etc.) I realized that
 portmaster is still populating /var/db/pkg with the old db scheme (i.e.
 one directory for each port). At the same time, local.sqlite exists and
 is up to date (I can query the db via pkg just fine).
 
 I switched to pkgng ages ago (sometime during the winter of 2011, I
 believe) - have I missed any necessary steps or is this a known quirk?
 
 I'm using pkg-1.3.0.a10 and pkgconf-0.9.5, uname:
 
 FreeBSD xxx 11.0-CURRENT FreeBSD 11.0-CURRENT #19
 r258254:264321M: Fri Apr 11 06:31:35 CEST 2014
 xxx:/usr/obj/usr/src/sys/SUBMARINE_SMP  amd64

It's a known thing.  portmaster uses a per-package subdir under
/var/db/pkg to stash some data about distfiles.  It just looks
superficially like the old-style pkg DB -- but rest assured, all the
data describing your installed packages is stored in local.sqlite

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: gptzfsboot problem on HP P410i Smart Array

[Matthew Seaman]

On 09/04/2014 22:17, Rainer Duffner wrote:
 Hi,
 
 I found this old thread….
 
 I can’t boot FreeBSD 10 installed with zfsroot on a DL380G7 (P410i 
 controller).
 I tried the installer and I tried installing with mfsbsd10se.
 System has 48GB RAM.
 
 Is there a PR for this?
 
 
 Now, I’ve got to waste 2’600 GB disks (and 300-odd I/Os) for a boot-disk…..

You've got more than 8 drives in your zpool?  I ran into a similar
problem a while back: the bios only tells the OS about the first 8
drives during boot, and that isn't enough to assemble a workin zpool from.

Solution I adopted was to have a USB mem stick with /boot on it --
enough to get the kernel up and running and to assemble the zpool, which
could then provide the root fs perfectly well.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: gptzfsboot problem on HP P410i Smart Array

[Matthew Seaman]

On 09/04/2014 22:52, Rainer Duffner wrote:
 And no, as the server is in a remote datacenter, an USB-stick is not an 
 option.
 
 It’s slow enough booting via a virtual USB-image over iLO...

Uh... it only has to read the kernel+modules from the USB stick one time
while booting.  Otherwise, there really shouldn't be any IO inside /boot
unless you login and do stuff in that directory manually.  Your root
filesystem would be on the normal hard drives.

Anyhow the question is moot, since you don't have the same problem I did.

 No, it’s actually just a single RAID6-0 disk created by the P410i…

If you're going to use the RAID controller to generate a virtual drive,
do you really need to use ZFS on top of that?   Couldn't you partition
your virtual drive and put / onto a small UFS partition and then make a
zpool on the rest?

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: docbook and old package management

[Matthew Seaman]

On 18/02/2014 18:16, John Baldwin wrote:
 On Tuesday, February 18, 2014 12:31:15 pm Michael Butler wrote:
 Seems the old package management went EOL *way* before September .. none
 of the docbook ports install now :-(

 [Updating the pkgdb format:bdb_btree in /var/db/pkg ... - 183 packages
 found (-1 +0) (...) done]
 
 Hmm, does this mean you are using pkg(8) and pkg_add?
 

That's portupgrade(8) and pkg_add.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: PACKAGESITE spam

[Matthew Seaman]

On 22/12/2013 14:55, olli hauer wrote:
 What I agree is having it would be nice to have dedicated syslog
 (like yum.log) instead /var/log/messages since this is already
 spammed by most every process.

Add something like this to /etc/syslog.conf:

!pkg
*.* /var/log/pkg.log

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: PACKAGESITE spam

[Matthew Seaman]

On 22/12/2013 16:08, Matthew Seaman wrote:
 On 22/12/2013 14:55, olli hauer wrote:
 What I agree is having it would be nice to have dedicated syslog
 (like yum.log) instead /var/log/messages since this is already
 spammed by most every process.
 
 Add something like this to /etc/syslog.conf:
 
 !pkg
 *.*   /var/log/pkg.log

... although, yes, it's a bit archaic that there are as standard
LOG_UUCP and LOG_NEWS facilities in syslog(3) -- neither of which are
much used nowadays -- while there is only the generic LOG_DAEMON for
things like webservers, LDAP, XMPP, PPP and many other network services,
and nothing at all for system maintenance type things like updating
packages.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.

PGP: http://www.infracaninophile.co.uk/pgpkey
JID: matt...@infracaninophile.co.uk



signature.asc
Description: OpenPGP digital signature

Re: problems with pkg2ng

[Matthew Seaman]

On 29/11/2013 02:02, Nilton Jose Rizzo wrote:
  in the near past, when I update ports repository via svn, I always use
  pkg2ng to upgrade (or update?) the database, but today I can not do it.

Uh?  pkg2ng is a one time thing.  Its only use is when converting a
system that had previously used pkg_tools to pkg(8).  That happens once,
and after that, you never need it again.

  The command pkg2ng show this error message:
 
 root@valfenda:/home2/rizzo # pkg2ng
 pkg: PACKAGESITE in pkg.conf is deprecated. Please create a repository
 configuration file
 Converting packages from /var/db/pkg
 pkg: PACKAGESITE in pkg.conf is deprecated. Please create a repository
 configuration file
 pkg: fstat() failed for(/usr/local/etc/cups/cupsd.conf.N): No such file or
 directory

This is not pkg2ng specific, but comes from the underlying calls to
various pkg(8) actions.

As it says, the format of pkg.conf has changed with this update, and it
is instructing you on how to modify your pkg.conf so it conforms to the
new norms.  See pkg.conf(5) for details on creating a repository
configuration file.

The fstat() message is a different thing again -- looks like one of the
files owned by the cups port has been deleted.  You can fix that by
forcing a reinstall of that package, but it's pretty innocuous really.

 but when I search in /usr/ports/UPDATING I can not found anything about this
 and no man pages about pkg2ng 

A lot of people seem confused and/or alarmed by this update.  Yes, there
have been a number of problems reported, which are being dealt with.
Whether this warrants an entry in UPDATING I don't know, but the general
tenor of any such entry would be modify your configuration files as
shown in the output of pkg(8).

No, there isn't a man page for pkg2ng.  It was never thought necessary.
 However, if anyone wants to contribute a page for pkg2ng I'm sure it
would be gratefully received.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.

PGP: http://www.infracaninophile.co.uk/pgpkey
JID: matt...@infracaninophile.co.uk



signature.asc
Description: OpenPGP digital signature

Re: pkg(1) ipv6 address not listening

[Matthew Seaman]

On 22/11/2013 01:58, Stanisław Halik wrote:
 Hey,
 
 pkg(1) doesn't like ipv6 from two different subnets:
 
 | ^C15720: connect(6,{ AF_INET6 [2001:41c8:112:8300::50:1]:80 },28) ERR#4 
 'Interrupted system call'
 
 It's up fine, just not listening. Could you fix it, please?

pkg(8) works fine over IPv6 for me.

It also does not have any sort of network listener internally.  Acts as
a client accessing external servers only. Please explain clearly exactly
what you're doing in order to see this error.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkg(1) ipv6 address not listening

[Matthew Seaman]

On 22/11/2013 07:19, Stanisław Halik wrote:
 Hey,
 
 On Fri November 22 2013 07:15:00 Matthew Seaman wrote:
 pkg(1) doesn't like ipv6 from two different subnets:
 | ^C15720: connect(6,{ AF_INET6 [2001:41c8:112:8300::50:1]:80 },28) ERR#4

 pkg(8) works fine over IPv6 for me.
 
 Failing command is:
 
 % pkg update
 
 using default repo configuration. The above line was producted by:
 
 % truss -f pkg update
 
 Unable to access pkg.freebsd.org IPv6 server over two subnets.
 Tested with `nc -v'
 
 2001:470:600d:dead:beae:c5ff:fee1:4407/48
 2a01:4f8:192:50e3::/64
 
 1001 ~ % ssh ananke.laggy.pk cat /etc/pkg/FreeBSD.conf
 FreeBSD: {
   url: pkg+http://pkg.FreeBSD.org/${ABI}/latest;,
   mirror_type: srv,
   enabled: yes
 }

Are you running the ports-mgmt/pkg-devel port or did you install from
GitHub sources?  (I deduce this because the config format with curly
braces does not work with pkg-1.1.4...)

Can't reproduce with 1.1.4:

xenophobe:/usr/local/etc:# pkg -vv
 Version: 1.1.4
 PACKAGESITE:
   PKG_DBDIR: /var/db/pkg
PKG_CACHEDIR: /var/cache/pkg
PORTSDIR: /usr/ports
  PUBKEY:
   HANDLE_RC_SCRIPTS: no
   ASSUME_ALWAYS_YES: no
   REPOS_DIR: /usr/local/etc/pkg/repos/
  PLIST_KEYWORDS_DIR:
  SYSLOG: yes
AUTODEPS: no
 ABI: freebsd:9:x86:64
  DEVELOPER_MODE: no
  PORTAUDIT_SITE: http://portaudit.FreeBSD.org/auditfile.tbz
VULNXML_SITE: http://www.vuxml.org/freebsd/vuln.xml.bz2
 MIRROR_TYPE: SRV
 FETCH_RETRY: 3
 PKG_PLUGINS_DIR: /usr/local/lib/pkg/
  PKG_ENABLE_PLUGINS: yes
 PLUGINS:
   DEBUG_SCRIPTS: no
PLUGINS_CONF_DIR: /usr/local/etc/pkg/
  PERMISSIVE: no
 REPO_AUTOUPDATE: yes
  NAMESERVER:
  EVENT_PIPE:
   FETCH_TIMEOUT: 30
 UNSET_TIMESTAMP: no
SSH_RESTRICT_DIR:
 PKG_ENV:
   DISABLE_MTREE: no

Repositories:
  pkg-test:
 url: pkg+http://pkg.freebsd.org/freebsd:9:x86:64/latest
 key:
 enabled: yes
 mirror_type: SRV
xenophobe:/usr/local/etc:# pkg update -f
Updating repository catalogue
digests.txz 100%  996KB 996.1KB/s 996.1KB/s
00:01
packagesite.txz 100% KB   1.4MB/s   1.1MB/s
00:04
Incremental update completed, 0 packages processed:
0 packages updated, 0 removed and 0 added.

This is using an IPv6-only jail:

xenophobe:/usr/local/etc:# ifconfig em0
em0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500

options=4219bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO
ether 68:05:ca:0b:3d:42
inet6 2001:8b0:151:1:54f9:9484:e8b0:12d1 prefixlen 128
nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
media: Ethernet autoselect (100baseTX full-duplex)
status: active

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: cron(8) improvement

[Matthew Seaman]

On 08/11/2013 04:51, Allan Jude wrote:
 My use case is puppet etc, not ports/packages, so I'll leave the policy
 about packages up to portsmgr@, I just want a less sloppy way to manage
 crontabs with my orchestration system (and feature parity with Linux)

There's two questions here:

   1) Should cron(8) support use of /etc/cron.d and/or
/usr/local/etc/cron.d ?

   Clearly yes it should.  Seems a no-brainer to me.

   2) Should ports / packages populate these cron.d directories?

   This is a much more interesting question.  Effectively its asking
   if a port / package should provide some level of automatic
   configuration -- a thing that has previously been a no-no for
   FreeBSD.

   However, I personally would not be completely against this
   *given* the switch to use of sub-packages.  I think having a
   foo-config sub-package as an optional extra would provide the
   best of both worlds.  People who want the same sort of behaviour
   as you get with most Linux distributions can install the
   pre-canned configuration bits; those who prefer the FreeBSD
   traditional approach can simply not install them.

Done right this should also facilitate people writing their own
customized configuration sub-ports.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.

PGP: http://www.infracaninophile.co.uk/pgpkey
JID: matt...@infracaninophile.co.uk



-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Official FreeBSD Binary Packages now available for pkgng

[Matthew Seaman]

On 02/11/2013 01:55, Eric van Gyzen wrote:
 This kind of proxy configuration is not uncommon.  It would be awesome
 if this would Just Work.  It would remove an impediment to adoption,
 which is especially important in the kind of environments that have this
 kind of proxy configuration.
 
 Simply adding the mirrors' A (and ) records to pkg.freebsd.org might
 suffice.

You seem hung up on the idea that pkg.freebsd.org should resolve to a
list of IP addresses.  It doesn't and for very good reasons.
Admittedly, using eg. 'http://' as the URL scheme for PACKAGESITE URLs
was an error -- it contravenes RFC 2616 -- which is why we will be
switching to a new 'pkg+http://' (or 'pkg+https://', 'pkg+ftp://', etc.)
set of URL schemes with pkg-1.2.x

There certainly are all of the necessary A and  records in the DNS
for the real servers that host the repositories.

If I understand what you're complaining about is that you see behavious
like the following:

   * You download package foo-1.2.3.txz from pkg.freebsd.org

   * Internally, that gets resolved to an HTTP request to eg.
 pkg0.isc.freebsd.org

   * Your web proxy caches this package

   * On another host, you also want to download foo-1.2.3.txz

   * This time the SRV record gets resolved to a different mirror,
 say pkg1.nyi.freebsd.org

   * Your proxy has no way of knowing that foo-1.2.3.txz from pkg1.nyi
 is exactly the same file as foo-1.2.3.txz from pkg0.isc so it
 downloads the whole package all over again.

Yes, this is certainly undesirable behaviour.  I need to run some tests
to determine if this is actually what does happen in practice.  If so,
I've an idea about how this problem might be addressed, but it will
require some changes to the repository configuration.

In the mean time, I suggest just choosing which ever of the
pkg.freebsd.org repositories is closest to you and using it directly -- eg.

cat EOF  /usr/local/etc/pkg/repos/myrepo.conf
pkg0.isc {
url: http://pkg0.isc.freebsd.org/${ABI}/latest
enabled: yes
mirror_type: none
}
EOF

Obviously, substitute which ever one of

   pkg0.isc.freebsd.org   (US West)
   pkg1.nyi.freebsd.org   (US East)
   pkg0.bme.freebsd.org   (Europe)

is appropriate.  And be prepared to deal with that specific mirror being
down or replaced by some other server.

 Alternatively, running an HTTP-redirection service on a host named
 pkg.freebsd.org would offer as much flexibility as the SRV records, if
 not more.  However, it would require maintenance of yet another central
 service.

This is already supported in pkg when using the HTTP mirror type.  This
would entail significantly more administrative effort and hardware
requirement to maintain and keep consistent in the specific case of
pkg.freebsd.org  which is exactly why the SRV mirror type was selected.

Cheers,

Matthew


-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Official FreeBSD Binary Packages now available for pkgng

[Matthew Seaman]

On 02/11/2013 10:15, Matthias Andree wrote:
 I understand from Eric's pist that the issue is that through his
 limiting proxies, the SRV are not available at all so he does not even
 get to the point where he could get the pkgN.nyi.freebsd.org
 http://pkgN.nyi.freebsd.org name back.

That doesn't make sense.  All the DNS SRV lookups on pkg.freebsd.org are
done internally to pkg(8), which then issues an HTTP GET to the specific
mirror selected by its internal algorithms.  The web cache won't see
literal 'pkg.freebsd.org' anywhere in the HTTP traffic -- as far as it
is concerned, it's a simple HTTP request to a specific mirror
'pkg1.nyi.freebsd.org', and can be cached using the usual processes.

What makes it cache unfriendly is that as far as the web cache is
concerned each of the different mirrors appears to be completely
independent of the others.  So at the moment the chance of getting a
cache hit is reduced by a factor of three because of the traffic
distribution across the three mirrors.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Official FreeBSD Binary Packages now available for pkgng

[Matthew Seaman]

On 02/11/2013 11:37, Kurt Jaeger wrote:
 Hi!
 
 On 02/11/2013 10:15, Matthias Andree wrote:
 I understand from Eric's pist that the issue is that through his
 limiting proxies, the SRV are not available at all so he does not even
 get to the point where he could get the pkgN.nyi.freebsd.org
 http://pkgN.nyi.freebsd.org name back.

 That doesn't make sense.  All the DNS SRV lookups on pkg.freebsd.org are
 done internally to pkg(8),
 
 ... which only works, if the DNS server queried answers SRV queries
 with SRV values.
 
 Which is not always true, especially in heavily firewalled environments.

I feel no obligation to do anything to encourage people that
deliberately break the DNS.  They've made their bed, and now they have
to lie in it.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Official FreeBSD Binary Packages now available for pkgng

[Matthew Seaman]

On 31/10/2013 21:04, Allan Jude wrote:
 I wonder if the http+pkg:// protocol can solve this, likely will require
 a patch to fetch to implement the logic to do the dns lookup and make
 the proxies request for the real hostname

It's pkg+http:// or pkg+https:// or pkg+ssh:// or -- well, you get the idea.

pkg+http:// is really exactly the same as the current http://
PACKAGESITE URLs -- the new code pretty much checks that the first four
characters are 'pkg+', moves the string pointer to the following
character (ie the h in http://) and then carries on exactly as it works
right now.  We'll be accepting either form certainly throughout the
lifetime of 1.2.x release, but printing a warning to switch to
pkg+http:// where relevant.

The reason for doing this is that according to RFC 2616 in a URL of the
form http://some.add.ress/ the 'some.add.ress' bit has to be either an A
or a CNAME record that can be resolved to an IP address.  Since we can't
change the meaning of 'http://' in URLs, we've just invented our own URL
scheme.  Once it is in reasonably widespread use it's pretty much going
to be de-facto accepted, and we can apply to ICANN to have the scheme
officially registered.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Official FreeBSD Binary Packages now available for pkgng

[Matthew Seaman]

On 31/10/2013 21:38, Bryan Drewery wrote:
 On 10/31/2013 4:25 PM, Eric Camachat wrote:
  Same result, neither pkg+http:// nor http+pkg:// worked with proxy server.
  
 Top-posting kills babies
 
 pkg+http is NOT supported in 1.1 and as I said, changes nothing.

Also the request that pkg(8) makes after resolving the SRV record is a
bog standard HTTP GET to one of the pkg.freebsd.org servers.  It's using
libfetch, so all the usual environment variables to do with HTTP
proxying should just work.  If you do some traffic capture with eg.
wireshark, you'll be able to see that for yourself, and look at the
details of the HTTP packets.

pkg(8) does take some care to present the modification time of any local
copy of a package it is trying to download thus allowing a web server to
send a 304 'Not Modified' response where relevant.  However there's no
recommendation on what (if any) Expires or Cache-Control headers the
repo's web server should use.  Personally, I just take whatever the
defaults are that come with Apache on my own local repos. Which works
for me just fine, but then again, I don't have any proxying to deal with
in my setup.

If you think that the settings used on the pkg.freebsd.org servers could
be improved, then make your case -- if your arguments have merit, then
I'm sure the server admins will listen.  Note however that this is all
server-side, and not something under the control of your local copy of pkg.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.

PGP: http://www.infracaninophile.co.uk/pgpkey
JID: matt...@infracaninophile.co.uk



signature.asc
Description: OpenPGP digital signature

Re: Hang with nvidia-driver

[Matthew Seaman]

On 13/08/2013 19:37, Alex wrote:
 Hi. I just upgraded to revision 254271 and am experiencing a hang when slim
 starts. The monitor goes blank (no signal) and the fans increase in speed,
 suggesting high CPU usage. I have a GeForce GT 440. As suggested on the
 forums, I tried setting machdep.disable_mtrrs=1 at the boot loader, but
 this did not help.
 
 Does anyone have any advice?

I saw the same symptoms with a recent update to x11/nvidia-driver.

Are you starting slim from /etc/ttys ?  What fixed it for me was to
scrap that and use the rc.d script to start slim.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] No more pkg_install on HEAD by default

[Matthew Seaman]

On 14/07/2013 04:18, Teske, Devin wrote:

 ASIDE: For efficiency, I will actually need three things: (1) a list
 of all packages (2) their descriptions and (3) their run-time dependencies.

That would be the repository catalogue, which you can download by 'pkg
update' (or it will happen automatically before pkg operations.)  Or,
more precisely, the repository /catalogues/ for each of the package
repositories enabled in pkg.conf

 NOTE: I'll need run-time dependencies so that as they're checking
 boxes in the UI, I can update accordingly (has nothing to do with how
 the dependencies are handled when the packages get installed; I'll let
 pkg handle that when it comes time, but for the UI and for the review
 screen, I want the user to be informed and I can do that more
 efficiently if I have a master-list and don't have to make continuous
 queries).

In 1.1 the repository catalogue is just the collection of all the YAML
format metadata from all the packages in the repository, plus optional
cryptographic signatures.  While you could try and parse that directly,
there's little point in re-inventing that wheel -- pkg will load the
catalogue data into a sqlite database, and it provides 'pkg search' or
'pkg rquery' specifically for searching catalogue data. ('pkg rquery' is
aimed at scripting use, 'pkg search' is more for interactive use.)

 That being said, I was planning on doing a pkg rquery to get that
 master list of [minimally] 3-pieces of information.

Something like:

pkg rquery -a '%n-%v %o\n%e\n%dn-%dv %do\n---\n'

 What protocol does that rquery run on? What would one have to do
 to set up their own server that can accept an rquery? (our customers
 don't have Internet access)

It's built around SQL queries against the sqlite database built from the
repository catalogues.  Once you've got the repo catalgoue, it doesn't
need network access to just query the data.

 Last but not least...
 
 Can you do an rquery on a local repository? (say, one that has
 been mounted via NFS or some other media, local or otherwise but looking like
 a local filesystem once-mounted). What would be required to get a local
 repository like that going?

You can certainly build a local repository -- essentially you just
assemble a set of packages in some sort of directory heirarchy, and then
just run 'pkg repo' to create the catalogue.

You can then access the repo by any of the protocols supported by
libfetch -- which includes file://, suitable for a repo on a local or
NFS mounted directory.  Beyond what libfetch provides, you can also use
ssh://.

You'ld usually use poudriere or similar to build your own set of
packages to populate your private repository, but there's nothing to
stop you eg. mirroring some selection of packages from a public package
repository and building your private repo from them.  Or configuring
your systems to use a local package repository /and/ a public
repository.  See pkg-repository(5).

 I do suspect that HTTP_PROXY support is probably not available as I
 recall seeing a post where someone was asking about that getting
 implemented for fetch. I'll have to research that again, though.

 Thanks. Keep me up to date on that.

pkg will use the proxing capabilities of libfetch -- so it will abide by
any HTTP_PROXY or FTP_PROXY or any of a number of other environment
variables as described in fetch(3).  You can use the PKG_ENV setting in
pkg.conf(5) to set the environment use by pkg commands.

 Neither sysinstall nor bsdinstall really give HTTP_PROXY access much
 thought (they support it, but only minimally). They just construct a
 raw HTTP header with the FTP URL and send that directly to the proxy.
 One cute thing it does (when initializing the connection) is test for
 Squid and if-so, appends ;type=i to the URL (to force binary
 download versus ascii).

Everything that pkg downloads is currently in the form of an xz(1)
compressed tar archive[*] so definitely needs to downloaded in binary mode.

[*] Even in some cases where the tarball contains only one file, which
is a bit of a nonsense, but not hugely so.  That may well be changed in
future versions.

 No support for proxy-server authentication (however user/pass
 authentication for the FTP server is passed through to the
 proxy-server).

pkg uses libfetch's proxy auth support.  You can also authenticate
access to a repository via FTP or HTTP or HTTPS by encoding a username
and password in the repository URL in the usual way.  For access over
SSH, use any of the authentication mechanisms provided by the ssh server
-- use of ssh-agent(1) and key-based auth is recommended, to avoid
having to type in passphrases repeatedly.

I think if you pkgng-ify a test system and have a play with the
capabilities pkg(8) provides (and maybe poudriere(8) too), you will be
pleased, maybe even pleasantly surprised, by what it can do.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc

Re: [HEADSUP] No more pkg_install on HEAD by default

[Matthew Seaman]

On 14/07/2013 06:48, Teske, Devin wrote:
 Question: Where can I learn more about the actual format of what's in
 the new tarballs? This is going to be important not for bsdconfig,
 but $work (we have our own build platform; I'm going to have to
 rewrite it from mastering PLIST files to mastering YAML MANIFEST
 files and I want to know all the gritty details).

We do need a pkg-manifest(5) man page, which can double as a
pkg-tarball(5) page since the manifest file is where most of the
interesting bits are.

A pkg tarball is a compressed tar archive like so:

lucid-nonsense:...cache/pkg/All:% tar -tvf pkg-1.1.4.txz
-rw-r--r--  0 root   wheel 530 Jan  1  1970 +COMPACT_MANIFEST
-rw-r--r--  0 root   wheel6385 Jan  1  1970 +MANIFEST
-rw-r--r--  0 root   wheel   17567 Jan  1  1970 +MTREE_DIRS
-r--r--r--  0 root   wheel   19453 Jul  7 12:26
/usr/local/etc/bash_completion.d/_pkg.bash
-r-xr-xr-x  0 root   wheel 629 Jul  7 12:26
/usr/local/etc/periodic/daily/400.status-pkg
-r-xr-xr-x  0 root   wheel 823 Jul  7 12:26
/usr/local/etc/periodic/daily/411.pkg-backup
-r-xr-xr-x  0 root   wheel 678 Jul  7 12:26
/usr/local/etc/periodic/daily/490.status-pkg-changes
-r-xr-xr-x  0 root   wheel2558 Jul  7 12:26
/usr/local/etc/periodic/security/410.pkg-audit
-r-xr-xr-x  0 root   wheel 611 Jul  7 12:26
/usr/local/etc/periodic/security/460.pkg-checksum
-r--r--r--  0 root   wheel 839 Jul  7 12:26
/usr/local/etc/pkg.conf.sample
-r--r--r--  0 root   wheel   43432 Jul  7 12:26 /usr/local/include/pkg.h
-r--r--r--  0 root   wheel  727558 Jul  7 12:26 /usr/local/lib/libpkg.a
lrwxr-xr-x  0 root   wheel   0 Jul  7 12:26 /usr/local/lib/libpkg.so
- libpkg.so.1
-r--r--r--  0 root   wheel 1227064 Jul  7 12:26 /usr/local/lib/libpkg.so.1
-rw-r--r--  0 root   wheel 187 Jul  7 12:26
/usr/local/libdata/pkgconfig/pkg.pc
[... etc ...]

There must at least be a +MANIFEST -- other meta data files are
optional.  +COMPACT_MANIFEST is a subset of the full +MANIFEST.  They're
both YAML documents.  +COMPACT_MANIFEST looks like this, for example:

---
name: pkg
version: 1.1.4
origin: ports-mgmt/pkg
comment: New generation package manager
arch: freebsd:9:x86:64
www: http://wiki.freebsd.org/pkgng
maintainer: port...@freebsd.org
prefix: /usr/local
licenselogic: single
licenses:
- BSD
flatsize: 6311507
desc: |
  New Generation package management tool for FreeBSD

  WWW: http://wiki.freebsd.org/pkgng
categories:
- ports-mgmt
shlibs_required:
- libpkg.so.1
shlibs_provided:
- libpkg.so.1
message: |
  If you are upgrading from the old package format, first run:

# pkg2ng

+MTREE_DIRS is a compatibility thing with the old pkg_tools.  It's not
needed in general as +MANIFEST can provide all that meta data itself.
It isn't going to be deprecated for at least as long as the ports tree
continues to support pkg_tools though.

Beyond that, the rest of the pkg tarball just contains a tar archive of
all the files, directories, sym-links etc to be installed by the
package.  Note that pkg(8) has no problem with creating an empty
directory for a package, unlike pkg_tools.

Now, while you can grovel through the details of pkg tarballs and
internal data formats like this, be aware: the format of the manifest
and the details of the meta-data included in the pkg-tarballs is subject
to change without warning as we develop pkg(8) further.  We only promise
API stability through the pkg(8) commands or for the libpkg.so library
functions; reports of breakage from usage outside those APIs will
receive little sympathy.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] No more pkg_install on HEAD by default

[Matthew Seaman]

On 14/07/2013 11:12, Teske, Devin wrote:

 Interesting. I notice that (while looking ahead to see a prefix: of
 /usr/local in the +MANIFEST), the tarball itself has files that include
 /usr/local in their path.

Yes -- we consider the $PREFIX to be 'baked into' the package at compile
time.  You can relocate a package to some extent at installation time --
so for instance you can install into a jail or chroot from outside it --
but the software would expect all the files to be arranged as shown in
the package at runtime.

 Does pkg handle packages where the prefix (in +MANIFEST) is
 /usr/local _but_ tar tf of the unxz'd tarball is _not_ /usr/local?
 (e.g. pkg_install style) or is this the new way going forward?

No -- it always wants a fully qualified path there.

 So just to give you a better idea of how we manage packages here.
 
 We've realized that if you want to package for FreeBSD (in 9 and
 older), you could get by alright if you knew how to create a
 +CONTENTS file from scratch. For RedHat, it's the SPECFILE. And now
 for FreeBSD 10, it looks an awful lot like a SPECFILE (they are both
 in-fact YAML).
 
 So rather than teach the people here how to use tools, I taught them
 what I think is more important... how to manage +CONTENTS, SPECFILE,
 and now up-and-coming, +MANIFEST.
 
 However, I'd be lying if I said I taught them *all* the gory details.
 In reality, for +CONTENTS they edit a PLIST which is essentially
 +CONTENTS without the @comment MD5: entries. For SPECFILE, they
 manage the full thing except there's a small section that is the
 standard RPM bootstrap that is labeled as do not touch.
 
 From what you posted of +COMPACT_MANIFEST was nice, except it lacks
 the list of files.

There's actually two lists: files and directories.  The principle
difference being that the files list contains SHA256 checksums of each
file.  (Although the files list does also contain sym-links etc, which
don't have a checksum).

There are other data in the full manifest but not in the compact one --
pre- and post- installation or deinstallation scripts are the most
important bits not yet mentioned.

 The basic principle here is that if you have to maintain a list of
 files, and that list ends up being compiled into a +MANIFEST, why not
 just teach your peers how to read/write/maintain +MANIFEST files (in
 version control of course) so that there are never any mysteries as
 to how the package performs.
 
 I know this sounds screwy... but (as with our other YAML files), it's
 really nice because (as is the case with SPECFILE), we
 version-control things as-close to the end-product as possible (take
 for example the case of pre-install or post-install et cetera
 scripts -- I'm sure a tool like pkg create or pkg_create or other
 could do a good job of assembling the pieces into the +MANIFEST, but
 then it's not being version-controlled as closely.

Actually, 'pkg create' basically takes the manifest and uses it to build
the package.  It has a special mode to combine data as presented during
ports compilation into a manifest -- again, this is a compatibility
thing.  If the ports didn't have to support pkg_tools still we could
generate package metadata more directly in the format needed by the
manifest.

 Before this workflow, mistakes were rampant and there wasn't much
 hope of wrangling the oops, forgot to touch the PLIST or oops,
 forgot to update the post-install script mistakes.
 
 I see a nice bright future in re-working my pkgbase to be able to
 supplant metadata into a revision-controlled +MANIFEST.
 
 Ideally, I don't want them to have to be burdened with maintaining
 both a +MANIFEST and +COMPACT_MANIFEST, so it seems most logical to
 generate one from the other (the latter being the revision-controlled
 copy sans-meta-data; the meta-data is added at make time before
 then running validations and generating the tarball).

Internally, the +COMPACT_MANIFEST is generated from the +MANIFEST --
probably best to think in terms of dividing the manifest into sections,
some of which (lists of files, install scripts) your dev people would
actively modify, some of which can be automatically generated by pkg(8)
(shlib provides/requires tracking[*]) and the rest as static boilerplate
you can just paste together as required to create a manifest.

[*] Actually, this is automatically generated at the time the package is
assembled into a tarball.  You don't need to fill in that section of the
manifest eplicitly.

 The problem of breakage to the system by API changes is less
 important, because we track security releng branches and use binaries
 so changes are very slow to creep in. But since *can* use a different
 framework for each/every major release branch, we can track API
 changes quite easily.

Ah.  Remember that pkg(8) isn't part of the base system and is developed
on a timetable independent of the timetable used by FreeBSD itself.
When we release a new version of pkg, it would be applied simultaneously
on all branches 

Re: BSD sleep

[Matthew Seaman]

On 29/05/2013 05:59, Michael Sierchio wrote:
 On Tue, May 28, 2013 at 4:45 PM, Joshua Isom jri...@gmail.com wrote:
 
 
 You think it's trivial until you read this:

 http://infiniteundo.com/post/**25326999628/falsehoods-**
 programmers-believe-about-timehttp://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time


 Some days have 86400 seconds, some have 86401.  There is a provision for
 two leap seconds to be applied at once, but that hasn't ever happened.
  Still, a truly correct clock, set to UTC, might someday read
 
 23:59:59
 23:59:60
 23:59:61
 00:00:00
 
 How many seconds did that hour have?

Right.  The fact that on very rare occasions a minute may not have 60
seconds in it plus many other corner cases in calculating the current
wall-clock time is an amusing irrelevance.

First of all, sleep deals in local elapsed time, which is a well defined
property even if the displayed wall-clock time would be all over the
place due to DST changes or relativistic effects or whatever.

In this case, I'd be pretty surprised if GNU sleep's algorithm was
anything more complicated than to convert the stated time into seconds
and then sleep that number of seconds.  And to do that conversion, it
wwould just define one minute as 60 seconds, one hour as 60 minutes, one
day as 24 hours, one week as 7 days, perhaps one month as 30 days, one
year as 365 days[*].  Sure, it's simplistic and unsophisticated, but as
an engineering solution it's good enough for the vast majority of
purposes.

Cheers,

Matthew

[*] I haven't checked on GNU sleep, but (for example) this is exactly
what dnssec-keygen(8) does.

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Light humour

[Matthew Seaman]

On 29/04/2013 10:40, Julian H. Stacey wrote:
 Daniel Kalchev wrote:

 On Apr 29, 2013, at 2:37 AM, Diane Bruce d...@db.net wrote:

 http://www.softpanorama.org/Copyright/License_classification/social_roots_of_GPL.shtml

 By any measure a very good one. Could use some editing of course to make it 
 easier to comprehend for readers of different cultures though … and simplify 
 English sentences . :)

 Daniel
 
 I suggest others save time  not read URL above,
   A skim finds pretentious verbiage, socioligist's analysis
   of different views of GPL v BSD people v. Stallman ...
   throughout the XIX century into the early XX

You've just got to love those old Victorian steam-powered computers...

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.

PGP: http://www.infracaninophile.co.uk/pgpkey
JID: matt...@infracaninophile.co.uk
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [HEADS UP] pkgng binary packages regression in 1.0.9. Fixed in 1.0.9_1

[Matthew Seaman]

On 20/03/2013 15:20, Matthias Gamsjager wrote:
  Due to the security incident, there are still no official FreeBSD
 packages.

 
 
 Do you know what the status is on that issue?

Unchanged so far.  No official pkgng packages yet.

However, an end to the wait is apparently in sight.  There has been
mention of work on pkgng building systems.

Cheers,

Matthew

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: using multiple interfaces for same Network Card

[Matthew Seaman]

On 12/03/2013 10:41, Yasir hussan wrote:
 Thanks for notic but all the elebration was for make alias on one interface
 but i want to have multiple interface, i can no where that some one would
 have tring to creating new interfaces and using them, or may be i am
 missing something, just send its solution if have, solution should be for

Sorry, we're not understanding you very well.

If you have a network card with several ethernet sockets on it, then the
OS will already present you with an interface per socket.  That's pretty
obvious, so I guess that's not what you're really after.

Do you mean you want to use VLans?  (virtual local area networks) --
that involves creating what are effectively separate virtual interfaces,
one for ech vlan, all based on the same physical interface.

Note: you need support and configuration for this in you networking
switch gear.

However, to configure vlan interfaces on FreeBSD, edit /etc/rc.conf
to add first a setting to show what vlan interfaces you want to attach
to your physical interface:

vlans_em0=vlan101 vlan102 vlan107

Then you can set the vlan tag on creation of each clan by adding:

create_args_vlan101=vlan 101
create_args_vlan102=vlan 102
create_args_vlan107=vlan 107

plus you'll need to set up IP addresses on the new vlan interfaces
exactly as you would for a physical interface/

See vlan(4) for more detail.

Cheers,

Matthew



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 21/10/2012 10:33, Alex Keda wrote:
 I try use it on my home server.
 In make conf, I have
  WITHOUT_X11=yes
  NO_GUI=yes
 I run pkg2ng, set mirror in pkg.conf 
 and, run
 pkg upgrade -y
 
 It update some packages, and install ~20 new packages, named x*
 
 How I can say It's server, I do not need X on them?

At the moment, that isn't possible using pkgs from the default pkgrepo.
Those packages are compiled using the default options settings in the
ports, which generally means X support is enabled.

So, you need to point your systems at a repo where the pkgs are compiled
with your preferred set of options.  Which boils down to set up your
own repo.  poudriere is a good way of doing that, but possibly overkill
if you only have one system to manage.  portmaster with the pkgng
patches does the job for me.

Eventually we hope to make pkgng much more flexible with respect to
these sort of configuration choices, but that involves a number of
fairly significant changes to the ports (stagedir, sub-packages) as well
as some major new functionality in pkgng itself (importing a full
solver).  Even so, I suspect that for the ultimate in control, compiling
from ports will still beat just about anything else.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkgng | portmaster

[Matthew Seaman]

On 15/10/2012 18:13, Darrel wrote:
 Hello,
 
 Concerning the new pkg system, I reckon that my port updates can take a
 drought.
 
 Would it be best to deinstall portmaster for now?  Portmaster typically
 updates itself, but this might be a special case.
 
 Currently, my postmaster run ends like this:
 
 === Cannot continue
 === Aborting update
 
 === No ORIGIN in /var/db/pkg/portmaster-3.14/+CONTENTS
 
 === No ORIGIN in /var/db/pkg/postgresql-client-9.2.1/+CONTENTS
 
 === No ORIGIN in /var/db/pkg/postgresql-docs-9.2.1/+CONTENTS
 
 === No ORIGIN in /var/db/pkg/postgresql-server-9.2.1/+CONTENTS
 
 Terminated
 (48) @ 12:58:40

Did you apply the pkgng compatibility patch to portmaster?  It's
available in ports-mgmt/portmaster now if you select the option in the
config dialogue.

Once patched, portmaster should interoperate with pkgng pretty smoothly.
 Certainly it should be using pkgng's local.sqlite database to pull out
package contents rather than trying to parse +CONTENTS files.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey





signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 10/10/2012 23:20, Vincent Hoffman wrote:
 That's if you were to patch an already installed copy of portmaster.
  The patch is designed to be placed in
 
  ${PORTSDIR}/ports-mgmt/portmaster/files/
 
  so it would be applied as part of the normal process of building the
  portmaster port. In which case portmaster.sh.in is definitely the
  correct target.

 Actually not so, maybe something has changed recently?

I stand corrected.  Looks like Bryan switched things around a bit when
he imported everything to GitHub.

I'll fix the patch pro-tem although it should become redundant Real Soon
Now.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 10/10/2012 16:52, Simon L. B. Nielsen wrote:
 I read UPDATING, but I'm still not sure what this means when I use
 ports and not packages.

It means that if you're a user of HEAD, and you don't opt out by setting
WITHOUT_PKGNG=yes in make.conf, then:

  * the next time you use the ports, ports-mgmt/pkg will be installed
as a dependency

  * pkgng will be used to register all the ports you subsequently
install into /var/db/pkg/local.sqlite

However, unless you take some preventive action, any ports that were
installed before this update won't be added to the registry in
local.sqlite.  You'll end up with a mix of stuff using the old subdirs
of /var/db/pkg from pkg_tools and the new local.sqlite from pkgng.

Sorting that out is a one-time job to import the pkg_tools data into
pkgng's database using pkg2ng, which is what the instructions in
UPDATING describe.

 Does it mean that I should install pkg to have /var/db/pkg managed,
 but otherwise ports keeps working the same way, or?

pkgng will need to be installed, yes.  You'll need to switch to using
pkgng commands rather than pkg_tools -- eg:

pkg info -a

to get a list of all installed ports.  You need to patch portmaster(8)
if you use that -- although a patched version will soon be available in
ports.  Apart from that, the ports will work pretty much exactly as they
used to do.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 10/10/2012 17:43, Bruce Cran wrote:
 On 10/10/2012 17:31, Baptiste Daroussin wrote:
 Yes there is http://pkg.FreeBSD.org (no website in there no need to
 try to there) which will point you to pkgbeta.freebsd.org where some
 packages resides.
 
 On my systems pkg.freebsd.org doesn't seem to exist:
 
 ping pkg.freebsd.org
 ping: cannot resolve pkg.freebsd.org: No address associated with name
 

% dig IN SRV _http._tcp.pkg.freebsd.org

;  DiG 9.8.3-P1  IN SRV _http._tcp.pkg.freebsd.org
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: NOERROR, id: 34727
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;_http._tcp.pkg.freebsd.org.IN  SRV

;; ANSWER SECTION:
_http._tcp.pkg.freebsd.org. 3600 IN SRV 10 10 80 pkgbeta.FreeBSD.org.

;; Query time: 71 msec
;; SERVER: ::1#53(::1)
;; WHEN: Wed Oct 10 17:50:20 2012
;; MSG SIZE  rcvd: 83



-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 10/10/2012 15:07, O. Hartmann wrote:
 Is ports-mgmt/portmaster now dealing with pkgng?

Not yet.  bdrewery has taken over the portmaster port and pkgng related
updates are expected in the near future.

Until then, you still need to follow the instructions here:

https://github.com/pkgng/pkgng/blob/master/FAQ.md#15

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP] current switched by default to pkgng

[Matthew Seaman]

On 10/10/2012 19:35, Stefan Esser wrote:
 Am 10.10.2012 19:14, schrieb Matthew Seaman:
 On 10/10/2012 15:07, O. Hartmann wrote:
 Is ports-mgmt/portmaster now dealing with pkgng?

 Not yet.  bdrewery has taken over the portmaster port and pkgng
 related updates are expected in the near future.

 Until then, you still need to follow the instructions here:

 https://github.com/pkgng/pkgng/blob/master/FAQ.md#15
 
 In order to get the portmaster-pkgng patch to apply,
 the filename in the second line must be changed:
 
 from  ./portmaster.sh.in
 to./portmaster

That's if you were to patch an already installed copy of portmaster.
The patch is designed to be placed in

   ${PORTSDIR}/ports-mgmt/portmaster/files/

so it would be applied as part of the normal process of building the
portmaster port.  In which case portmaster.sh.in is definitely the
correct target.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkg (aka pkgng) 1.0 released

[Matthew Seaman]

On 09/07/12 12:30, Ivan Voras wrote:
 On 06/09/2012 18:44, Matthew Seaman wrote:
 On 06/09/2012 16:37, Ivan Voras wrote:
 
 Hi,

 It looks like the pkg port installs pkg.conf.sample with the line:

 PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest

 ... which is finally a good step in the direction of making pkgng
 working by default, except that the pkg.freebsd.org site doesn't exist
 in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should
 be a DNS CNAME for the other?

 It's a SRV record:

 seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org
 10 10 80 pkgbeta.FreeBSD.org.
 
 Hi,
 
 What are the benefits of doing it this way?

Yeah -- it's a bit OTT right now given there's just the one publicly
available pkg repository available.  It will pay off later when there
are more pkg repositories available -- repositories can be added to (or
removed from) the list in the SRV record without end-users having to
know the details.

It may also be possible to replicate what portupdate has done and use
geolocation based services to steer end users to a nearby repository
site automatically.

Cheers,

Matthew


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: pkg (aka pkgng) 1.0 released

[Matthew Seaman]

On 09/07/12 12:54, Matthew Seaman wrote:
 It may also be possible to replicate what portupdate has done and use
 geolocation based services to steer end users to a nearby repository
 site automatically.

D'Oh! I meant portsnap of course.

Matthew
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: pkg (aka pkgng) 1.0 released

[Matthew Seaman]

On 06/09/2012 16:37, Ivan Voras wrote:
 On 30/08/2012 16:19, Baptiste Daroussin wrote:
 Hi all,

 Since Julien Laffaye and I started pkgng lots of things has happened and 
 here we
 are now.

 After 2 years of development (first commit Tue Sep 7 2010), more than 2000
 commits, 43 different contibutors.  The pkgng team is proud to release 
 pkg-1.0!
 
 Hi,
 
 It looks like the pkg port installs pkg.conf.sample with the line:
 
 PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest
 
 ... which is finally a good step in the direction of making pkgng
 working by default, except that the pkg.freebsd.org site doesn't exist
 in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should
 be a DNS CNAME for the other?

It's a SRV record:

seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org
10 10 80 pkgbeta.FreeBSD.org.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

[Matthew Seaman]

On 02/09/2012 01:04, Tim Kientzle wrote:
 Will new versions of pkgng support old packages?
 
 Some folks maintain their own package repositories and
 will get rather grumpy if an update to pkgng requires them
 to rebuild their entire repository.

There's a distinction between the format of pkg tarballs, and the
formats of the repository catalogue database or the locally installed
packages database.

If you're maintaining your own repository, then an update to the repo
catalogue format means you'ld just need to re-run 'pkg repo'.  You won't
need to rebuild all the existing package tarballs in your repository.
If the repository catalogue format has changed, pkg repo will detect
this, and automatically do a full repo catalogue rebuild rather than an
incremental one.

As rebuilding the repo database is something you'ld do routinely anyhow
as part of normal maintenance I don't see this as being a significant
extra burden.

Similarly, an update to the locally installed packages database schema
will be applied transparently when you first use the updated version of
pkgng.  It won't require you to reinstall any packages.

There aren't any plans to change the pkg tarball format that I know of
at the moment, but if there were, then they certainly would have to
maintain backwards compatibility -- old pkg tarballs will still work
with the newer pkgng.  Not sure about any guarantees that vice-versa
would always work, but way the YAML metadata in the pkg tarball is
handled is tolerant of new additions, so it should usually be possible
to arrange things so that an older pkgng can cope with a newer pkg tarball.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

[Matthew Seaman]

On 02/09/2012 08:16, Doug Barton wrote:
 On 09/01/2012 23:01, Matthew Seaman wrote:
 As rebuilding the repo database is something you'ld do routinely anyhow
 as part of normal maintenance
 
 Errr ... what? Why would this be true? Doesn't pkg keep the repo
 database up to date as it's making changes?

Other tools like poudriere or tinderbox are used to maintain the
repository -- building ports etc.  These tools invoke 'pkg create' to
create each individual pkg tarball, and at the end of a session of
package building invoke 'pkg repo' one time to update the repository
catalogue.  It's that last step I was describing.

Mind you, having a mode to add a package to the repo and update the
catalogue all in one would be pretty useful.  Good idea.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap

[Matthew Seaman]

On 01/09/2012 18:43, Tijl Coosemans wrote:
 In this scenario the ports tree needs to keep support for older releases,
 but that's a consequence of the fact that there's only one ports tree for
 all releases. Somewhere in between the ports and the various releases there
 has to be some form encapsulation, not just for pkg, but for all the tools
 used by the ports tree. Given how the ports tree currently encapsulates
 both the old and new pkg tools I don't see how supporting multiple versions
 of pkgng would be a problem because presumably the difference between pkgng
 versions is going to be much smaller than the difference between the old
 and new tools.

New functionality already in the process of development will entail
making non-backwards compatible changes to the DB schema.  If we're tied
to supporting a version of pkgng bundled with a release, that's going to
make rolling out such changes much harder.  On the other hand, if pkgng
is in ports, then we can release a new version and simultaneously
publish new package sets (incorporating the update to pkgng) from the
repositories which will have been built using the updated DB schema.

The ports tree doesn't track the versioning of the base system, and it
makes perfect sense to me that tools for dealing with the ports should
follow changes to ports rather than changes to the base.

Cheers,

Matthew


-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: Script to set/unset automatic status in PKGNG database

[Matthew Seaman]

On 30/08/2012 22:44, John Nielsen wrote:
 After dialog(1) exits the script has a list of packages to mark as
 automatic. Is there a non-SQL way to efficiently get the inverse?
 I.e. the set { all_packages - new_automatic_package_list } ?

Use pkg query - it is really quite powerful.  This shows all
non-autoremove packages as name-version:

pkg query -e '%a == 0' '%n-%v'

and this shows the port origin for all autoremove packages:

pkg query -e '%a == 1' '%o'

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: portmaster and pkgng

[Matthew Seaman]

On 23/07/2012 09:43, Hartmann, O. wrote:

 I'd like to try pkgng with portmaster. I see that pkg2ng is involving
 the directory /var/db/pkg, so this implies that there may implications
 also for usage with ports-mgmt/portmaster. portmaster is supposed to be
 the tool completely dependend on system's toolsets, isn't it?
 
 I know that pkg is supposed to be more for binary maintainance of the
 system, but I'd like to be stuck with compiling my ports. Is there an
 issue with that?

As Chris says, making your own repository with poudriere is pretty simple.

However, unless you're going to be using pkgng to manage several
systems, you might not want even the (fairly small) bother of setting up
poudriere at all.

Which is fine.

So long as you follow these instructions:

https://github.com/pkgng/pkgng/blob/master/FAQ.md#15

you can then use portmaster(8) pretty much as usual, but with the pkgng
packaging format and package database.

There remains one significant chunk of portmaster functionality still
missing: the ability to install from pre-built packages rather than
ports.  Depending on how you use portmaster, this may or may not have
any day-to-day impact on you.

In theory it is possible to mix usage of binary packages from external
repositories with locally compiled packages, but at the moment this
suffers from exactly the same compatibility problems as doing the
equivalent with pkg_tools would.  Fixing that is definitely coming, but
it's not going to be in release-1.0.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: portmaster and pkgng

[Matthew Seaman]

On 23/07/2012 10:31, Hartmann, O. wrote:
 portmaster now is not recognizing anymore the format of the /var/db/pkg
 folder - for those considered the knowledged no surprise, for me simply
 the indication that portmaster usage isn't usable as usual.

You need to patch portmaster separately from installing pkgng.  Once
that is done, it doesn't complain about missing stuff in /var/db/pkg

Note: use the latest version of the patch from git in preference to what
is included in pkgng distfiles: the patch gets updated following
portmaster's release schedule, not pkgng's.

Here:

https://github.com/pkgng/pkgng/raw/master/ports/patch-portmaster-pkgng

 Well, if I understand it right, pkg is considered to be for binary
 packages and does not make portmaster obsolete, if I'm inclined
 compiling my ports myself, am I right?

Correct.

 Well, I thought I read in here that pkg has now a much more
 sophisticated tracking of dependencies - usage of SQLite implies, that
 there is now a great opportunity of doing well in tracking problems and
 versioning (I might be wrong).

Again, correct.  pkgng replaces grepping through a lot of files under
/var/db/pkg with doing some fairly simple SQL queries, and is in general
a much faster at that sort of thing.

 I tried to follow the chat on the list about pkgng, but for the rush I
 didn't figured out whether portmaster is considered obsolete - I saw
 patches for portupgrade flushing in, so my logic has been falsified by
 that implicitely ...

portmaster is definitely not obsolete.  pkgng doesn't /do/ ports at all,
only packages.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

[Matthew Seaman]

On 16/07/2012 04:32, Peter Jeremy wrote:
 On 2012-Jul-12 10:01:10 +, Baptiste Daroussin b...@freebsd.org wrote:
 What is pkg
 ---
 pkg is a new package manager for FreeBSD. It is designed as a replacement for
 the pkg_* tools, and as a full featured binary package manager.
 
 A couple of specific questions that I haven't seen answered during
 this thread or in the wiki:
 - Can pkgng cope with parallel installs?  What happpens if I
   simultaneously (attempt to) install conflicting packages?

No.  Parallel installs will not work -- the first to start will lock the
DB, and the second won't be able to proceed.

 - If I use pkg delete -f, what happens to packages that depended
   on the forcibly-deleted package?

Nothing.  If you forcibly delete a package it's assumed you understand
that you know you're doing something that can break your system.
pkg check will detect missing dependency packages and reinstall as required.

 - What happens if I delete a package where I've modified one of the
   files managed by the package?

The package is removed, but modified file is not:

# pkg check -s pciids
pciids-20120625: checksum mismatch for /usr/local/share/pciids/pci.ids
# pkg delete pciids
The following packages will be deinstalled:

pciids-20120625

The deinstallation will free 788 kB
Deinstalling pciids-20120625...pkg: /usr/local/share/pciids/pci.ids
fails original SHA256 checksum, not removing
pkg: rmdir(/usr/local/share/pciids/): Directory not empty
 done
# pkg info pciids
pkg: No package(s) matching pciids
# ls -l /usr/local/share/pciids/pci.ids
-rw-r--r--  1 root  wheel  752925 Jul 16 07:05
/usr/local/share/pciids/pci.ids


 - What facilities does it have for auditing and repairing the package
   database? (ie checking for inconsistencies between installed files
   and the content of the package database)

See pkg-check(8)

 - How does it handle the situation where I install a package that
   depends on foo version 1.2.3 but have foo version 1.2.4 (or 1.2.2)
   installed?  What about if I have bar version 1.3, which is ABI-
   compatible with foo version 1.2.3, installed?

This is an open issue at the moment.  If you have foo-1.2.2 installed,
it will upgrade for foo-1.2.3 (which is OK).  If you have foo-1.2.4
installed, at the moment it attempts to downgrade to foo-1.2.3; the
response should be to refuse to do that unless forced.

 - Will it detect that a package install would overwrite an existing
   file?  What does it do in this case?

No.  Existing files are overwritten:

# pkg install pciids
Updating repository catalogue
Repository catalogue is up-to-date, no need to fetch fresh copy
The following packages will be installed:

Installing pciids: 20120625

The installation will require 788 kB more space

92 B to be downloaded
pkg: cached package pciids-20120625: checksum mismatch, fetching from remote
pciids-20120625.txz 100%  163KB 163.5KB/s 163.5KB/s
00:00
Checking integrity... done
Installing pciids-20120625... done


 - I gather it handles update package more intelligently than
   uninstall old package, install new package.  Will it avoid
   replacing an old file with an identical one in the new package?

Yes exactly that.  Files in the older package that are identical in the
newer one are left untouched.  Otherwise, files from the older package
are removed, and files from the newer package are installed.

   If so, what happens to the file metadata (particularly uid, gid
   and mtime)?

Nothing.

 - Can it track user-edited configuration files that are associated
   with packages?

This works in exactly the same way as it does currently in the ports.

 - Can it do 2- or 3-way merges of package configuration files?

No.  In general the package will install sample configuration files,
and will only touch the live config files if either the live configs
don't exist, or the live configs are identical to the sample configs.
This is the standard way things work in the ports at the moment.

 - The README states Directory leftovers are automatically removed if
   they are not in the MTREE.  How does this work for directories
   that are shared between multiple packages?  Does this mean that if
   I add a file to a directory that was created by a package, that
   file will be deleted automatically if I delete the package?

No.  Directories have to be empty before they will be removed.

Cheers,

Matthew


-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey





signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

[Matthew Seaman]

On 16/07/2012 05:22, Jeremy Messenger wrote:
 It's one of reason why I do not agree to remove the shared library
 version from the LIB_DEPENDS, so that way in future someone can add
 support in the package to check on shared library version then prevent
 package to install because it's not ABI compatible. Unless someone
 prefer to do it in the different way than putting shared library
 version in the LIB_DEPENDS is good to me either.

Two points here:

Firstly LIB_DEPENDS is all about *building* packages.  In that case, the
thing that matters is *API* compatibility, not ABI.  Library APIs tend
to be much more stable than ABIs, meaning you can compile your code
against practically any version of a shared library. However, you won't
be able to run your compiled program against a shared library with a
different ABI.  If the API does change incompatibly, then it is fine to
use constraints on the ABI version in a port, but doing this as a matter
of course is just being obstructive to people that may not want to
upgrade dependency shlibs just yet.

Secondly, the ABI version of shared libraries has no effect on the
current dependency resolution mechanisms when installing packages
(either pkgng or the old pkg_tools).  At the moment, the only thing that
is considered are package version numbers.

This is an area where we have plans for dramatic changes with pkgng.  We
want to import a general solver mechanism so that a package can have a
list of generic requirements:

 File /usr/local/bin/foo exists and is executable
 Shared library libfoo.so.3 is installed
 Perl Module Foo::Bar  1.23 is available
 Package foo-0.99 has option BLURFL enabled
 etc. etc.

Packages will similarly have a list of facilities they provide.  The job
of the solver will be to find a set of packages such that there is a
provider for every requirement constrained by the user requirement that
their required package set is installed.

However, making this mechanism workable implies significant changes to
the ports -- introducing sub-packages in particular -- which are
basically incompatible with the existing pkg_tools.  So we need to pkgng
1.0 in place to be able to proceed with further changes.  Also a generic
solver is in itself a substantial piece of code to introduce.  Which is
why it hasn't happened yet.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey





signature.asc
Description: OpenPGP digital signature

Re: [HEADSUP CFT] pkg 1.0rc1 and schedule

[Matthew Seaman]

On 13/07/2012 13:14, Fbsd8 wrote:
 What I want to know is this new pkg system going to remove the
 requirement of having the complete ports tree on my system?
 
 What I am looking for in an port system, is to install a port and any
 files needed for the parent port and its dependents to automatically be
 downloaded. So in the end my system ports tree only contain the files
 used to install the ports I use and their dependents.

Yes, you will be able to use pkgng without having a full ports tree
installed on your system.  You can pretty much do that already, although
the central pkgng package repository is still only in beta.

The only bit of pkgng that still requires the ports to be installed is
'pkg version' which is not critical for maintaining your system.
Modifying 'pkg version' so that it doesn't need to use the ports tree is
an open issue on github:

   https://github.com/pkgng/pkgng/issues/195

Patches / pull requests gratefully received.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW





signature.asc
Description: OpenPGP digital signature

Re: PORTS_MODULES fix

[Matthew Seaman]

On 09/06/2012 18:26, Chris Rees wrote:
 On 9 June 2012 18:15, Doug Barton do...@freebsd.org wrote:
 I have recently tried the PORTS_MODULES knob, and found a problem. The
 ports tree searches for some dependencies by finding a binary in PATH,
 and that fails since by default /usr/local/ isn't there. The attached
 patch fixes that problem.

 It would be more robust to use PREFIX there instead of /usr/local
 explicitly, but I'm not sure how to unravel the mk maze to get that
 value. If anyone has a suggestion for that, I'd be happy to include it.
 
 As you mention, PREFIX is only defined in ports/Mk, and it'd
 definitely be undesirable to be including any of those files :)
 
 The most robust (but unpleasant) solution would be one of the following:
 
 PREFIX?=/usr/local
 PORTSMODULESENV=SYSDIR=${SYSDIR} PATH=${PATH}:${PREFIX}/bin:${PREFIX}/sbin
 
 or the equivalent (and perhaps cleaner, not leaving PREFIX defined)
 
 .if !defined(PREFIX)
 PORTSMODULESENV=SYSDIR=${SYSDIR} PATH=${PATH}:/usr/local/bin:/usr/local/sbin
 .else
 PORTSMODULESENV=SYSDIR=${SYSDIR} PATH=${PATH}:${PREFIX}/bin:${PREFIX}/sbin
 .endif
 
 Both of these will respect make.conf's setting of PREFIX.
 

Shouldn't you be looking for LOCALBASE rather than PREFIX in this context?

Cheers,

Matthew


-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW





signature.asc
Description: OpenPGP digital signature

Re: UFS+J panics on HEAD

[Matthew Seaman]

On 24/05/2012 00:05, Mark Linimon wrote:
 On Wed, May 23, 2012 at 10:58:48PM +0100, Steven Hartland wrote:
  While it might be a shame to see FFS go by the wayside are there any
  big reasons why you would rather stick with FFS instead of moving
  to ZFS with all the benefits that brings?

  - ZFS eats bytes for breakfast.  It is completely inappropriate
for anything with less than 4GB RAM.
 
  - ZFS performs poorly under disk-nearly-full conditions.

  - ZFS is not optimal for situations where there are a lot of small,
randomly dispersed IOs around the disk space.  Like in any sort of
RDBMS.

Even so, ZFS is certainly my personal default nowadays.  On a machine of
any size, the question is not should I use ZFS? but are there any
good reasons why I shouldn't use ZFS? (And if so, what could I do to
make it possible to use ZFS anyhow...)

With Andriy's recent patches to zfboot to extend support for Boot
Environments, it's all starting to look particularly sexy.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: UFS+J panics on HEAD

[Matthew Seaman]

On 24/05/2012 00:05, Mark Linimon wrote:
 On Wed, May 23, 2012 at 10:58:48PM +0100, Steven Hartland wrote:
  While it might be a shame to see FFS go by the wayside are there any
  big reasons why you would rather stick with FFS instead of moving
  to ZFS with all the benefits that brings?

  - ZFS eats bytes for breakfast.  It is completely inappropriate
for anything with less than 4GB RAM.
 
  - ZFS performs poorly under disk-nearly-full conditions.

  - ZFS is not optimal for situations where there are a lot of small,
randomly dispersed IOs around the disk space.  Like in any sort of
RDBMS.

Even so, ZFS is certainly my personal default nowadays.  On a machine of
any size, the question is not should I use ZFS? but are there any
good reasons why I shouldn't use ZFS? (And if so, what could I do to
make it possible to use ZFS anyhow...)

With Andriy's recent patches to zfsboot to extend support for Boot
Environments, it's all starting to look particularly sexy.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: [RFC] Un-staticise the toolchain

[Matthew Seaman]

On 26/04/2012 20:01, Chris Rees wrote:
 hydra# cd /usr/ports  time make MAKE=~crees/bin/make-static index
 
 Generating INDEX-9 - please wait.. Done.
 729.770u 120.841s 7:45.10 182.8%920+2676k 5251+116484io 7750pf+0w
 
 hydra# time make MAKE=~crees/bin/make-dynamic index
 
 Generating INDEX-9 - please wait.. Done.
 771.320u 133.540s 8:07.83 185.4%609+2918k 474+116484io 570pf+0w
 
 We have a 10% slowdown (or 11% speedup, depending on your figures) when
 using a dynamically loaded make.

I don't think you can validly conclude much from just one sample of each
type.  Try repeating those tests enough that you can do some decent
statistics.

Oh, and you should probably either discard the first few results, or
else take pains to flush[*] the buffer cache between each run, so you
end up measuring the same thing repeatably.

Cheers,

Matthew

[*] Unmount and remount /usr/ports should do that I believe.

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: Using TMPFS for /tmp and /var/run?

[Matthew Seaman]

On 31/03/2012 03:05, Benjamin Kaduk wrote:
 P.S. I am somewhat unconvinced by this:
 http://wiki.debian.org/ReleaseGoals/RunDirectory

Those who do not understand /var are condemned to reinvent it?

-- 
Dr Matthew J Seaman MA, D.Phil.
PGP: http://www.infracaninophile.co.uk/pgpkey




signature.asc
Description: OpenPGP digital signature

Re: ABI/architecture identification for packages

[Matthew Seaman]

On 19/03/2012 22:04, Baptiste Daroussin wrote:
 On Mon, Mar 19, 2012 at 06:01:23PM -0400, Eitan Adler wrote:
 On Mon, Mar 19, 2012 at 5:35 PM, Baptiste Daroussin b...@freebsd.org wrote:
 Hi all,

 In order to identify architectures I need to find a uniq id for every
 possibilities (for pkgng)

 arch-class-os-majorversion(-archi_specific_extension)

 os will always be freebsd :) (lower case)

 So why bother?
 
 because some people from other oses are interested in pkgng (not work started 
 on
 this)
 because some vendors might want to change it.
 So I read it from the elf note section

What about if the package happens to contain linux executables?

Cheers,

Matthew


-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Matthew Seaman]

On 19/12/2011 08:27, Lev Serebryakov wrote:
   Here is one problem: we have choice from three items:
 
 (1) Make FreeBSD looks good on benchmarks by fixing FreeBSD
 
 (2) Make FreeBSD looks good on benchmarks by fixing Phoronix
 (communication with them, convincing, that they benchamrks are unfare
 / meaningless, ets)

(2a) Ignore Phoronix, other than explaining concisely why their numbers
are complete balderdash.  Publish our own benchmarks, done with care and
rigour and using well defined, repeatable, peer reviewed methodology
that anyone can repeat.  Aggressively publicise these results.

 (3) Lose [potential] userbase.

Indeed.  Unfortunately performance is /the/ deciding factor in many OS
choices, never mind that it is an impossibly complex subject to
generalise to a few management-friendly numbers in a one-size-fits-all
abstract way.  Having only one source of published numbers suggesting
that OS Foo is better *even if those numbers are completely bogus*
will have a disproportionate effect.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: FreeBSD 9.0-RC2 Available...

[Matthew Seaman]

On 18/11/2011 10:53, Thomas Mueller wrote:
 *default release=cvs tag=RELENG_9
 
 Am I screwed, am I OK, or do I simply have to rerun csup with RELENG_9_0 
 instead of RELENG_9 ?

Not screwed, but you'll be running 9.0-PRERELEASE rather than 9.0-RC2.

If you want to switch to the 9.0-RELEASE branch, change the tag to
RELENG_9_0 and cvsup again.  Then redo the whole buildworld dance.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: Upgrade from source to RC1: problems with /etc : lost users and dbus

[Matthew Seaman]

On 28/10/2011 11:05, Thomas Mueller wrote:
 I still can't login as any nonroot user, even though I see the lines
 in /etc/master.passwd, which I copied back from backup, and if I
 startx as root, there is no response to keyboard or mouse.

pwd_mkdb -p /etc/master.passwd

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: subversion-freebsd dependencies

[Matthew Seaman]

On 06/10/2011 11:34, Andre Oppermann wrote:
 On a newly installed development machine I installed subversion-freebsd
 from ports and ended up with a huge dependency chain. Eventually I got
 the usual gnu-hell (auto-*, lib*), but also python27, tcl-8.5, perl-5.12
 and m4. This is a bit too much. The last four should not be required to
 check out the FreeBSD source tree. They also may conflict with newer
 versions one wants to have on a development machine (python3, perl6, ...).
 
 Is there a way to cut this down a bit and just have a svn client with only
 the necessary stuff?

As it happens, I'm working on some scripts to generate GraphViz
dependency diagrams for ports, and purely coincidentally the
subversion-freebsd port is one of my test cases.  You can see the result
here:

http://www.infracaninophile.co.uk/svnf.png

Now this reflects some local settings on my system, like installing
openssl from ports, but overall, the picture should still be pretty
standard.

One thing to note is that at runtime you only need the dependencies
connected by a continuous sequence of edges marked 'R' (run) or 'L'
(library) (maybe with other letters) and with a green arrow head.  So
all of the autotools stuff, perl, python, tcl, gmake, libtool are
disposable once you've built the port.  Or build packages off-line and
install those, which will avoid installing any fetch/extract/patch/build
dependencies in the first place.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: FreeBSD 9.0-BETA2 or 3?

[Matthew Seaman]

On 28/09/2011 10:28, Thomas Mueller wrote:
 I looked at ftp://ftp.freebsd.org/pub/FreeBSD/releases/
 
 and found a BETA3 but only for some platforms not including i386 and amd64, 
 but that was yesterday.
 
 I looked later during the day and found the BETA3 for i386 and amd64.

Yup.  The stable/9 branch was created a few days ago in SVN, and
correspondingly the RELENG_9 cvs branch. It's taking a little while for
all that and various related changes to percolate through the system.

 Now the question is how to update without trashing the BETA2
 installation with all the applications build from ports.

BETA2 to BETA3 should not require reinstalling any ports.  All the pain
of updating ABI version numbers for 9.x has already happened, so
anything that ran under BETA2 should just work under BETA3.

 I don't want to rebuild everything from ports every time there is a
 new beta or release candidate; that would be a very inefficient use
 of time.

Yes.  That would be silly.  And a waste of time and energy.

 If I can't update with the installer, can I update from source?

This is FreeBSD.  Of course you can use the source.

 Would the best way be to download only the source (src.txz), then
  rm -rf /usr/src/*
  then extract the new source?
 Otherwise I don't know what to cvsup to, and I did read the FreeBSD Handbook. 
  I also read /usr/src/README and UPDATING.
 
 I don't want to update source to HEAD which might now be 10.0-CURRENT.
 

For cvsup / csup, you need to change your supfile to use the RELENG_9
tag.  Something like this:

*default host=CHANGE_THIS.FreeBSD.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_9
*default delete use-rel-suffix
*default compress
src-all

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: ZFS V28

[Matthew Seaman]

On 16/04/2011 00:26, George Kontostanos wrote:
 http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20110317.patch.xz

 This patch failes at sys/cddl/compat/opensolaris/sys/sysmacros.h which i
 think should just be deleted. I tried deleting this file and then
 building produces some errors then walls of text and then aborts.
 The first errors look like this:

Hmmm... this worked for me on stable/8 r220471 a week ago

Yes, you do have to delete sysmacros.h by hand. I never did understand
why that header couldn't be deleted cleanly by the patch, but
wotthehell: it's just one file out of the hundreds touched by this patch.

Also use the -E flag to patch to remove empty files.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: tzsetup disregards setting TZ to UTC

[Matthew Seaman]

On 28/03/2011 12:08, Matthias Andree wrote:
 Perhaps the installer should instead:
 
 display CMOS and system time, and ask which one of them is correct, and
 offer a third option to actually correct the timezone or time if neither
 is correct.
 
 That's much easier to grasp.

... and a 4th option for when both are correct.  Happens quite a lot
round these parts in the winter.  However, there are very few people for
whom DST is the same as UTC.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: why panic(9) ?

[Matthew Seaman]

On 12/01/2011 16:23, Erik wrote:
 On one of my first linux desktops, I had a screensaver which displayed
 rotated dumpscreens of all kinds of different Operation systems. Apple,
 Basic, linux and BSOD.. (come to think about it BSD was not included)
 

I once had someone commiserate with me on having so many problems with
my desktop while running that screen saver...

Later versions of the BSoD screen saver certainly did have a pretty
convincing FreeBSD panic and dump sequence.

Cheers,

Matthew



-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: [Call for Tests] PAT issue on Apple hardware

[Matthew Seaman]

On 22/11/2010 05:55, Matthew Seaman wrote:
 lucid-nonsense:/:# zfs snapshot -r zr...@20101122-0549
 lucid-nonsense:/:# cd /.zfs/snapshot/20101122-0549
 /.zfs/snapshot/20101122-0549: Not a directory.
 

I can't reproduce this.  False alarm.  Sorry for the noise.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: [Call for Tests] PAT issue on Apple hardware

[Matthew Seaman]

On 21/11/2010 23:16, army.of.root wrote:
 On 10\11\19 19:54, Jung-uk Kim wrote:
 On Tuesday 16 November 2010 03:30 pm, Jung-uk Kim wrote:
 On Monday 15 November 2010 08:36 pm, Jung-uk Kim wrote:
 Often times I hear complaints like my Mac hangs after upgrading
 to 8.1 or snapshot CD hangs on my brand new Mac.  I know some
 of these complaints started happening when we switched to new PAT
 layout.  It is so puzzling because it never happened on non-Apple
 hardware, AFAIK.  I really like to fix this problem but I cannot
 afford a Mac. :-P

 If you are one of those lucky people, please test the attached
 patch and report your hardware model and any improvement or
 regression.

 Also, I added a new tunable vm.pmap.pat_works so that you can
 turn it off from loader (i.e., set vm.pmap.pat_works=0) and
 restore old behaviour without recompiling a new kernel.

 I revised this patch to make it more robust.

 http://people.freebsd.org/~jkim/pat-current.diff

 Also, I prepared a patch for stable/8.  If you have recent Apple
 hardware and it hangs with 8.1 or stable/8, please test this patch.

 http://people.freebsd.org/~jkim/pat-stable.diff

 Anyone?  I don't want to commit it blindly. :-(

 Jung-uk Kim
 
 works for me too!
 
 Hi,
 
 I patched the current 8.1-STABLE snapshot CD's kernel and baked it back
 into the iso.
 
 Before the patch, the kernel would hang, now the livefs cd boots fine on
 my MacBookPro5,5 (2009 13 alu).
 
 Thank you so much ! - I love it.
 
 I attached a dmesg. - Please ask if you need more information, I'd be
 happy to help.
 
 Keep up the awesome work :)
 
 Thanks

Still trying to get the patched 8.1-STABLE built as a release DVD so I
can try it out on my MacBookPro, but...

... on the build box, I'm getting problems accessing ZFS snapshots using
RELENG_8 from Sat 20th with the pat-stable.diff patches:

lucid-nonsense:/:# zfs snapshot -r zr...@20101122-0549
lucid-nonsense:/:# cd /.zfs/snapshot/20101122-0549
/.zfs/snapshot/20101122-0549: Not a directory.

lucid-nonsense:/:# zfs list -t all
NAME  USED  AVAIL  REFER  MOUNTPOINT
zroot25.3G   416G  2.09G  legacy
zr...@20101122-054945K  -  2.09G  -
zroot/tmp 148K   416G   120K  /tmp
zroot/t...@20101122-054928K  -   120K  -
zroot/usr12.8G   416G  2.13G  /usr
zroot/u...@20101122-0549  0  -  2.13G  -
zroot/usr/home   7.62G   416G  7.62G  /usr/home
zroot/usr/h...@20101122-0549 0  -  7.62G  -
zroot/usr/obj1.21G   416G  1.21G  /usr/obj
zroot/usr/o...@20101122-054953K  -  1.21G  -
zroot/usr/ports  1.54G   416G   375M  /usr/ports
zroot/usr/po...@20101122-05490  -   375M  -
zroot/usr/ports/distfiles1.04G   416G  1.04G
/usr/ports/distfiles
zroot/usr/ports/distfi...@20101122-0549  0  -  1.04G  -
zroot/usr/ports/packages  134M   416G   134M
/usr/ports/packages
zroot/usr/ports/packa...@20101122-0549   0  -   134M  -
zroot/usr/src 306M   416G   305M  /usr/src
zroot/usr/s...@20101122-0549   504K  -   305M  -
zroot/var10.3G   416G   297M  /var
zroot/v...@20101122-0549   162K  -   297M  -
zroot/var/crash  21.5K   416G  21.5K  /var/crash
zroot/var/cr...@20101122-05490  -  21.5K  -
zroot/var/db 10.0G   416G  10.0G  /var/db
zroot/var/d...@20101122-0549   0  -  10.0G  -
zroot/var/db/pkg 10.5M   416G  10.5M  /var/db/pkg
zroot/var/db/p...@20101122-0549   0  -  10.5M  -
zroot/var/empty18K   416G18K  /var/empty
zroot/var/em...@20101122-05490  -18K  -
zroot/var/log3.36M   416G  3.36M  /var/log
zroot/var/l...@20101122-0549  0  -  3.36M  -
zroot/var/mail   21.5K   416G  21.5K  /var/mail
zroot/var/m...@20101122-0549 0  -  21.5K  -
zroot/var/run 114K   416G  97.5K  /var/run
zroot/var/r...@20101122-054917K  -  97.5K  -
zroot/var/tmp 371K   416G   371K  /var/tmp
zroot/var/t...@20101122-0549  0  -   371K  -

Possibly coincidental -- I'm building world without the patches now, but
I won't know the result until I get back from work this evening.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW

Re: /tmp and swap space

[Matthew Seaman]

On 29/07/2010 16:35, gahn wrote:

 is it possible to create /tmp directory under swap space? under
 solaris, it is automatically created under swap unless one specifically
 instructs the system not to do so..

Yes, this is certainly possible.  Just add:

   tmpmfs=YES
   tmpsize=32m

to /etc/rc.conf, and then either reboot, or run:

   /etc/rc.d/tmp start

Note that this will mount a /tmp filesystem on top of anything that's
already there, thus making those files inaccessible until you unmount
again.

(use whatever value of 32m you prefer, of course)

Cheers,

Matthew



signature.asc
Description: OpenPGP digital signature

Re: periodic script in base system to run csup

[Matthew Seaman]

On 17/07/2010 24:04:38, Lowell Gilbert wrote:
 Alex Kozlov s...@rm-rf.kiev.ua writes:
 
 On Fri, Jul 16, 2010 at 04:27:39PM +0200, Gabor Kovesdan wrote:
 Em 2010.07.16. 16:23, Alex Kozlov escreveu:
 On Fri, Jul 16, 2010 at 03:58:33PM +0200, Gabor Kovesdan wrote:

 Thousands pc simultaneously try to access cvsup servers?
 Sound like a ddos to me.
 Yes, this was the only concern and that's why I started this discussion.
 And because its periodic, We can't use portsnap solution (random delay
 before csup start).
 
 It's not completely impossible; periodic could spin off a separate shell
 for it, with a random delay.  It's not clear what the best way to deal
 with the output would be, although several approaches present themselves.
 It would be a lot more complicated than Gabor's approach, though.

Simply ensuring the csup periodic job is the last one to run
(/etc/periodic/daily/1000.csup ?) should give you the best of both
worlds.  You can insert a random delay of up to an hour and still deal
with csup as a foreground job.  All of the other periodic jobs will run
as normal (and should help with randomising the time distribution of the
csup runs too) -- you'll just have to wait a bit longer for the nightly
e-mail to be produced.

Even so, I think this is still likely to upset the cvsup servers: a
whole timezone worth of machines hitting a small number of servers
within one or two hours might be doable with portsnap / freebsd-update
but cvsup requires a lot more effort server-side.

Cheers

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW



signature.asc
Description: OpenPGP digital signature

Re: Regression in GSSAPI/libxh509 linking? [PR bin/147175]

[Matthew Seaman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/07/2010 15:14:28, Andrew Reilly wrote:
 So: how should I fix this, properly, on my -current system? Is it
 as simple as installing heimdal from ports? I can't remove openssl-1.0:
 that has 191 ports listed in its REQUIRED_BY file.

Rebuild the port of openssl-1.0.0 after modifying the OPTIONS to include
MD2=on ?

Cheers,

Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwzfUQACgkQ8Mjk52CukIx/gwCfW2/S+OgDEKz5ubUa3Ajv9V0x
suUAn0r5zUiodJRiwrekZOLuKaI4uFHX
=Zh4/
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Regression in GSSAPI/libxh509 linking? [PR bin/147175]

[Matthew Seaman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06/07/2010 23:26:03, Matthias Andree wrote:
 Am 06.07.2010, 21:00 Uhr, schrieb Matthew Seaman:
 
 On 06/07/2010 15:14:28, Andrew Reilly wrote:
 So: how should I fix this, properly, on my -current system? Is it
 as simple as installing heimdal from ports? I can't remove openssl-1.0:
 that has 191 ports listed in its REQUIRED_BY file.

 Rebuild the port of openssl-1.0.0 after modifying the OPTIONS to include
 MD2=on ?
 
 Not good given that MD2 is broken. Very broken, not just by a factor of
 2^5 or something.
 
 Where upon rests the earlier assertion (not by Matthew) that Kerberos V
 needed MD2 checksums?
 I can't seem to find that in the KRB5 protocol and checksum RFCs. If
 it's not mandatory we may want to nuke MD2 from Kerberos to remedy a
 weakness... Chapter and Verse welcome.

Yeah.  Even so, lots of software still expects it to be present and
won't link without it.  I hope no one is actually using it, or running
with a cipher configuration that would permit it to be used.

Cleaning all reliance on MD2 out of the ports and base would make a very
good project for a bunch of people, and pushing those changes upstream
would certainly help make the internet a better place.  Probably should
start with an experimental run on a tinderbox somewhere trying to build
all ports that are OpenSSL consumers against security/openssl with MD2
turned off.

Cheers,

Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkw0CfsACgkQ8Mjk52CukIzTAQCeOmkWeudx4UCnxI5wFBNrcAuY
x80AnivuyK8mPfOPHPUe7Y95uMMpUSVo
=PHpX
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: [TESTING]: ClangBSD branch needs testing before the import to HEAD

[Matthew Seaman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31/05/2010 16:03:07, Daniel Eischen wrote:
 Is clangBSD able to support all our architectures?  Does it
 cross build for powerpc, mips, etc?  Has it made a ports run
 and does it successfully build and run most of our ports on
 Tier-1 archs, and does it compare similarly with gcc for ports
 on other archs?

Mostly agree, but waiting until clang can compile most of the ports is
going to be a really long wait.  Lots of projects out there won't want
to support anything other than gcc for the forseable future. Presumably
the import of clang to the base does not mean the immediate removal of
gcc.  Is it really such a bad thing to have gcc as a build-dependency
for various ported applications?

Cheers,

Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
JID: matt...@infracaninophile.co.uk   Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwD3UsACgkQ8Mjk52CukIwCiACdGEn8qPpCoCnGN2u+E3S96BnQ
mHAAnAy74w651EtuHf7bkWtjd9WSR/Ny
=5QiA
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: ports and PBIs

[Matthew Seaman]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/04/2010 05:59:34, Robert Noland wrote:
 On Sat, 2010-04-10 at 15:18 +0100, Bruce Simpson wrote:
 On 04/10/10 02:31, Julian Elischer wrote:

 Alfred Perlstein , Matt at ix systems Kris (Mr PBI), some
 others and I, felt that these ideas seemed to make some sense
 and so I put them here for comment.

 Please do. Someone has to do something about deployment.

 For what it's worth, I've tripped over the garden rake on the ground, 
 that is 'unsatisfied dependency' one too many times in commercial work.

 If PBIs can address this, even for FreeBSD's embedded and server use 
 cases, they will likely be well recieved.
 
 If I understood the PBI construct correctly... How is this really that
 different than just producing static binaries?  I mean, as I understood
 it, your bundling the binary and all of it's required libraries into a
 private directory tree and then playing linker games.

Speaking as a recent MacOS re-convert (I used to be a NeXTie a long,
long time ago...) I do like the convenience of the MacOS .dmg format,
and the idea that FooBar.app is a self-contained directory containing
not only the app binary, but all of the various other necessary bits:
supporting docco, icon images and so forth.

If the idea of PBI is to do the same thing for FreeBSD, then yay! All
for it.  But (and you knew there would be a but...) there's a big
difference between the MacOS X environment and FreeBSD.  In MacOS, the
windowning system (Carbon, Cocoa, all that jazz) is part of the /base/
system.   How does that translate into the PBI context?  X and (Gnome or
KDE) as super-packages that you can assume are already there?

Similarly, if you're thinking about server-side applications in the same
way -- if I want to install phpmyadmin as a PBI, does that mean I need
to have a dedicated instance of apache+mod_php for each PHP based app I
want to install?  Or should there be a common Web App environment basic
to all such packages?

Cheers,

Matthew

- -- 
Dr Matthew J Seaman MA, D.Phil.   7 Priory Courtyard
  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
  Kent, CT11 9PW
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvBkBAACgkQ8Mjk52CukIy5eQCcCEU1PmaGZXIkd7BfUTV8kfPc
ES0An08UPz5brQWSf9XNeLtomeg8fIDL
=7sQf
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: named (bind) in jail does not start

[Matthew Seaman]

On Sat, Nov 29, 2003 at 03:23:48PM +0100, Axel S. Gruner wrote:
 
 Hi.
 
 I have configured named in jail (FreeBSD 5.1-RELEASE-p10).
 If i want to start named in the jail
 
 /usr/sbin/named
 
 i get this error message:
   
 opensocket_f: bind([0.0.0.0].53): Address already in use
 
 Ok, Port 53 is not in use in the jail nor the hostsystem.
 I think the problem is 0.0.0.0, and i have to bind named on the IP of
 the jail. 
 
 I tested same named configuration on the hostsystem, i thought about
 some misconfigration, but on the hostsystem named starts perfectly.
 
 I also tried to start named with -g and -u in the jail, same error.
 
 So, my short question is, how can i run named in the jail?
 Any ideas?

Yes.  The problem is that named is attempting to bind(2) to
INADDR_ANY.  In a jail, that includes the loopback address.  Problem
is, jails don't get their own loopback addresses -- there's just the
one loopback shared between the host system and all jails.  Which
effectively means that jailed processes can't bind to the loopback.

The answer is to configure named to only bind to the jail IP number --
see http://www.isc.org/products/BIND/docs/config/ (for bind8) or
http://www.nominum.com/content/documents/bind9arm.pdf (for bind9)
[available in HTML as
file:///usr/local/share/doc/bind9/arm/Bv9ARM.html if you've installed
the bind9 port.]

In bind9 you need to add something like the following to named.conf --
bind8 will be similar:

options {

[...]

listen-on {
192.168.1.1;
};
query-source address 192.168.1.1 port 53;
transfer-source  192.168.1.1 port 53;
notify-source192.168.1.1 port 53;
};

There are equivalent IPv6 statements if you're an IPv6 user.

Cheers,

Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.   26 The Paddocks
  Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey Marlow
Tel: +44 1628 476614  Bucks., SL7 1TH UK


pgp0.pgp
Description: PGP signature