Re: There has to be a better way of merging /etc during a major freebsd-update

2015-03-10 Thread Ben Morrow
Quoth Peter Olsson list-freebsd-sta...@jyborn.se:

 (But I will try running freebsd-update without merging /etc,
 and use mergemaster -F instead. Should solve my problem.)

I'm fairly sure this won't do what you want, and in fact won't work at
all, unless your /etc is identical to the stock /etc installed from the
ISO. (Which it isn't, of course.)

installworld specifically avoids installing the files in /etc; then,
when you run mergemaster, it installs the new versions of those files
into a temporary directory and merges them with the existing /etc. 

freebsd-update works a little differently: because it doesn't have a
source tree available, it has to fetch the stock versions of the files
in /etc for the release you're upgrading from, so that it can patch them
to the new release and then merge the changes into your current /etc. If
you tell freebsd-update to install /etc without merging it will blindly
update files you haven't changed (which is probably what you want) but
(I think) will fail to update the files that you have changed, because
it uses binary patches which won't apply to your modified versions.

If you want a rather hackish solution, you could try something like
this:

- Rename /etc to /oldetc.
- Find yourself a copy of the stock /etc for the version you are
  upgrading from. (tar -xpf base.txz --include /etc)
- Run freebsd-update with /etc removed from the merge list. This
  will (should?) give you a stock /etc for the version you are
  upgrading to.
- Rename /etc - /tmp/etc, /oldetc - /etc and run mergemaster with
  -t /tmp.

Obviously I would script this if I was doing more than one or two
machines.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: 9.2-PRE: switch off that stupid Nakatomi Socrates

2013-09-27 Thread Ben Morrow
Quoth David Demelier demelier.da...@gmail.com:
 
 I personally think (but you may totally disagree with) that an operating
 system *is* an operating system. And I really hate easter eggs or
 anything else not serious being integrated into the system. I think
 about a new user installing FreeBSD 9.2, I would not imagine his
 reaction front of this kind of tribute or whatever you call that. For
 me it stands for that's not serious, it looks like a toy.

Personally I thoroughly approve of a joke every now and then, as long as
it doesn't get out of hand. It reassures me there are actual human
people working on this who care about what they're doing, rather than
some faceless humourless corporation only interested in profit margins.
This in turn reassures me that standards will be kept high and bugs will
be taken seriously, because that's what people who care do.

 Fortunately now it's just a version but I would really not imagine
 when the screen was looking with the tribute loader_logo.

While I like the idea of release names in general, I think it's
important not to make them more prominent than the real version numbers.
I've had to deal with Ubuntu users (I know very little about Ubuntu) in
other open-source contexts, and they tend to talk about 'hardy heron' or
'constipated koala' or what-have-you, which just leaves me thinking
'yes, but which is *more recent*?'. So I might rather the default
version string was something more like 'FreeBSD 9.2-RELEASE (Nakatomi
Socrates)'.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Package database

2013-09-04 Thread Ben Morrow
Quoth Freddie Cash fjwc...@gmail.com:
 On Wed, Sep 4, 2013 at 10:41 AM, Jim Ballantine j.ballant...@gmail.comwrote:
 
  My /var/db/pkg has become corrupt, and I can't find an archive of it to
  install in it's place.  Does one exist and if so where?  If not any ideas
  on
  how to rebuild the db?
 
 Are you using PKGng or the old pkg_* tools?
 
 Meaning, is your data stored as individual files under
 /var/db/pkg/PORTNAME/*, or as a single sqlite database under /var/db/pkg?
 
 If using PKGng, there's a backup copy under /var/db/backup*
 
 If using the older pkg_* tools, you're screwed.  :)

With either package system /etc/periodic/daily/220.backup-pkgdb backs up
the whole of /var/db/pkg into /var/backups/pkgdb.bak.tbz*. 

With pkgng /usr/local/etc/periodic/daily/411.pkg-backup also backs up
the pkgng db into /var/backups/pkgng.db; probably this means
220.backup-pkgdb should be turned off, but this doesn't happen by
default.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: LDAP authentication confusion

2013-07-15 Thread Ben Morrow
Quoth Jan Bramkamp cr...@rlwinm.de:
 On 15.07.2013 21:51, Daniel Eischen wrote:
  
  Wouldn't it be easier just to edit /etc/nsswitch.conf
  anyway?
 PAM and NSS switch are two different subsystems. NSS is just for
 resource lookups (users, groups, hosts, ...). PAM is for access control.
 
 With ldap in nsswitch.conf for users and groups you can lookup a LDAP
 user but the user can't log into $service through PAM. This requires
 pam_ldap.so in pam.d/$service.

The default pam_unix.so calls getpwent, so if nss_ldap returns cryptable
passwords in its result I think pam_unix can authenticate against those.

This is not the same as authenticating by LDAP bind, but may end up
accepting the same passwords.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: request for your comments on release documentation

2013-06-12 Thread Ben Morrow
Quoth h...@freebsd.org:
 
  I would like your comments on release notes for each release.
  Although I have been working on editing them for years, the workflow
  is still not optimal and sometimes delay of the preparation became an
  obstacle for release process.  I would like to improve it, but before
  that I would like to know what are desired of the contents which
  people think.
 
  Release Notes is just listing the changes between the two releases.
  It includes user-visible change (bugfix and/or UI change), new
  functionality, and performance improvement.  Minor changes such as
  one in kernel internal structure are omitted.  I always try to keep
  these series of relnotes items are correct and reasonably
  comprehensive, but this lengthy list may be boring and
  technically-correct descriptions can be cryptic for average users.

I find the lengthy list extremely valuable. It takes a little time to go
through it carefully, but being able to be reasonably sure nothing
important is missing makes upgrades easier, not harder.

  So, my questions are:
 
  1. What do you think about current granularity of the relnotes items?
 Too detailed, good, or too rough?  Currently, judgment of what is
 included or not is based on user-visible, new functionality, or
 performance improvement.  Applicable changes are included as
 relnotes items even if the changes are small,

Seems pretty good to me. The only thing I might change is the order:
generally speaking, I'm most interested in the 'User-visible
incompatibilites' section, then in the userland and contrib changes, and
then the kernel changes. The security advisories section is least
useful, because it generally just lists advisories I've already seen and
know have been already fixed; it's a good thing it's there, if only to
make it clear the project takes security seriously, but I might move it
to the end.

  2. Do you want technical details?  For example, just disk access
 performance was improved by 50% or Feature A has been added.
 This changes the old behavior because ..., and as a result, it
 improves disk access performance by 50%.

It's interesting, but IMHO only worth it if it's easy. It's not worth
holding a release up for.

  3. Is there missing information which should be in the relnotes?
 Probably there are some missing items for each release, but this
 question is one at some abstraction level.  Link to commit log and
 diff, detailed description of major incompatible changes, and so
 on.

The only important additional thing that might be useful would be links
to relevant mailing-list threads in addition to the SVN links. I can see
that might be quite a bit of work to compile, though, so it may not be
possible.

  Although the other release documentations---Errata, Installation
  Notes, ReadMe, and Hardware Notes---also need some improvements,
  please focus on Release Notes only.  And you might think quality of
  English writing are not good, please leave that alone for now.

There's nothing wrong with your English.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: FreeBSD 9.1 excessive memory allocations [SOLVED]

2013-03-28 Thread Ben Morrow
Quoth Unga unga...@yahoo.com:
 
  I think you may be reading too much into the malloc manpage.� When it
  mentions the use of per-thread small-object caches to avoid locking it's
  talking about performance, not thread safety.� Allocations of all sizes
  are thread-safe, the library just assumes that huge allocations are rare
  enough that it doesn't use extra per-thread resources to avoid locking
  for them, it just uses locking for huge blocks.
 
 Good to note all allocations are thread safe in FreeBSD. Is it by some
 standard that malloc should be thread safe regardless the OS (BSDs,
 Linux, Windows, Android, etc)?

POSIX (well, SUSv4 at least) says that malloc and free must be
threadsafe. Note that Windows is not a POSIX system, though I belive
malloc is also always threadsafe on Windows.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: ZFS stalls -- and maybe we should be talking about defaults?

2013-03-05 Thread Ben Morrow
Quoth Steven Hartland kill...@multiplay.co.uk:
 - Original Message - 
 From: Daniel Kalchev
  On Mar 6, 2013, at 12:09 AM, Jeremy Chadwick j...@koitsu.org wrote:
  
  I say that knowing lots of people use ZFS-on-root, which is great -- I
  just wonder how many of them have tested all the crazy scenarios and
  then tried to boot from things.
  
  I have verified that ZFS-on-root works reliably in all of the following
  scenarios: single disk, one mirror vdev, many mirror vdevs, raidz.
  Haven't found the time to test many raidz vdevs, I admit. :)
 
 One thing to watch out for is the available BIOS boot disks. If you try
 to do a large RAIDZ with lots of disk as you root pool your likely to
 run into problems not because of any ZFS issue but simply because the
 disks the BIOS sees and hence tries to boot may not be what you expect.

IIRC the Sun documentation recommends keeping the root pool separate
from the data pools in any case.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: ZFS stalls -- and maybe we should be talking about defaults?

2013-03-04 Thread Ben Morrow
Quoth Karl Denninger k...@denninger.net:
 
 Note that the machine is not booting from ZFS -- it is booting from and
 has its swap on a UFS 2-drive mirror (handled by the disk adapter; looks
 like a single da0 drive to the OS) and that drive stalls as well when
 it freezes.  It's definitely a kernel thing when it happens as the OS
 would otherwise not have locked (just I/O to the user partitions) -- but
 it does. 

Is it still the case that mixing UFS and ZFS can cause problems, or were
they all fixed? I remember a while ago (before the arc usage monitoring
code was added) there were a number of reports of serious probles
running an rsync from UFS to ZFS.

If you can it might be worth trying your scratch machine booting from
ZFS. Probably the best way is to leave your swap partition where it is
(IMHO it's not worth trying to swap onto a zvol) and convert the UFS
partition into a separate zpool to boot from. You will also need to
replace the boot blocks; assuming you're using GPT you can do this with
gpart bootcode -p /boot/gptzfsboot -i gpt boot partition.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Musings on ZFS Backup strategies

2013-03-02 Thread Ben Morrow
Quoth Karl Denninger k...@denninger.net:
 Quoth Ben Morrow:
  I don't know what medium you're backing up to (does anyone use tape any
  more?) but when backing up to disk I much prefer to keep the backup in
  the form of a filesystem rather than as 'zfs send' streams. One reason
  for this is that I believe that new versions of the ZFS code are more
  likely to be able to correctly read old versions of the filesystem than
  old versions of the stream format; this may not be correct any more,
  though.
 
  Another reason is that it means I can do 'rolling snapshot' backups. I
  do an initial dump like this
 
  # zpool is my working pool
  # bakpool is a second pool I am backing up to
 
  zfs snapshot -r zpool/fs at dump
  zfs send -R zpool/fs at dump | zfs recv -vFd bakpool
 
  That pipe can obviously go through ssh or whatever to put the backup on
  a different machine. Then to make an increment I roll forward the
  snapshot like this
 
  zfs rename -r zpool/fs at dump dump-old
  zfs snapshot -r zpool/fs at dump
  zfs send -R -I @dump-old zpool/fs at dump | zfs recv -vFd bakpool
  zfs destroy -r zpool/fs at dump-old
  zfs destroy -r bakpool/fs at dump-old
 
  (Notice that the increment starts at a snapshot called @dump-old on the
  send side but at a snapshot called @dump on the recv side. ZFS can
  handle this perfectly well, since it identifies snapshots by UUID, and
  will rename the bakpool snapshot as part of the recv.)
 
  This brings the filesystem on bakpool up to date with the filesystem on
  zpool, including all snapshots, but never creates an increment with more
  than one backup interval's worth of data in. If you want to keep more
  history on the backup pool than the source pool, you can hold off on
  destroying the old snapshots, and instead rename them to something
  unique. (Of course, you could always give them unique names to start
  with, but I find it more convenient not to.)
 
 Uh, I see a potential problem here.
 
 What if the zfs send | zfs recv command fails for some reason before
 completion?  I have noted that zfs recv is atomic -- if it fails for any
 reason the entire receive is rolled back like it never happened.
 
 But you then destroy the old snapshot, and the next time this runs the
 new gets rolled down.  It would appear that there's an increment
 missing, never to be seen again.

No, if the recv fails my backup script aborts and doesn't delete the old
snapshot. Cleanup then means removing the new snapshot and renaming the
old back on the source zpool; in my case I do this by hand, but it could
be automated given enough thought. (The names of the snapshots on the
backup pool don't matter; they will be cleaned up by the next successful
recv.)

 What gets lost in that circumstance?  Anything changed between the two
 times -- and silently at that? (yikes!)

It's impossible to recv an incremental stream on top of the wrong
snapshot (identified by UUID, not by its current name), so nothing can
get silently lost. A 'zfs recv -F' will find the correct starting
snapshot on the destination filesystem (assuming it's there) regardless
of its name, and roll forward to the state as of the end snapshot. If a
recv succeeds you can be sure nothing up to that point has been missed.

The worst that can happen is if you mistakenly delete the snapshot on
the source pool that marks the end of the last successful recv on the
backup pool; in that case you have to take an increment from further
back (which will therefore be a larger incremental stream than it needed
to be). The very worst case is if you end up without any snapshots in
common between the source and backup pools, and you have to start again
with a full dump.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Musings on ZFS Backup strategies

2013-03-02 Thread Ben Morrow
Quoth Phil Regnauld regna...@x0.dk:
 
  The only risk that makes me uncomfortable doing this is that the pool is
  always active when the system is running.  With UFS backup disks it's
  not -- except when being actually written to they're unmounted, and this
  materially decreases the risk of an insane adapter scribbling the
  drives, since there is no I/O at all going to them unless mounted. 
  While the backup pool would be nominally idle it is probably
  more-exposed to a potential scribble than the UFS-mounted packs would be.
 
   Could zpool export in between syncs on the target, assuming that's not
   your root pool :)

If I were feeling paranoid I might be tempted to not only keep the pool
exported when not in use, but to 'zpool offline' one half of the mirror
while performing the receive, then put it back online and allow it to
resilver before exporting the whole pool again. I'm not sure if there's
any way to wait for the resilver to finish except to poll 'zpool
status', though.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Musings on ZFS Backup strategies

2013-03-01 Thread Ben Morrow
Quoth Karl Denninger k...@denninger.net:
 Dabbling with ZFS now, and giving some thought to how to handle backup
 strategies.
[...]
 
 Take a base snapshot immediately and zfs send it to offline storage.
 Take an incremental at some interval (appropriate for disaster recovery)
 and zfs send THAT to stable storage.
 
 If I then restore the base and snapshot, I get back to where I was when
 the latest snapshot was taken.  I don't need to keep the incremental
 snapshot for longer than it takes to zfs send it, so I can do:
 
 zfs snapshot pool/some-filesystem@unique-label
 zfs send -i pool/some-filesystem@base pool/some-filesystem@unique-label
 zfs destroy pool/some-filesystem@unique-label
 
 and that seems to work (and restore) just fine.

For backup purposes it's worth using the -R and -I options to zfs send
rather than -i. This will preserve the other snapshots, which can be
important.

 Am I looking at this the right way here?  Provided that the base backup
 and incremental are both readable, it appears that I have the disaster
 case covered, and the online snapshot increments and retention are
 easily adjusted and cover the oops situations without having to resort
 to the backups at all.
 
 This in turn means that keeping more than two incremental dumps offline
 has little or no value; the second merely being taken to insure that
 there is always at least one that has been written to completion without
 error to apply on top of the base.  That in turn makes the backup
 storage requirement based only on entropy in the filesystem and not time
 (where the tower of Hanoi style dump hierarchy imposed both a time AND
 entropy cost on backup media.)

No, that's not true. Since you keep taking successive increments from a
fixed base, the size of those increments will increase over time (each
increment will include all net filesystem activity since the base
snapshot). In UFS terms, it's equivalent to always taking level 1 dumps.
Unlike with UFS, the @base snapshot will also start using increasing
amounts of space in the source zpool.

I don't know what medium you're backing up to (does anyone use tape any
more?) but when backing up to disk I much prefer to keep the backup in
the form of a filesystem rather than as 'zfs send' streams. One reason
for this is that I believe that new versions of the ZFS code are more
likely to be able to correctly read old versions of the filesystem than
old versions of the stream format; this may not be correct any more,
though.

Another reason is that it means I can do 'rolling snapshot' backups. I
do an initial dump like this

# zpool is my working pool
# bakpool is a second pool I am backing up to

zfs snapshot -r zpool/fs@dump
zfs send -R zpool/fs@dump | zfs recv -vFd bakpool

That pipe can obviously go through ssh or whatever to put the backup on
a different machine. Then to make an increment I roll forward the
snapshot like this

zfs rename -r zpool/fs@dump dump-old
zfs snapshot -r zpool/fs@dump
zfs send -R -I @dump-old zpool/fs@dump | zfs recv -vFd bakpool
zfs destroy -r zpool/fs@dump-old
zfs destroy -r bakpool/fs@dump-old

(Notice that the increment starts at a snapshot called @dump-old on the
send side but at a snapshot called @dump on the recv side. ZFS can
handle this perfectly well, since it identifies snapshots by UUID, and
will rename the bakpool snapshot as part of the recv.)

This brings the filesystem on bakpool up to date with the filesystem on
zpool, including all snapshots, but never creates an increment with more
than one backup interval's worth of data in. If you want to keep more
history on the backup pool than the source pool, you can hold off on
destroying the old snapshots, and instead rename them to something
unique. (Of course, you could always give them unique names to start
with, but I find it more convenient not to.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Musings on ZFS Backup strategies

2013-03-01 Thread Ben Morrow
Quoth Daniel Eischen deisc...@freebsd.org:
 
 Yes, we still use a couple of DLT autoloaders and have nightly
 incrementals and weekly fulls.  This is the problem I have with
 converting to ZFS.  Our typical recovery is when a user says
 they need a directory or set of files from a week or two ago.
 Using dump from tape, I can easily extract *just* the necessary
 files.  I don't need a second system to restore to, so that
 I can then extract the file.

As Karl said originally, you can do that with snapshots without having
to go to your backups at all. With the right arrangements (symlinks to
the .zfs/snapshot/* directories, or just setting the snapdir property to
'visible') you can make it so users can do this sort of restore
themselves without having to go through you.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Musings on ZFS Backup strategies

2013-03-01 Thread Ben Morrow
Quoth David Magda dma...@ee.ryerson.ca:
 On Mar 1, 2013, at 15:39, Daniel Eischen wrote:
  On Fri, 1 Mar 2013, kpn...@pobox.com wrote:
  
  What about extended attributes? ACLs? Are those saved by tar?
  
  I think tar (as root or -p) will attempt to preserve those.
 
 Specifically bsdtar (with libarchive) and star:
 
 https://github.com/libarchive/libarchive/wiki/TarPosix1eACLs

But since ZFS doesn't support POSIX.1e ACLs that's not terribly
useful... I don't believe bsdtar/libarchive supports NFSv4 ACLs yet.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Old ICH7 SATA-2 question

2013-02-24 Thread Ben Morrow
Quoth Jeremy Chadwick j...@koitsu.org:
 
 If there are people out there using, for example, SSDs on an ICH7 in
 non-AHCI mode, it would be good to know and get pciconf -lvbc output
 (specifically the entry for their ATA/SATA controller).  But as with all
 publicly released operating systems, most Requests For Feedback are
 ignored, things are then changed, and only afterwards do people crawl
 out of their hobbit holes and speak up.  :-)

Since you asked:

isab0@pci0:0:31:0:  class=0x060100 card=0x50011458 chip=0x27b88086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
device = '82801GB/GR (ICH7 Family) LPC Interface Bridge'
class  = bridge
subclass   = PCI-ISA
cap 09[e0] = vendor (length 12) Intel cap 1 version 0
 features: Quick Resume, SATA RAID-5, 6 PCI-e x1 slots, SATA 
RAID-0/1/10, SATA AHCI
atapci0@pci0:0:31:1:class=0x01018a card=0xb0011458 chip=0x27df8086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
device = '82801G (ICH7 Family) IDE Controller'
class  = mass storage
subclass   = ATA
bar   [20] = type I/O Port, range 32, base 0xf000, size 16, enabled
atapci1@pci0:0:31:2:class=0x01018f card=0xb0021458 chip=0x27c08086 rev=0x01 
hdr=0x00
vendor = 'Intel Corporation'
device = 'N10/ICH7 Family SATA IDE Controller'
class  = mass storage
subclass   = ATA
bar   [10] = type I/O Port, range 32, base 0xe800, size  8, enabled
bar   [14] = type I/O Port, range 32, base 0xe900, size  4, enabled
bar   [18] = type I/O Port, range 32, base 0xea00, size  8, enabled
bar   [1c] = type I/O Port, range 32, base 0xeb00, size  4, enabled
bar   [20] = type I/O Port, range 32, base 0xec00, size 16, enabled
cap 01[70] = powerspec 2  supports D0 D3  current D0

The motherboard manual claims it supports SATA 2 (3Gb/s). However, ada2
is an SSD, and camcontrol identify ada2 reports:

pass2: KINGSTON SVP200S37A60G 502ABBF0 ATA-8 SATA 3.x device
pass2: 150.000MB/s transfers (SATA, UDMA5, PIO 8192bytes)

protocol  ATA/ATAPI-8 SATA 3.x
device model  KINGSTON SVP200S37A60G
[...]

so FreeBSD doesn't appear to know that.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: svn - but smaller?

2013-02-24 Thread Ben Morrow
Quoth Jeremy Chadwick j...@koitsu.org:
 On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote:
  On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote:
   
   Also, John, please consider using malloc(3) instead of heap-allocated
   buffers like file_buffer[6][] (196608 bytes) and command[] (32769
   bytes).  I'm referring to this: 
  
  Why?  I absolutely do not understand why people are always objecting to
  large stack-allocated arrays in userland code (sometimes with the
  definition of large being as small as 2k for some folks).
 
 This is incredibly off-topic, but I'll bite.
 
 I should not have said heap-allocated, I was actually referring to
 statically-allocated.
 
 The issues as I see them:
 
 1. Such buffers exist during the entire program's lifetime even if they
 aren't actively used/needed by the program.  With malloc(3) and friends,
 you're allocating memory dynamically, and you can free(3) when done with
 it, rather than just having a gigantic portion of memory allocated
 sitting around potentially doing nothing.

If the 'allocated' memory isn't touched, it will never be paged in. Even
once it is paged in, if it then goes back to being unused it can be
paged out to swap (when necessary) and then stay there. (It would be
nice if there were some way to tell the system 'this memory is dead,
don't write it out to swap', but I don't think there is.)

Of course, you may get up to two pages of an object paged in just
because some other object on the same page was touched, but 'two pages'
is not a lot of memory to waste (especially given that you're saving the
malloc overhead). I don't know if the compiler is clever enough to
page-align large static allocations; that would reduce the potential
waste to one page.

 2. If the length of the buffer exceeds the amount of stack space
 available at the time, the program will run but the behaviour is unknown
 (I know that on FreeBSD if it exceeds stacksize per resource limits,
 the program segfaults at runtime).  With malloc and friends you can
 gracefully handle allocation failures.

(SIGSEGV on stack overflow is specified by POSIX, including the fact
that the signal must be fatal unless the signal handler uses an
alternate stack.)

Statically-allocated objects are not allocated on the stack, they are
allocated in the bss or data sections of the executable. When it is
possible and reasonable to do so, it's actually better to allocate all
memory statically, as then once the process has started successfully you
can be certain it won't fail because of a memory shortage.

Large auto objects (including alloca) *are* dangerous.

 3. Statically-allocated buffers can't grow; meaning what you've
 requested size-wise is all you get.  Compare this to something that's
 dynamic -- think a linked list containing pointers to malloc'd memory,
 which can even be realloc(3)'d if needed.

Under many circumstances a statically-allocated fixed-size buffer, *if
properly used*, is safer than a potentially-unbounded malloced one. Of
course, it's possible to put limits on the size of a malloced buffer as
well, but (given that parts of the buffer you don't use won't ever be
paged in) there isn't much advantage in doing so.

Fixed allocations are only useful in small, single-use programs where
you can accurately predict the memory requirements in advance. I
wouldn't be surprised if svnsup fell into that bracket.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Poudriere questions

2013-02-21 Thread Ben Morrow
Quoth Patrick M. Hausen hau...@punkt.de:
 
 OK, tried manually wihtout Poudriere:
 
 cd /usr/ports/www/apache22
 make deinstall
 rm -r /var/db/ports/apache22
 make clean
 make -DBATCH -DPROXY=on -DPROXY_HTTP=on -DSUEXEC=on
 -DSUEXEC_DOCROOT=/var/apache
 -DSUEXEC_LOGFILE=/var/apache/GLOBAL/suexec_log install

You need to read man make. Make's -D option doesn't work the same as
cc's; this defines variables called PROXY=on and so on to the value 1.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: svn - but smaller?

2013-01-24 Thread Ben Morrow
Quoth 'Jeremy Chadwick' j...@koitsu.org:
 
 Regarding your svn-lite theory of having that added to src/contrib/,
 let me introduce you to Subversion's actual dependencies, and I'll
 explain why these would have to remain enabled (for a base system
 Subversion) as well:
 
 * SQLite3 (used for bits/pieces in .svn/ directories)
   -- License: http://www.sqlite.org/copyright.html
   -- Not in the base system

It is in HEAD, because the new Heimdal needs it. (The library is
currently called libheimsqlite, and is built under kerberos5/lib, but
both of those could be changed if necessary.)

 * APR (used for HTTP fetching (not necessarily HTTPS))
   -- License: http://www.apache.org/licenses/LICENSE-2.0.html
   -- Not in the base system
 
 * Expat 2.x (XML parsing/generation library
   -- License: http://en.wikipedia.org/wiki/MIT_License
   -- Not in the base system

So these could potentially be brought in? I'm not sure if the Apache
license is considered sufficiently BSDish?

 * Neon or Serf (used for HTTPS fetching)

This could be considered optional, for a base-system svn, as long as the
repository is available over HTTP. If necessary the binary could be
renamed to bsdsvn so as not to conflict with a full-featured svn
installed from ports. (Of course, HTTPS would be *nice*.)

 * gettext and libintl (used for character set support)
   -- gettext license: GPL (not sure what version)
   -- libintl license: LGPL (not sure what version)
   -- Neither are in the base system

Don't know about these, but I'd be surprised if it wasn't possible to
build svn without them. That means losing i18n, but I'm fairly sure csup
didn't have i18n either.

 * libiconv (used for character set conversion)
   -- License: LGPL (not sure what version)
   -- Not in the base system

There is a BSD-licensed libiconv in HEAD, though it currently isn't
built by default.

So, it looks to me as though it would be possible, in principle, to
bring svn into the base. I'm not at all sure I think that would be a
good idea (in particular, bringing in both APR and Expat just to
download a few files seems excessive), but it could perhaps be provided
as a make.conf option for those who are concerned.

(Personally I would not willingly use svn again for any reason, so I'm
keeping my source up-to-date using the github mirror. I wonder whether a
BSD-licensed checkout-only git client would be easier to write than
svnsup? I'm certain the wire protocol is more efficient.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: svn - but smaller?

2013-01-24 Thread Ben Morrow
At  9AM + on 24/01/13 you (Ben Morrow) wrote:
 Quoth 'Jeremy Chadwick' j...@koitsu.org:
  
  Regarding your svn-lite theory of having that added to src/contrib/,
  let me introduce you to Subversion's actual dependencies, and I'll
  explain why these would have to remain enabled (for a base system
  Subversion) as well:
snip
  * APR (used for HTTP fetching (not necessarily HTTPS))
-- License: http://www.apache.org/licenses/LICENSE-2.0.html
-- Not in the base system
  
  * Expat 2.x (XML parsing/generation library
-- License: http://en.wikipedia.org/wiki/MIT_License
-- Not in the base system

Correction: expat is in base already, as libbsdxml (rather confusingly
built under lib/libexpat).

So AFAICS the only remaining piece is APR (and svn itself), and I
suspect that if only the bits required for a svn client were brought in
(assuming the licence is deemed acceptable) that would be a lot smaller
than a full APR build. (Again, this would need to be built as libbsdapr
to avoid conflicts with real APR from ports.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: freebsd-update IDS

2013-01-17 Thread Ben Morrow
Quoth Mark Felder f...@feld.me:
 On Thu, 17 Jan 2013 07:22:26 -0600, Alex Povolotsky  
 tark...@webmail.sub.ru wrote:
 
  It was a break-in. Some dumb php script running with user privileges  
  managed FreeBSD to hang on disk io up to stopping responding to anything  
  besides reset.
 
 Yikes! Make sure to run freebsd-update IDS to check the base OS's  
 checksums and if you're using pkgng you can use pkg check-s to look for  
 any tampered with files owned by packages.

Make sure you read the caveats in the freebsd-update manpage before
trusting the IDS result. At the very least you need to delete
/var/db/freebsd-update, /etc/freebsd-update.conf and
/usr/sbin/freebsd-update itself and replace them with known-good copies.

Ideally you should run the tests from an entirely separate known-good
instance of the OS, though in practice it's probably easier to just
replace the OS and packages from known-good sources and then set about
recovering and verifying the data. cf. the story about patching cc to
patch cc to patch login...

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Failsafe on kernel panic

2013-01-16 Thread Ben Morrow
Quoth Sami Halabi sodyn...@gmail.com:
 בתאריך 17 בינו 2013 05:18, מאת Ian Lepore i...@freebsd.org:
 
  From src/sys/conf/NOTES, this may be what you're looking for...
 
  #
  # Don't enter the debugger for a panic. Intended for unattended operation
  # where you may want to enter the debugger from the console, but still want
  # the machine to recover from a panic.
  #
  options KDB_UNATTENDED
 
  But I think it only has meaning if you have option KDB in effect,
  otherwise it should just reboot itself after a 15 second pause.

 Its only a kernel option? There is no flag to pass to the loader?

~% sysctl -ad | grep panic
kern.stop_scheduler_on_panic: stop scheduler upon entering panic
kern.sync_on_panic: Do a sync before rebooting from a panic
debug.trace_on_panic: Print stack trace on kernel panic
debug.debugger_on_panic: Run debugger on kernel panic
debug.kdb.panic: set to panic the kernel
machdep.enable_panic_key: Enable panic via keypress specified in kbdmap(5)
machdep.panic_on_nmi: Panic on NMI

These are also tunables which can be set in /boot/loader.conf or at the
loader prompt.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: CLANG 3.2 breaks security/pam_ssh_agent_auth on stable/9

2013-01-16 Thread Ben Morrow
Quoth Kimmo Paasiala kpaas...@gmail.com:
 
 Doesn't the change to strnvis() break the ABI on FreeBSD 9.X? I
 thought you could always compile a binary on an earlier version of
 FreeBSD 9.X and trust it to work without recompiling on any later
 minor version of the same major version line.

No, it doesn't. No existing prototypes are changed, there are just a
number of *nvis* functions added to complement the existing ones that
didn't have length arguments. The only problem is that the port assumes
that if a system has strnvis, it has a prototype matching OpenBSD's,
which the new one doesn't.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: IPv6 Tunnel Shared With Jails via epair Devices

2013-01-15 Thread Ben Morrow
Quoth Shawn Webb latt...@gmail.com:
 On Tue, Jan 15, 2013 at 12:29 AM, Ben Morrow b...@morrow.me.uk wrote:
  Quoth Shawn Webb latt...@gmail.com:
  
   # ifconfig bridge0
   bridge0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
   1500
   ether 02:fe:21:34:d3:00
   inet6 2001:470:8142:1::1 prefixlen 64
   nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
   id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
   maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
   root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
   member: epair0a flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
  ifmaxaddr 0 port 19 priority 128 path cost 2000
   member: epair1a flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
  ifmaxaddr 0 port 21 priority 128 path cost 2000
   member: bge0 flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
  ifmaxaddr 0 port 5 priority 128 path cost 20
 
  Why have you added the physical interface to the bridge? AFAICT you
  don't need to: a bridge will bridge epairs just fine, and as you
  explained in that blog post you have to route rather than bridge into
  the tunnel, since the tunnel isn't an Ethernet device.
 
 I did it so that I have an IPv4 address directly on the LAN for each of my
 jails.

Hmm, OK. 

   # jexec Dev Template ifconfig epair0b
   epair0b: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
   1500
   options=8VLAN_MTU
   ether 02:80:03:00:14:0b
   inet6 2001:470:8142:1::5 prefixlen 64 tentative
   inet6 fe80::80:3ff:fe00:140b%epair0b prefixlen 64 tentative scopeid 0x2
   inet 10.7.1.92 netmask 0xfe00 broadcast 10.7.1.255
   nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
 
  I suspect the addresses are only marked tentative because the interface
  has been marked IFDISABLED. This causes all current addresses to be
  marked tentative, because the kernel isn't allowed to send or receive
  IPv6 packets and so can't defend the addresses any more.
 
  Is it possible something in the jail's startup scripts is causing the
  interface to be marked IFDISABLED after the inet6 address has been
  assigned? Some of the functions in network.subr mark interfaces
  IFDISABLED automatically if they don't think they have IPv6 addresses.
 
 I was thinking the same thing. One problem is that I can't remove the
 IFDISABLED flag. This is what happens when I try:
 
 # jexec Dev Template ifconfig epair0b -ifdisabled
 ifconfig: ioctl(SIOCGIFINFO_IN6): Invalid argument

ifconfig epair0b inet6 -ifdisabled

I don't know why you get that error when you miss out the 'inet6'; it's
not exactly very clear.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: IPv6 Tunnel Shared With Jails via epair Devices

2013-01-15 Thread Ben Morrow
At  5PM -0500 on 15/01/13 you (Shawn Webb) wrote:
 
 I figured it out. In my jail initialization scripts, I'm running '/bin/sh
 /bin/rc' after doing initial network setup. The rc script puts the
 interface in IFDISABLED mode. So if I run the ifconfig command to remove
 the flag, I'm golden.

Yes, that's what I thought. You should be able to avoid this by
specifying either

ifconfig_epair0b_ipv6=inet6 auto_linklocal

or

ipv6_activate_all_interfaces=YES

in the jail's rc.conf. This is cleaner than running ifconfig explicitly
outside the jail.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: IPv6 Tunnel Shared With Jails via epair Devices

2013-01-14 Thread Ben Morrow
Quoth Shawn Webb latt...@gmail.com:
 
 I've been working on sharing a 6in4 IPv6 tunnel (via a gif device) I have
 with Hurricane Electric (tunnelbroker.net) to my jails via epair devices.
 My setup is a bit unique in that the IPv6 tunnel is behind an OpenVPN
 connection. I've had varying degrees of success. I might have a bug to
 report, but I thought I'd post here to get input from people who know
 better than I do about these kinds of things.
 
 I have a bridge device (we'll call it bridge0) with a /64 IPv6 address
 (2001:470:8142:1::1). Each jail's epair[n]b device will get an IPv6 address
 in that same prefix. For example, one of my jails is 2001:470:8142:1::3.
 The default IPv6 gateway is the IPv6 address of bridge0.
 
 Giving one jail an IP address works fine. For each jail after that, the
 IPv6 address stays in tentative mode. FreeBSD gets stuck trying to use DAD
 to figure out if there's an address conflict. It never leaves tentative
 mode. This is the bug I'm working out.
 
 Here's bridge0's config:
 
 # ifconfig bridge0
 bridge0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
 1500
 ether 02:fe:21:34:d3:00
 inet6 2001:470:8142:1::1 prefixlen 64
 nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
 member: epair0a flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
ifmaxaddr 0 port 19 priority 128 path cost 2000
 member: epair1a flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
ifmaxaddr 0 port 21 priority 128 path cost 2000
 member: bge0 flags=143LEARNING,DISCOVER,AUTOEDGE,AUTOPTP
ifmaxaddr 0 port 5 priority 128 path cost 20

Why have you added the physical interface to the bridge? AFAICT you
don't need to: a bridge will bridge epairs just fine, and as you
explained in that blog post you have to route rather than bridge into
the tunnel, since the tunnel isn't an Ethernet device.

 Here's the relevant epair device for the jail whose IPv6 stack is working:
 
 # jexec ClamAV_Dev ifconfig epair1b
 epair1b: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
 1500
 options=8VLAN_MTU
 ether 02:fb:c0:00:16:0b
 inet6 2001:470:8142:1::3 prefixlen 64
 inet6 fe80::fb:c0ff:fe00:160b%epair1b prefixlen 64 scopeid 0x2
 inet 10.7.1.172 netmask 0xfe00 broadcast 10.7.1.255
 nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
 media: Ethernet 10Gbase-T (10Gbase-T full-duplex)
 status: active
 
 Here's the relevant epair device for the jail whose IPv6 stack isn't
 working:
 
 # jexec Dev Template ifconfig epair0b
 epair0b: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu
 1500
 options=8VLAN_MTU
 ether 02:80:03:00:14:0b
 inet6 2001:470:8142:1::5 prefixlen 64 tentative
 inet6 fe80::80:3ff:fe00:140b%epair0b prefixlen 64 tentative scopeid 0x2
 inet 10.7.1.92 netmask 0xfe00 broadcast 10.7.1.255
 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL

I suspect the addresses are only marked tentative because the interface
has been marked IFDISABLED. This causes all current addresses to be
marked tentative, because the kernel isn't allowed to send or receive
IPv6 packets and so can't defend the addresses any more.

Is it possible something in the jail's startup scripts is causing the
interface to be marked IFDISABLED after the inet6 address has been
assigned? Some of the functions in network.subr mark interfaces
IFDISABLED automatically if they don't think they have IPv6 addresses.

 media: Ethernet 10Gbase-T (10Gbase-T full-duplex)
 status: active
 
 I brought up the Dev Template jail after bringing up the ClamAV_Dev jail.
 If there's any other output you'd like to see, let me know. If you're
 confused about my setup, visit my blog post about the subject here:
 http://0xfeedface.org/blog/lattera/2013-01-12/tunneled-ipv6-freebsd-jails
 
 I'm curious to know if I've got a legit bug or if it's something I'm doing
 wrong. The one thing I haven't tried is setting up rtadvd on the bridge.
 That'd be kindof interesting, since my physical NIC is a member on the
 bridge. I'd rather not dish out IPv6 addresses for all devices on the
 network (a network with lots of devices I don't own or control).

As I said, I don't believe you need the physical interface on the
bridge, unless you have to for IPv4 (and you can't route or proxyarp
instead). However, before you can run rtadvd you will need to give the
bridge its proper link-local address, which probably also means locking
down its hardware address in rc.conf. Bridges don't get auto link-local
addresses, for reasons I've never entirely understood, and RAs have to
use ll addresses.

You will need to set up routing so that packets coming in through the
tunnel destined for the jails get routed out of the bridge, and packets
coming in on the bridge destined for the IPv6 Internet get routed out of
the tunnel. Probably that will have happened already, just by assigning
an inet6 

Re: Determining which process needs to be restarted after update

2013-01-12 Thread Ben Morrow
Quoth Derek Kulinski tak...@takeda.tk:
 
 I personally really like OpenSuSE command which is: zypper ps
 What it does is it lists all processes that have files opened that
 currently don't exist (i.e. link count is 0). This helps tremendously
 in determining which processes need to be restarted after an update.
 
 Is there something similar for FreeBSD? I was thinking of using
 lsof +L1, but on FreeBSD that command is not capable of displaying
 names of files that were deleted, many entries returned are for
 example processes that have open sockets. It also does not list names
 of the deleted/replaced files.
 
 Is there a tool that is capable to do such task, or maybe some
 additional options to lsof? I'm not too familiar with it myself.

procstat -fa, look for entries with 'v' in the 'T' column and '-' in the
'NAME' column (or get awk to look for you). You may also want to check
the 'V' column: see the manpage for the codes. This won't tell you what
the file used to be called before it was deleted: I don't think the
kernel keeps that information.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Determining which process needs to be restarted after update

2013-01-12 Thread Ben Morrow
Quoth Mateusz Guzik mjgu...@gmail.com:
 On Sat, Jan 12, 2013 at 11:29:14PM +, Ben Morrow wrote:
  Quoth Derek Kulinski tak...@takeda.tk:
   
   I personally really like OpenSuSE command which is: zypper ps
   What it does is it lists all processes that have files opened that
   currently don't exist (i.e. link count is 0). This helps tremendously
   in determining which processes need to be restarted after an update.
   
   Is there something similar for FreeBSD? I was thinking of using
   lsof +L1, but on FreeBSD that command is not capable of displaying
   names of files that were deleted, many entries returned are for
   example processes that have open sockets. It also does not list names
   of the deleted/replaced files.
   
   Is there a tool that is capable to do such task, or maybe some
   additional options to lsof? I'm not too familiar with it myself.
  
  procstat -fa, look for entries with 'v' in the 'T' column and '-' in the
  'NAME' column (or get awk to look for you). You may also want to check
  the 'V' column: see the manpage for the codes. This won't tell you what
  the file used to be called before it was deleted: I don't think the
  kernel keeps that information.
  
 
 This has at least 2 problems:
 - it will not show shared libraries (-v is required)

That's a good point.

 - it will report processes with open unlinked files, which is completely
   normal

Isn't that exactly what the OP wants to find? I agree that it happens
for reasons other than a software update, but I don't see how that can
be avoided. Presumably the SuSE tool mentioned would give the same false
positives.

 But even if we use -v, I don't think we can reliably distinguish
 regular unlinked file mapping from shared library mapping (for
 unlinked files we will get - as a name, just like in -f case). I didn't
 dig into this though.

A process currently running an unlinked shared library will have at
least one procstat -v entry with 'x' in the 'PRT' column, 'vn' in the
'TP' column and nothing in the 'PATH' column.

 Instead I would go upwards in package dependency tree and for each daemon
 check if it is running (should be doable without much hackery). Checking
 for all binaries may be more problematic.

Yes, something like that might be better, though that will also get a
lot of false positives.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: sendmail vs ipv6 broken after upgrade to 9.1

2013-01-09 Thread Ben Morrow
Quoth Hiroki Sato h...@freebsd.org:
 Gregory Shapiro gshap...@freebsd.org wrote
   in 20130108180920.gj36...@rugsucker.smi.sendmail.com:
 
 gs  How can I unstupid sendmail here?
 gs
 gs I don't think sendmail is being stupid here as it is doing what it has
 gs been doing under 8.x and 9.1 (the code is the same).  I think
 gs something changed with the upgrade to 9.1.  As far as tracking it
 gs down, the sendmail code does:
 gs
 gs getipnodebyname(acme.spoerlein.net, AF_INET6, AI_DEFAULT|AI_ALL,
 gs err);
 gs
 gs This will only return an IPv4 mapped address if:
 gs
 gs 1. There are no IPv6 addresses configured on the interfaces. snip
 gs
 gs 2. The query for an  record for acme.spoerlein.net failed.
 gs snip 

This is not quite right. 

AI_DEFAULT is AI_V4MAPPED | AI_ADDRCONFIG.

AI_V4MAPPED says 'if there are no  records, query for A records
and return them as v4-mapped addresses'. 

AI_ALL is only valid with AI_V4MAPPED, and says 'always query for A
records and return v4-mapped addresses'. 

AI_ADDRCONFIG says 'only query for  records if there is at least
one interface with an IPv6 address; only query for A records if
there is at least one interface with an IPv4 address'. (Loopback
explicitly doesn't count for this purpose.) 

The resulting list of addresses is sorted according to ip6addrctl.

So getipnodebyname is behaving correctly here: the host has both IPv4
and IPv6 addresses, and Sendmail is requesting both native and v4-mapped
addresses be returned in all cases. The v4-mapped addresses are then
sorted to the top of the list.

On FreeBSD, where net.inet6.ip6.v6only is on by default, I believe this
is incorrect, and Sendmail should be passing 0 for the flags argument,
unless it's going to check or clear the IPV6_V6ONLY socket option. There
is no point binding a socket to a v4-mapped address if the kernel isn't
going to deliver IPv4 connections to it. Sendmail should also be binding
to all the addresses returned, if it isn't already, rather than just the
first: this would make the problem go away, since both v4-mapped and
native IPv6 sockets would be bound, and the v4-mapped ones would simply
never get any connections.

Fixing this by setting ipv6_prefer is not necessarily a good idea; this
will cause IPv6 addresses to be preferred across the whole system, and
unless your IPv6 connectivity is at least as good as your IPv4, that
probably isn't what you want.

  Just curious, but is there any specific reason not to return an error
  when Family=inet6 and no  RR?

In this case, Sendmail explicitly requested that v4-mapped addresses be
returned in all cases...

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: sendmail vs ipv6 broken after upgrade to 9.1

2013-01-09 Thread Ben Morrow
Quoth Hajimu UMEMOTO u...@freebsd.org:
  On Wed, 09 Jan 2013 23:01:52 +0900
  Hajimu UMEMOTO u...@freebsd.org said:
 
 ume I changed getipnodebyname to obey ip6addrctl in years past.  I read
 ume RFC 2553 again, and realize that it mentions IPv6 addresses are
 ume returned 1st.  So, my past change might be bad thing. X-(

Where does it say that? All I can find (but I might be being stupid) is
the bit in the description of AI_ALL where it says 'A query is first
made for  records and if successful, the IPv6 addresses are
returned. Another query is then made for A records and any found are
returned as IPv4-mapped IPv6 addresses.'. I don't believe that is meant
to indicate the  results are returned first in the list, just that
both sets of results are included.

Also, RFC 6724 (which is more recent), says 'we intend that
implementations of APIs such as getaddrinfo() will use the destination
address selection algorithm specified here to sort the list of IPv6 and
IPv4 addresses that they return.'. AFAICS 'APIs such as getaddrinfo()'
is supposed to include getipnodebyname and gethostbyname2, and the whole
list of v4 and v6 addresses is supposed to be sorted by those rules.

However, given that FreeBSD disables the use of v4-mapped addresses on
AF_INET6 sockets by default, it might be sensible to change the rules a
little. An application making an AF_INET6 query is probably going to use
the result with an AF_INET6 socket, so a v4-mapped address is going to
be mostly useless.

 I've just committed to disable it:
 
 http://svnweb.freebsd.org/base?view=revisionrevision=245225

I don't think that's the right answer. Even if the code should be
changed to always return addresses from A records last, the IPv6
addresses from  records should still be sorted according to
ip6addrctl. Otherwise sites with multiple prefixes (say, a ULA prefix
and a global prefix) won't be able to control their use properly.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: sendmail vs ipv6 broken after upgrade to 9.1

2013-01-09 Thread Ben Morrow
Quoth Hajimu UMEMOTO u...@freebsd.org:
 Hi,
 
  On Wed, 9 Jan 2013 16:29:00 +
  Ben Morrow b...@morrow.me.uk said:
 
 ben Where does it say that? All I can find (but I might be being stupid) is
 ben the bit in the description of AI_ALL where it says 'A query is first
 ben made for  records and if successful, the IPv6 addresses are
 ben returned. Another query is then made for A records and any found are
 ben returned as IPv4-mapped IPv6 addresses.'. I don't believe that is meant
 ben to indicate the  results are returned first in the list, just that
 ben both sets of results are included.
 
 It is the sentence you mentioned.  It was not thought those days that
 a query order and an order of the value to return were another.  So, I
 think that it implies the order of the value to return.

OK. Since this is a legacy API, the real question is 'what do existing
applications calling it expect?'. You may be right that they will expect
to see IPv6 addresses first if there are any.

 ben Also, RFC 6724 (which is more recent), says 'we intend that
 ben implementations of APIs such as getaddrinfo() will use the destination
 ben address selection algorithm specified here to sort the list of IPv6 and
 ben IPv4 addresses that they return.'. AFAICS 'APIs such as getaddrinfo()'
 ben is supposed to include getipnodebyname and gethostbyname2, and the whole
 ben list of v4 and v6 addresses is supposed to be sorted by those rules.
 
 I thought so at the time when I implemented it.  However,
 getipnodebyname has IPv4-mapped addresses issue as compared with
 getaddrinfo.
 Since gethostbyname2 doesn't treat multiple families at once, it is
 out of scope, IMHO.  gethostbyname2 in FreeBSD doesn't obey
 ip6addrctl.

ip6addrctl does more than just order v4 vs v6: it also sorts the v6
addresses, in a way which can be quite important. IMHO both the v6
addresses returned from getipnodebyname and the addresses returned from
gethostbyname2(AF_INET6) ought to be sorted according to ip6addrctl,
even if getipnodebyname special-cases the v4-mapped addresses to appear
last.

 ben However, given that FreeBSD disables the use of v4-mapped addresses on
 ben AF_INET6 sockets by default, it might be sensible to change the rules a
 ben little. An application making an AF_INET6 query is probably going to use
 ben the result with an AF_INET6 socket, so a v4-mapped address is going to
 ben be mostly useless.
 
 There is IPV6_V6ONLY socket option.  Still, an application has two
 choices:
 - convert IPv4-mapped address to IPv4 address, or
 - issue IPV6_V6ONLY socket option.
 In anyway, I think it is important that an application conscious of
 using the IPv4-mapped address.

Yeah; I agree that the v4-mapped option is pretty useless (even when
using a stack which supports it). I suspect the IETF people were trying
too hard to account for the case of a v6-only stack talking to the v4
Internet, when AFAIK there aren't any v6-only stacks yet, nor are there
likely to be for the forseeable future. That's why I think Sendmail
ought to be changed to pass 0 flags, so it doesn't see v4-mapped
addresses at all: after all, there's little point binding separate v4
and v6 sockets if the v6 socket is just going to end up bound to a
v4-mapped address.

  I've just committed to disable it:
  
  http://svnweb.freebsd.org/base?view=revisionrevision=245225
 
 ben I don't think that's the right answer. Even if the code should be
 ben changed to always return addresses from A records last, the IPv6
 ben addresses from  records should still be sorted according to
 ben ip6addrctl. Otherwise sites with multiple prefixes (say, a ULA prefix
 ben and a global prefix) won't be able to control their use properly.
 
 getipnodebyname was deprecated by RFC 3493 and appropriate time has
 passed since then.  So, it is low-priority, IMHO.

Well, if important applications like Sendmail are still using it it's
probably important it works consistently with the rest of the system's
IPv6 support. I'd rather see it removed, or reimplemented as a wrapper
around getaddrinfo, than left in a broken state.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: ipv6_addrs_IF aliases in rc.conf(5)

2012-12-22 Thread Ben Morrow
Quoth =?UTF-8?B?xYF1a2FzeiBXxIVzaWtvd3NraQ==?= luk...@wasikowski.net:
 W dniu 2012-12-22 04:41, Kimmo Paasiala pisze:
 
  Yeah, this is problem in network.subr. An interface is not recognized
  as IPv6 capable if the interface is not in ipv6_network_interfaces
  and there's no ifconfig_IF_ipv6 in rc.conf(5), bummer. For IPv4 it
  just works because the interface is always assumed to be IPv4
  capable.
 
 Ok, I used ifconfig_em0_ipv6=up and it worked. So it looks like this:

The documented way to do this is to just set the link-local address in
ifconfig_IF_ipv6, since an interface is required to have a link-local
address. Either configure an fe80:: address explicitly or set

ifconfig_em0_ipv6=inet6 auto_linklocal

Alternatively, if you set ipv6_activate_all_interfaces all interfaces
will be considered IPv6-capable.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: portupgrade problem after upgrading to 9.1-(PRE)RELEASE

2012-12-19 Thread Ben Morrow
Quoth CeDeROM cede...@tlen.pl:
 
 I was using portupgrade/portinstall -PP on 9.1-RC3 with no problem.
 After freebsd-update -r 9.1-RELEASE I cannot install any port from
 packages. Should I additionally configure my system somehow? Where did
 the portupgrade took the packages from last time / before upgrade?

9.1-RELEASE isn't out yet. That means the packages may not have been
built yet, and even if they have, they may not be right. Wait for the
release announcement.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: 9.1-RC3 LiveCD missing features

2012-12-07 Thread Ben Morrow
Quoth CeDeROM cede...@tlen.pl:
 
 I have tried to chceck for badblocks on my / but I did not find badblocks
 program on LiveCD and there is no option to install it. This is very useful
 utility, please add it as part of LiveCD :-)

There is no badblocks utility in the FreeBSD base system. It wouldn't be
much use with modern disks in any case, since they do automatic
bad-block mapping internally and once any bad blocks become visible the
disk is failing and you will likely see lots more very soon. 

The Linux utility is available as part of the sysutils/e2fsprogs port.
If you just want to test a disk there are examples in the dd(1) manpage.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: How to clean up /

2012-11-27 Thread Ben Morrow
Quoth =?utf-8?Q?Efra=C3=ADn_D=C3=A9ctor?= efraindec...@motumweb.com:
 Hello. 
 
 I recently upgraded to 9.1-RC3, everything went fine, however the /
 partition its about to get full. Im really new to FreeBSD so I don’t
 know what files can be deleted safely.
 
 76M/compat/linux/usr/lib/locale/locale-archive.tmpl

This shouldn't be in /. You should create a new filesystem for /compat
alongside /usr and /var, or just make a symlink /compat - /usr/compat
and move everything there.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: How to clean up /

2012-11-27 Thread Ben Morrow
At  7PM -0600 on 27/11/12 Efraín Déctor wrote:
 From: Ben Morrow
 Quoth =?utf-8?Q?Efra=C3=ADn_D=C3=A9ctor?= efraindec...@motumweb.com:
 
  I recently upgraded to 9.1-RC3, everything went fine, however the /
  partition its about to get full. Im really new to FreeBSD so I don’t
  know what files can be deleted safely.
 
  76M/compat/linux/usr/lib/locale/locale-archive.tmpl
 
 This shouldn't be in /. You should create a new filesystem for /compat
 alongside /usr and /var, or just make a symlink /compat - /usr/compat
 and move everything there.

 I have a custom kernel on this system (for using IPFW) could be this reason 
 so compat is there?.

[Do you realise you don't need a custom kernel for ipfw? ipfw is
available as a module, so you can keep GENERIC and put

ipfw_load=YES

into /boot/loader.conf. This will cause ipfw to be loaded at boot time,
before the network is started; you can load it at runtime by running

kldload ipfw

]

No, this is nothing to do with a custom kernel: it is part of one of the
linux_base ports, which you must have installed. It needs to be where it
is, under /compat/linux, but /compat/linux itself should not be on the
root filesystem.

Since the list of large files you posted is relatively short, I assume
you are using the traditional partioning scheme, with a small root
partition and larger partitions for /usr and /var (and maybe /home).
There is not as much point to this as there used to be, but if you are
going to use this scheme you need to avoid putting anything in the root
partition that doesn't need to be there. 

/compat/linux contains libraries, binaries and other files needed for
FreeBSD's Linux API emulation; this most certainly does *not* need to be
in the root partition, so it should be moved somewhere else. There are
two possible 'somewhere else's: you can put it in its own partition,
which means you need to have space on your disk to allocate a new
partition; or you can move the whole lot onto one of your other existing
partitions, and use a symlink (or a nullfs mount, but let's not get
complicated) to make the directory appear where it's supposed to be.

Probably the latter is the easier option. /usr is the right filesystem
to be storing this sort of thing on, so you want to

mv /compat /usr
ln -s usr/compat /compat

The mv will take some time, since it is moving files cross-filesystem so
it has to copy them all.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: confirm that csup is still usable fos the new 9.1

2012-11-26 Thread Ben Morrow
Quoth Chris Rees utis...@gmail.com:
 On 26 Nov 2012 08:12, Perry Hutchison per...@pluto.rain.com wrote:
  Kevin Oberman kob6...@gmail.com wrote:
 
   ... don't bet that csup and cvs will be around long ...
   It's really time to get away from CVS and I suspect
   it will be going away sooner than had been planned.
 
  Once csup goes away, how will a base-only system update
  the sources, e.g. to follow a security branch?
 
 freebsd-update will update your sources for you.

It would be convenient if, before csup disappears, the 'Sychronising
your Source' section in the Handbook could be updated to contain
instructions for updating *just the source* using freebsd-update. Even
if this is as simple as 'Components src' it would be good to state that
explicitly; it would also be useful to give a procedure for moving from
a csupped tree to one which freebsd-update will be able to apply deltas
to, if that's possible without redownloading the whole source tree.

In general, it's not terribly clear to me what will happen if I run
freebsd-update on a system I have built from source. How does it know
where to start from when it downloads deltas: does it assume `uname -r`
accurately reflects the exact current state of the system? What will
happen if I patch my source and then run freebsd-update without
reverting those changes: will it revert them for me, like csup did, will
it make a mess, or will the update fail?

It would also be better, IMHO, to change the language in that section to
recommend freebsd-update for those running RELEASE branches, and reserve
the svn recommendation for those tracking development branches.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: FreeBSD 10-CURRENT and 9-STABLE snapshots

2012-10-10 Thread Ben Morrow
Quoth Brandon Allbery allber...@gmail.com:
 On Wed, Oct 10, 2012 at 8:21 PM, Claude Buisson clbuis...@orange.fr wrote:
 
  I can easily understand that developpers are happy to switch to new tools,
  but
  the question is: do FreeBSD developpers care a bit about non developpers,
  and
 
 FreeBSD developers are required to keep their development systems and tools
 with the needs of non-developers as the primary requirement?

We're not talking about the systems used for development, but the
systems used for distribution of the OS to users. Development moved away
from CVS a long time ago, but the mirrors were kept so that users could
keep using csup. If 'you' (FSVO) want to drop the mirrors, for obvious
reasons, it would be helpful to the rest of us if some alternative could
be provided which is functionally equivalent.

I am given to understand, but have not yet tested, that freebsd-update
can be used to update just the 'src' component. If this works reliably
(and doesn't mess anything else up in a built-from-source system), it
seems like a good answer to me, at least for those tracking -RELEASE
branches rather than -STABLE. One solution to this dilemma might be to
just document that as the officially-supported way to obtain the source.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: FreeBSD 9.1-RC1 Available...

2012-08-24 Thread Ben Morrow
Quoth Ken Smith kensm...@buffalo.edu:
 
 The latter.  If you are not using FreeBSD-Update to handle the updates
 of a machine you'll need to update your source tree using SVN for
 release branches (releng/*) from now on.  Updates of the CVS repository
 will continue for the existing stable/* and head for now.  I don't think
 anything has been decided on when that will stop.

Two questions:

1. Is is sensible|supported to use freebsd-update to update just the src
component, followed by a normal buildworld/buildkernel to update the
rest of the system? I would much prefer to avoid having to use svn,
especially given that it isn't in the base system.

2. If I have patched my source tree, what will freebsd-update do? csup
resets the patched files to the versions in the new tree, which is
satisfactory (I can reapply any patches afterwards, adjusting them if
necessary), but if freebsd-update leaves a mixture of new-and-unpatched
and old-and-patched files in the tree that could be a problem. (Not an
insoluble problem, of course, especially with ZFS snapshots.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: Multi node storage, ZFS

2010-03-24 Thread Ben Morrow
Quoth Michal mic...@ionic.co.uk:
 
 I do aswell :D The thing is, I see it two ways; I worked for a a huge
 online betting company, and we had the money for HP MSA's and big
 expensive SAN's, then we have a lot of SMB's with no where near the
 budget for that but the same problem with lots of data and the need for
 backend storage for databases. It's all well and good having 1 ZFS
 server, but it's fragile in the the sense of no redundancy, then we have
 1 ZFS server and a 2nd with DRBD, but that's a waste of money...think 12
 TB, and you need to pay for another 12TB box for redundancy, and you are
 still looking at 1 server. I am thinking a cheap solution but one that
 has IO throughput, redundancy and is easy to manange and expand across
 multiple nodes

If you do it right, you could have the 'SAN' box be one of the boxes
full of discs, with some or all of the others able to take over the
'SAN' role if it fails. That way you get redundancy without having to
have a machine sit idle. (You're still using more discs than you
strictly need to hold that much data, of course, but you can't avoid
that.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-16 Thread Ben Morrow
Quoth sth...@nethelp.no:
  If you installed dangerously dedicated and ended up
  with ad0s1a (note the s1), then you have an invalid
  partitioning and FreeBSD 8.x will not give you what
  you've been getting on FreeBSD 7.x. Most of the time
  you only need to wipe out the second sector on the
  disk to clean it up and have FreeBSD 8.x also give
  you ad0s1a.
 
 So what's an easy recipe we can run on 7.x hosts to see whether we
 would have problems with 8.x?

From what's been said so far: If you have adXsY devices in 7, *and* 

bsdlabel adX

finds a valid label (*note*: that is the whole disk, not the slice),
then you have conflicting BSD and MBR labels at the start of the disk
and you will have a problem in 8.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-16 Thread Ben Morrow
Quoth Marcel Moolenaar xcl...@mac.com:
 
 On Dec 16, 2009, at 3:27 AM, Marian Hettwer wrote:
  
  gee, thanks!
  That worked.
  r...@talisker:/root# ls /dev/ad8*
  /dev/ad8/dev/ad8s1  /dev/ad8s1a
  r...@talisker:/root# mount /dev/ad8s1a /BACKUP/
  r...@talisker:/root# umount /BACKUP/ 
  
  but, hm, whats that?
  r...@talisker:/root# fsck /dev/ad8s1a
  fsck: Could not determine filesystem type
 
 I minor inconvenience for the time being. fsck hasn't
 been thought about getting partition types from gpart,
 so that it can do a best-effort attempt at guessing
 which variant to run. For now, you need to be explicit:

I for one would rather *not* see fsck extended in this way. The whole
point of fsck is that you're running it on a potentially-broken disk;
guessing is not helpful at that point :).

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-16 Thread Ben Morrow
Quoth sth...@nethelp.no:
   
   So what's an easy recipe we can run on 7.x hosts to see whether we
   would have problems with 8.x?
  
  From what's been said so far: If you have adXsY devices in 7, *and* 
  
  bsdlabel adX
  
  finds a valid label (*note*: that is the whole disk, not the slice),
  then you have conflicting BSD and MBR labels at the start of the disk
  and you will have a problem in 8.
 
 So presumably if I have root on ad4s1a today, and bsdlabel shows
 
 # bsdlabel ad4
 bsdlabel: /dev/ad4: no valid label found
 
 then I am ready for FreeBSD 8.x?

I think so (but I'm no expert). This setup isn't in any way 'dangerously
dedicated', though, so there's no reason to think it wouldn't work. My
question was whether mounting from ad2d (with no MBR on ad2 at all)
would work; apparently it will, as long as there isn't an invalid BSD
disklabel in sector 2 of the first track.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-15 Thread Ben Morrow
Quoth Marcel Moolenaar xcl...@mac.com:
 On Dec 15, 2009, at 2:20 PM, Steven Friedrich wrote:
  
  FreeBSD 8.0 no longer supports dangerously dedicated disks.
 
 This is not true. The problem is that sysinstall creates an invalid
 dangerously dedicated disk, as demonstrated by doing:
   # fdisk ad8
   (shows FreeBSD slice information)
 
   # bsdlabel ad8
   (shows valid but empty disk label)
 
 Marian just needs to wipe out the second sector on the disk to
 remove the BSD disklabel that prevents the kernel from using
 the master boot record in the 1st sector. This exposes ad8s1.
 This then will pick up the BSD disklabel in sector 65 (i.e.
 the second sector in slice 1) to give ad8s1a...

Are you able to clarify exactly what is no longer working in 8? I've
read things here and there about dangerously dedicated disks no longer
being supported, but no detail about what exactly had changed. You seem
to be implying here that there is only a problem if there are invalid
and/or overlapping labels on the disk; elsewhere I have read that disks
without an MBR aren't supported at all (I presume the faked-up MBR on a
GPT disk counts). If I currently have a working ad2{b,c,d,e}, will they
be picked up by 8, or would I have to repartition slightly smaller with
a useless MBR slice in front?

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-15 Thread Ben Morrow
Quoth Xin LI delp...@gmail.com:
 On Tue, Dec 15, 2009 at 4:08 PM, Ben Morrow b...@morrow.me.uk wrote:
 
  Are you able to clarify exactly what is no longer working in 8? I've
  read things here and there about dangerously dedicated disks no longer
  being supported, but no detail about what exactly had changed. You seem
  to be implying here that there is only a problem if there are invalid
  and/or overlapping labels on the disk; elsewhere I have read that disks
  without an MBR aren't supported at all (I presume the faked-up MBR on a
  GPT disk counts). If I currently have a working ad2{b,c,d,e}, will they
  be picked up by 8, or would I have to repartition slightly smaller with
  a useless MBR slice in front?
 
 My $0.02: what about labelling them, say, tunefs -L on UFS partitions,
 and glabel for swap, then change corresponding entry in fstab.  Say:
snip

Yes, I've already done that. However, if gpart doesn't pick up the
underlying ad2d partition glabel won't know to look for a label, so the
entry in /dev/ufs won't show up either.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: update to 8.0-RELEASE -- partition gone

2009-12-15 Thread Ben Morrow
Quoth Marcel Moolenaar xcl...@mac.com:
 On Dec 15, 2009, at 4:08 PM, Ben Morrow wrote:
  Quoth Marcel Moolenaar xcl...@mac.com:
  On Dec 15, 2009, at 2:20 PM, Steven Friedrich wrote:
  
  FreeBSD 8.0 no longer supports dangerously dedicated disks.
  
  This is not true. The problem is that sysinstall creates an invalid
  dangerously dedicated disk, as demonstrated by doing:
snip
  
  Are you able to clarify exactly what is no longer working in 8?
 
 Everything is working, but behaviour has changed for invalid
 disks.

Right. However, if I were to take a system that worked with 7.x and
upgrade, and found that the disks were no longer detected, I would
consider that to be 'not working' :).

 Invalid disks are disks with conflicting partitioning
 information. In FreeBSD 8.x the behaviour is deterministic
 and for the broken dangerously dedicated disks that sysinstall
 creates this means that we use the partition information in
 the BSD disklabel. In FreeBSD 7.x this could come from either
 the MBR or the BSD disklabel, with the MBR the more common
 scenario.

OK, so this is all actually about a bug in sysinstall. It might be nice
if the UPDATING entry mentioned this: as it stands it is not clear this
doesn't affect people who created proper disklabels by hand (including
the obligatory dd to wipe out old MBR labels before starting).

  If I currently have a working ad2{b,c,d,e}, will they
  be picked up by 8, or would I have to repartition slightly smaller with
  a useless MBR slice in front?
 
 Yes, if you have ad2a and not ad2s1a, then you have a
 proper dangerously dedicated disk and FreeBSD 8.x will
 work correctly with your disk.

Thank you for explaining.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: RELENG_8 buildworld broken?

2009-12-09 Thread Ben Morrow
Quoth Roland Smith rsm...@xs4all.nl:
 On Wed, Dec 09, 2009 at 01:09:12PM -0800, Jeremy Chadwick wrote:
 
  Basically, all this comes back to the same thing: the entire base
  system concept needs to be revisited (that's a nice way of saying
  nuked from orbit, but that's my opinion). 
 
 Hmm, I kind of like having a usable base system as opposed to just a
 kernel. :-) It is one of the strong points of FreeBSD, IMHO. The fact that the
 base system is developed as a unit keeps it working very well.

Yes. At a bare minimum, the base system should contain enough to 'make
install' a typical port, including everything required to get the
network running. Application programs like sendmail and bind could
easily be moved out into ports (just as X11 was), but the core libraries
and the dev environment should be distributed as a unit. One of the
reasons I switched my home machine to FreeBSD was to get away from the
nightmare of trying to upgrade glibc in Gentoo. I don't believe I ever
got it to run right through without it falling over and having to go in
and build some package by hand: the recursive dependancies (portage
depends on python depends on glibc depends on a newer version of
portage... ) just got too complicated.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: status of flash9/flash10 support in RELENG_7 ?

2009-08-10 Thread Ben Morrow
Quoth Harald ha...@free.fr:
 On Sun, Aug 09, 2009 at 11:04:52PM +0100, Ben Morrow wrote:
  
  I was about to say 'I believe the vuxml entry for firefox is incorrect',
  but I see it's been fixed. Neither 3.0.13 nor 3.5.2 are vulnerable, and
  vuxml now correctly reports this.
 
 Today security/vuxml/vuln.xml says:
 
 affects
   package
 namefirefox/name
 namelinux-firefox/name
 rangelt3.*,1/lt/range
 rangegt3.*,1/gtlt3.0.13,1/lt/range
 rangegt3.5.*,1/gtlt3.5.2,1/lt/range
   /package
 
 1. Could someone tell me the meaning of the ``*'' values please ?
 I can't see the logic of the range lines.

3.* is the lowest possible version starting with '3.': in particular,
it's less than 3.0 and less than 3.a . So the lt3.*,1/lt will match
anything less than firefox3. The next two lines deal with the specifics
of which firefox3 versions are vulnerable.

 2. Yesterday I installed firefox quickly with ``pkg_add -r firefox3''
 and got firefox-3.0.10,1.
 Portaudit declares it vulnerable which seems to correspond
 to the second range line.
 I guess I have to compile firefox3 to be clean ?

3.0.10,1 is vulnerable, yes. If there aren't packages for 3.0.13,1 yet
you will need to compile it yourself.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: status of flash9/flash10 support in RELENG_7 ?

2009-08-09 Thread Ben Morrow
Quoth Harald Weis ha...@free.fr:
 
 There is a huge problem though:
 I've got now two vulnerable ports, firefox3 and linux-pango.
 The linux-pango case is apparently several months old.
 Any idea why the linux world doesn't seem to bother?
 
 How to persuade my user now not to use firefox, but w3m?
 Impossible.

I was about to say 'I believe the vuxml entry for firefox is incorrect',
but I see it's been fixed. Neither 3.0.13 nor 3.5.2 are vulnerable, and
vuxml now correctly reports this.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: portmaster -R (Was: Re: HEADS-UP: Shared Library Versions bumped...)

2009-07-31 Thread Ben Morrow
Quoth Doug Barton do...@freebsd.org:
 
 How about this? When the user has -[rf] but not -R, and there are flag
 files present, ask if they should be cleared before beginning to do
 anything. Otherwise (no -[rf]) ignore them. Sound good?

Since my machine has spent the last 48hrs or so rebuilding everything
that depended on jpeg-6b and python25 (it's a pretty old machine), I've
been wondering if an option to say '*don't* rewrite the dependencies of
other ports to refer to the new version' would be a good solution here.

Normally this is a helpful thing to do, but when you're trying to
reinstall a few ports low in the dependency chain and then rebuild
everything that needs rebuilding it would be helpful to have the ones
that haven't been rebuilt still depend on the old (now deleted) package,
so they can be identified.

-r (and -Rr) don't help here, since lots of large ports depend on *both*
jpeg and python, and I was specifically trying to avoid rebuilding them
all twice. AFAICT -r doesn't allow you to ask for two ports plus all
combined dependants at once. I ended up taking the pkg_info -R list for
both pkgs before the upgrade, sorting it into dependency order, and
stripping entries off the front every time something failed and I had to
restart, which is a little too manual for my taste :). (The list had to
be sorted, otherwise port A might depend on port B that came later in
the list, and when portmaster got to B in the list it would reinstall it
again unnecessarily.)

It also occurs to me that this could be completely automated if ports
had a LAST_COMPATIBLE_VER field or some such, which tells you whether or
not an upgrade-in-place is safe. portmaster could even ask 'you've just
upgraded from foo-1.2 to foo-2.1 which is not compatible: do you want to
rebuild list of pkgs which depended on the old version?'.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: portmaster -s text (Was: Re: HEADS-UP: Shared Library Versions bumped)

2009-07-28 Thread Ben Morrow

Doug Barton wrote:

On Thu, 23 Jul 2009, Ben Morrow wrote:


The problem with that is if you install pkg A deliberately, but it then
later becomes a dependancy of pkg B. If you remove pkg B (because it's
no longer needed) there is then no evidence that pkg A was installed on
purpose, rather than incidentally. portmaster -s will offer to remove
it, and if you refuse it will offer to remove the empty +REQUIRED_BY,
effectively promoting it to a 'manually installed' pkg again, though
it's perhaps not entirely clear from the question that that is what the
effect will be.


Thanks for pointing this out. Can you suggest an alternative message? 
Other than the mundane reason the current message says what it does 
because I sometimes prefer to leave the empty file there so that when I 
go back through at a later date I can re-evaluate the choice.


Yes, I can see it needs to be an option. Presumably 'don't ask me again' 
is too Microsoft :)? Maybe something like 'Mark this package as 
explicitly required?'? That's pretty much the user-visible effect.


I know the first time I saw that message, when I didn't really 
understand what it meant, my reaction was 'Delete something? No!' which 
wasn't the right answer.



This would be easy to solve in general by maintaining a 'world' package,
or some such, that had dependencies on everything installed explicitly;
but that would require modifying all the pkg and port installation tools
(probably including bsd.port.mk itself) to support that convention.


This sort of mechanism has been suggested before, but the problem you 
described (ports installed on purpose becoming a dependency of 
something else) is not an easy one to solve.


No. I have occasionally wondered if a sensible solution would be to drop 
portmaster/portupgrade altogether and just maintain a local 
sysutils/world port with a list of what I want installed, and 'make 
deinstall reinstall' whenever it changes (with something like 
pkg_cutleaves to clear out the trash). I suspect I would lose something 
I'm currently relying on (certainly, portmaster's -o and -r options 
wouldn't be trivial to emulate) but it does seem to me that both 
portmaster and portupgrade spend an awful lot of time doing things like 
tracking dependancies that bsd.port.mk already does for you.


Ben
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: HEADS-UP: Shared Library Versions bumped...

2009-07-22 Thread Ben Morrow
Quoth Andrew Reilly andrew-free...@areilly.bpc-users.org:
 On Wed, Jul 22, 2009 at 03:30:33PM -0400, David Schultz wrote:
  I do the same thing.  I wish the ports system kept track of which
  ports were installed explicitly by me, and which were only
  dependencies.  Then it would be possible to garbage collect ports
  that are no longer needed.
 
 Portmaster -s will cull ports that are no longer depended on.
 Seems to work fairly well.
 
 You can (probably) find the list of ports that you installed
 deliberately by trawling through /var/db/pkg for those that
 don't have a +REQUIRED_BY file.  (The portmaster -s trick works
 by finding ports that have a +REQUIRED_BY file that is empty, I
 believe.)

The problem with that is if you install pkg A deliberately, but it then
later becomes a dependancy of pkg B. If you remove pkg B (because it's
no longer needed) there is then no evidence that pkg A was installed on
purpose, rather than incidentally. portmaster -s will offer to remove
it, and if you refuse it will offer to remove the empty +REQUIRED_BY,
effectively promoting it to a 'manually installed' pkg again, though
it's perhaps not entirely clear from the question that that is what the
effect will be.

This would be easy to solve in general by maintaining a 'world' package,
or some such, that had dependencies on everything installed explicitly;
but that would require modifying all the pkg and port installation tools
(probably including bsd.port.mk itself) to support that convention.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: How to get djbdns to start early enough to satisfy ntpd at boot?

2009-01-16 Thread Ben Morrow
Quoth Andrew Reilly andrew-free...@areilly.bpc-users.org:
 On Thu, Jan 15, 2009 at 12:00:01PM +, Ben Morrow wrote:
  
  and I also have a /usr/local/etc/rc.d/dnscache which looks like
 [snip]
  
  and a /usr/local/etc/svc.subr as attached. It's somewhat more general
  than is needed for dnscache, as I also use it to start qmail.
 
 Wow, those scripts are an extremely cool integration of the
 rcorder framework and the svscan framework.  Could these be
 incorporated into the daemontools and djbdns ports as permanent
 goodness?

Well, it's hardly up to me :). I was wondering about submitting a patch
to the appropriate ports; if you want to preempt me, feel free :). (The
files would presumably need appropriate copyright and license lines
added, of course.)

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: How to get djbdns to start early enough to satisfy ntpd at boot?

2009-01-15 Thread Ben Morrow
Quoth Andrew Reilly andrew-free...@areilly.bpc-users.org:
 
 So: does anyone know how to modify the boot-time order so that
 svscan starts at (or before) the point in the boot cycle where
 BIND would, on other systems?  I suspect that it should be
 possible by changing the PROVIDE: in svscan.sh to include one of
 the things REQUIRE:'d by ntpd.  Or perhaps the REQUIRE: LOGIN in
 svscan.sh is incompatible with the BEFORE: LOGIN in ntpd?
 
 Has any other user of dnscache encountered and solved this
 problem?

I have, in my svscan.sh,

# PROVIDE: svscan
# REQUIRE: SERVERS cleanvar

and I also have a /usr/local/etc/rc.d/dnscache which looks like

#!/bin/sh

# PROVIDE: named
# REQUIRE: svscan

. /etc/rc.subr
. /usr/local/etc/svc.subr

name=dnscache
rcvar=`set_rcvar`

: ${dnscache_enable:=NO}

load_rc_config $name
load_svc_subr_config

run_rc_command $1

and a /usr/local/etc/svc.subr as attached. It's somewhat more general
than is needed for dnscache, as I also use it to start qmail.

Basically: the PROVIDE: named line is needed to get dnscache up before
ntpd, and the REQUIRE: LOGIN line needs to be changed.

Ben

#
# Subroutines for handling services under the control of svscan(8).
# Requires rc.subr is loaded first.
#

load_svc_subr_config () {
load_rc_config_var svscan svscan_servicedir

: ${svscan_servicedir:=/var/service}
: ${svc:=/usr/local/bin/svc}
: ${svok:=/usr/local/bin/svok}
: ${svstat:=/usr/local/bin/svstat}

: ${svcname:=${name}}   
: ${service_dir:=${svscan_servicedir}/${svcname}}
: ${svok_timeout:=30}
: ${svstat_timeout:=30}

: ${start_cmd:=svc_subr_start}
: ${stop_cmd:=svc_subr_stop}
: ${restart_cmd:=svc_subr_restart}

: ${extra_commands:=status reload flush}
: ${status_cmd:=svc_subr_status}
: ${reload_cmd:=svc_subr_reload}
: ${flush_cmd:=svc_subr_flush}
}

svc_subr_isup () {
$svstat $1 | grep -q : up
}

svc_subr_isdown () {
$svstat $1 | grep -q : down
}

svc_subr_waitfor () {
local n check timeout err

check=$1
timeout=$2
err=$3

n=0
while [ $n -lt $timeout ]  ! $check $service_dir
do
sleep 1
n=$(( n + 1 ))
done

if ! $check $service_dir
then
echo $err 2
exit 1
fi
}

svc_subr_start () {

echo -n Checking ${name} is up

svc_subr_waitfor $svok $svok_timeout \
supervise is not running in ${service_dir}!

$svc -u $service_dir

svc_subr_waitfor svc_subr_isup $svstat_timeout \
${service_dir} won't come up!

echo .
}

svc_subr_stop () {

echo -n Bringing ${name} down

$svc -d $service_dir

svc_subr_waitfor svc_subr_isdown $svstat_timeout \
${service_dir} won't come down, trying SIGKILL

svc -k $service_dir

svc_subr_waitfor svc_subr_isdown $svstat_timeout \
${service_dir} won't come down!

echo .
}

svc_subr_restart () {
echo -n Sending ${name} a SIGTERM
$svc -t $service_dir
echo .
}

svc_subr_reload () {
echo -n Sending ${name} a SIGHUP
$svc -h $service_dir
echo .
}

svc_subr_flush () {
echo -n Sending ${name} a SIGALRM
$svc -a $service_dir
echo .
}

svc_subr_status () {
$svstat $service_dir
}
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

Re: Medical database Vidal

2009-01-09 Thread Ben Morrow
In article 20090109184126.ga2...@pollux2.free.local.net you write:
 When mounting a (cd9660) CD-ROM of the medical database Vidal in order
 to try an installation with wine, I've discovered that I cannot see
 two files (visible under Windows), setup.exe and some .ini file the
 full name of which I have forgotten now, while I can perfectly see
 the merlin-vcd-data.zip file in the dat directory.
 
 How on Unix earth is this possible ??

As you may already know, native ISO9660 (well, level 1, which is what is
usually used) only supports very limited filenames (8.3, uppercase,
every file must have a version number). As a result, a number of
extensions have been developed: Unix systems use a system called Rock
Ridge, which supports long filenames and the usual POSIX metadata; Win32
systems use a system called Joliet, which uses Unicode filenames and
supports vfat-ish metadata; Apple have their own system which supports
HFSish metadata. 

Some CDs are built with more than one extension system, in which case
there are now several completely independant directory trees on the
disc. It's perfectly possible to make the different trees contain
different files; indeed, it's possible to make the same file appear to
contain different contents under different systems. See e.g. the -hide,
-hide-joliet, -hide-hfs options to mkisofs.

I would guess that your CD has both Rock Ridge and Joliet extensions,
and that the creator has hidden the Win32-specific files from the Unix
directory tree because they thought they wouldn't be useful. If for some
reason you need to see the CD as a Win32 machine would, you can use the
-r option to mount_cd9660.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: mergemaster broken -- take 2

2009-01-09 Thread Ben Morrow
In article 496819f...@vintners.net you write:
   cvsup -g -L 2 -h cvsup10.freebsd.org | tee /root/cvsup-090108.out
   make buildworld | tee /root/make-buildworld-090108.out
   make kernel KERNCONF=GENERIC | tee /root/make-kernel-090108.out

You know about script(1) I take it? It makes keeping a log like this
much easier.

Ben

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org