Re: CVS commit: src/sys/kern

2013-08-27 Thread Thor Lancelot Simon
On Tue, Aug 27, 2013 at 02:01:36PM +, Taylor R Campbell wrote:
 
 This is a stop-gap on a stop-gap on a stop-gap; we really ought to
 back out all of these stop-gaps, make bcm2835_rng call rnd_add_data
 asynchronously to work around the original symptom, and design a real
 solution when we have time to sort this mess out properly.

Having slept on it -- you're right.  Most of this series of changes
should be backed out (a few parts are worth keeping, like the tweaks
to the softint initialization).

In particular, the shortcut calls to rnd_process_events should go back
into the softint schedule routines, and be removed from the extract
functions.

Then, I think, as a better interim solution, we should:

1) Make the hardware RNGs loop, synchronously adding entropy
   until they've added a pool-full, at startup.

2) Make all the users of the callback interface add entropy
   asynchronously at startup, so early in kernel startup
   other RNGs are not initialized from a near-empty pool.

Thor


Re: CVS commit: src/lib

2013-03-22 Thread Thor Lancelot Simon
On Sat, Mar 23, 2013 at 12:12:49AM +0900, Joerg Sonnenberger wrote:
 On Fri, Mar 22, 2013 at 01:08:18PM +, Christos Zoulas wrote:
  In article 20130322030831.ga10...@britannica.bec.de,
  Joerg Sonnenberger  jo...@britannica.bec.de wrote:
  
  This is wrong and unnecessary as pointed out on the mailing lists. Why
  are technical objections silently ignored now? Especially if they add
  overhead without any gain and make the error case more mysterious.
  
  Technical discussions are not being ignored. This is considered
  the best compromise at this time to fix the regression for NetBSD-6.
  We cannot wait forever to fix problems. You are welcome to do
  better. No-one is right all the time. As mentioned in the commit
  message, this is not the final solution.
 
 I did already post a patch that fixes the problem for devel/glib.
 So yes, I provided a better fix and it was ignored.

And furthermore, there was no regression.  One example was provided of
an application (actually a PAM module) that now threw an error rather
than happening to get lucky and silently working rather than failing.

Joerg found the root cause of this problem -- a bug in glib -- and
fixed it.  Rather than checking in his fix to the buggy application, this
ill-considered change was made to libc and libpthread.  Not good.

It shouldn't be possible to shout down correct technical objections
by making a thread last forever so core won't have time to read and
think about it in its entirety.  I think that is what happened here.

Thor


Re: CVS commit: [tls-maxphys] src/sys/ufs/ufs

2012-10-16 Thread Thor Lancelot Simon
On Mon, Oct 15, 2012 at 04:24:54PM +0200, Manuel Bouyer wrote:
 
 I wonder if we could reuse bits from the read-ahead code for write-behind ?

I'm not sure the read-ahead code is working properly.  When I read through
the filesystem with cat or dd bs=1024k, on a device (wd on ahcisata)
that can do 1M transfers, I see that most transfers are considerably
smaller.

What I'm more worried about is that it may _never_ be a good idea, in the
general case, to put 1M transfers on the bus, particularly single writes,
since they may delay things like log transactions in the disk's write
cache.  We might need a much safer I/O scheduler before we can safely do
this.  Or we might need to let the user tune it on a per-filesystem basis.

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-10 Thread Thor Lancelot Simon
On Wed, Oct 10, 2012 at 11:34:48AM +0200, Manuel Bouyer wrote:
 
 I'm not sure what the best way to handle this would be.
 If we assume that maxphys is a power of 2, we could use a maxphys-derived
 mask here. Otherwise, maybe we should compute and cache the largest power-of-2
 value below maxphys in v_mount, as is done in vfs_vnops.c:vn_ra_allocctx()
 (actually, this would remove ra_iochunk as we could use the mount point's
 value)

I originally did that.  There were some negative performance consequences.

With two of the SCSI (RAID, actually) controllers I have on hand, the DMA
descriptor format effectively imposes a restriction of 192K on transfer
size.  But these are exactly the sort of devices that really like to see
large writes.  The next lower power of two is 128k...

I think a lot of the clever math with shifts and masks should perhaps go
away.  Any efficiency improvements it yields are, I think, swamped by the
5,000 function calls we make to do a single I/O.

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-10 Thread Thor Lancelot Simon
On Wed, Oct 10, 2012 at 11:34:48AM +0200, Manuel Bouyer wrote:
 On Tue, Oct 09, 2012 at 05:59:06PM -0700, Chuck Silvers wrote:
   [...]
   with a 'cat big_file  /dev/null'
   writes are still limited to 64k ...
  
  I would hope that cat'ing a file to /dev/null wouldn't result in any 
  writes.  :-)
  I assume you meant 'cat big_file  other_file' ?
 
 I use: dd if=/dev/zero of=bigfile bs=1g count=7
 
  
  if so, then the reason for the 64k writes would be this block of code in 
  ffs_write():
  
  if (!async  oldoff  16 != uio-uio_offset  16) {
  mutex_enter(vp-v_interlock);
  error = VOP_PUTPAGES(vp, (oldoff  16)  16,
  (uio-uio_offset  16)  16,
  PGO_CLEANIT | PGO_JOURNALLOCKED | PGO_LAZY);
  if (error)
  break;
  }
  
 
 that's it. I did s/16/32/g in the code above and now I get 128k writes.

32?  Not 17?

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-10 Thread Thor Lancelot Simon
On Tue, Oct 09, 2012 at 05:59:06PM -0700, Chuck Silvers wrote:
 
 if so, then the reason for the 64k writes would be this block of code in 
 ffs_write():
 
   if (!async  oldoff  16 != uio-uio_offset  16) {
   mutex_enter(vp-v_interlock);
   error = VOP_PUTPAGES(vp, (oldoff  16)  16,
   (uio-uio_offset  16)  16,
   PGO_CLEANIT | PGO_JOURNALLOCKED | PGO_LAZY);
   if (error)
   break;
   }

Upon reflection, I do not understand how this works.

Consider a write() of 131072 starting at file offset 0.

oldoff  16 will be 0; uio-uio_offset will be 131072, unless
this is the source of my misunderstanding, after the call to ubc_uiomove().
Otherwise, I don't see how uio-uio_offset gets updated at all.

Now, we VOP_PUTPAGES(vp, 0, (131072  16)  16)) which is of course
VOP_PUTPAGES(vp, 0, 131072).  So why does this only push out 64K?

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-09 Thread Thor Lancelot Simon
On Tue, Oct 09, 2012 at 03:50:18PM +0200, Manuel Bouyer wrote:
 On Tue, Oct 09, 2012 at 01:36:07PM +, Manuel Bouyer wrote:
  [...]
  Log Message:
  Support transfers of up to MACHINE_MAXPHYS in all pciide variants, and ahci.
  wd(4) limits its maxphys depending on the drives's capability (64k sectors
  for LBA48, 256 sectors for LBA and 128 sectors for older devices).
  
  I assumed all pciide controllers could do MACHINE_MAXPHYS transfers, but
  this may not be true. The capabilities of each controller variants should be
  looked at more closely.
 
 transfers with dd on the raw character device works fine, but I get this
 panic when booting multiuser, both with pciide and ahci controllers:
 Starting ntpd.
 panic: bad chunksize 139264, iochunk 524288, request size 139264

Hm.

This will be an attempt to transfer an entire file -- 139264 will be the
entire file size of something, probably an executable it's paging in.

I saw this before -- I can't remember what caused it -- the bad chunksize
is, I think, because it's not power-of-2.  The cause will be in the UVM
readahead code.

I could swear I fixed at least one cause of this already.

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-09 Thread Thor Lancelot Simon
On Tue, Oct 09, 2012 at 05:21:17PM +0200, Manuel Bouyer wrote:
 On Tue, Oct 09, 2012 at 11:04:20AM -0400, Thor Lancelot Simon wrote:
  Hm.
  
  This will be an attempt to transfer an entire file -- 139264 will be the
  entire file size of something, probably an executable it's paging in.
  
  I saw this before -- I can't remember what caused it -- the bad chunksize
  is, I think, because it's not power-of-2.  The cause will be in the UVM
  readahead code.
 
 It is. It looks like this code is different in tls-maxphys and HEAD.

Is it different in tls-maxphys-base and HEAD?  I played with this code
quite a bit while debugging previously, I believe.

 I wonder if we really want to read in power-of-2 chunks here, or just
 multiple of page size. Reading the code which calls ra_startio(),
 I'm not sure why size of chunksize would be a power-of-2.

I wondered that myself.  I think multiple of pagesize is probably okay,
but -- IIRC -- I couldn't convince myself we'd always do the right thing
with the residual.  Try it?

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-09 Thread Thor Lancelot Simon
On Tue, Oct 09, 2012 at 05:35:44PM +0200, Manuel Bouyer wrote:
 On Tue, Oct 09, 2012 at 11:24:13AM -0400, Thor Lancelot Simon wrote:
  On Tue, Oct 09, 2012 at 05:21:17PM +0200, Manuel Bouyer wrote:
   On Tue, Oct 09, 2012 at 11:04:20AM -0400, Thor Lancelot Simon wrote:
Hm.

This will be an attempt to transfer an entire file -- 139264 will be the
entire file size of something, probably an executable it's paging in.

I saw this before -- I can't remember what caused it -- the bad 
chunksize
is, I think, because it's not power-of-2.  The cause will be in the UVM
readahead code.
   
   It is. It looks like this code is different in tls-maxphys and HEAD.
  
  Is it different in tls-maxphys-base and HEAD?  I played with this code
  quite a bit while debugging previously, I believe.
 
 Well, the code which cause the panic changed from
   const size_t chunksize = RA_IOCHUNK;
 to
   const size_t chunksize = MIN(chunksz, round_page(sz));
 
 so the power-of-2 check changes from checking that a #define constant
 has the right value, to sanity-checking that the values we got from upper
 layers. And AFAIK then callers don't make any effort for these values to
 be a power-of-2, but it may make sure that it's page-size aligned
 (I'm not even sure of that: the end of the file is not necesserely
 page-aligned ...)

But the chunk-size really has two effects: we round short transfers *up*
to (a multiple of) the chunk-size if the file is large enough
(because intermediate layers lose the actual requested I/O size even
for read(), again IIRC) and we break large readheads *down* into multiple
reads of chunk-size.

That's the reason for the somewhat odd definition.  So I think the power
of 2 check should in fact go, now that I've thought through it more.

It seems to me if we got the residual at the end of the file right before
we probably still do now, but that could definitely use another set of
eyes.

Thor


Re: CVS commit: [tls-maxphys] src/sys/dev

2012-10-09 Thread Thor Lancelot Simon
On Tue, Oct 09, 2012 at 10:15:06PM +0200, Manuel Bouyer wrote:
 
 Now, iostat -x shows that sequential read of a large file (either with cat
 or dd bs=1m) is done in 32k transfers at the disk level. So I guess something
 is not working properly here ...

I saw that too, though I managed to get 64K transfers.  I have been meaning
to test with mmap, since that is more like the code path through the upper
layers that produces the attempt to page in an entire executable.

I've looked all through every layer I can identify and I do not see where
the seemingly still remaining 64K restriction comes from.  I thought it
was FFS, but in fact *every vestige* of the maxcontig or maximum extent
size use seems to have been excised from our FFS code.  So it's a mystery
to me.

Chuck, any ideas?

Thor


Re: amd64 direct map causing crashes at boot (Re: CVS commit: src/sys/arch)

2011-12-08 Thread Thor Lancelot Simon
On Thu, Dec 08, 2011 at 07:47:07AM -0800, Chuck Silvers wrote:
 
 I'd like to leave the direct-map stuff enabled for now so we can
 gather some more data on where it works and where it doesn't.
 if you (and anyone who sees this problem) could send me the output from
 dmesg and cpuctl identify, hopefully that will give me some clues.
 actually if people who don't see this problem with a kernel from monday
 or later  could send me that info too, that will also hope.

Built the kernel with the _HAVE_DIRECT_MAP block turned off.  Works
fine, gives me the attached dmesg and cpuctl identify.

Thor
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011
The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.

NetBSD 5.99.59 (RNDVERBOSE) #34: Thu Dec  8 22:28:28 EST 2011

luser@beaky.local:/Volumes/csfs/amd64-build/objdir/sys/arch/amd64/compile/RNDVERBOSE
total memory = 511 MB
avail memory = 479 MB
rnd: initialised (4096) with counter
rnd: WARNING! initial entropy low (2).
rnd: entropy sink arc4random wants 16 bytes of data.
timecounter: Timecounters tick every 10.000 msec
rnd: WARNING! initial entropy low (0).
cprng kernel: WARNING insufficient entropy at creation.
rnd: WARNING! initial entropy low (1).
timecounter: Timecounter i8254 frequency 1193182 Hz quality 100
innotek GmbH VirtualBox (1.2)
mainbus0 (root)
cpu0 at mainbus0 apid 0: Intel(R) Core(TM)2 CPU T7400  @ 2.16GHz, id 
0x6f6
cpu1 at mainbus0 apid 1: Intel(R) Core(TM)2 CPU T7400  @ 2.16GHz, id 
0x6f6
ioapic0 at mainbus0 apid 2: pa 0xfec0, version 11, 24 pins
acpi0 at mainbus0: Intel ACPICA 20110623
acpi0: X/RSDT: OemId VBOX  ,VBOXXSDT,0001, AslId ASL ,0061
acpi0: SCI interrupting at int 9
timecounter: Timecounter ACPI-Safe frequency 3579545 Hz quality 900
hpet0 at acpi0: high precision event timer (mem 0xfed0-0xfed00400)
timecounter: Timecounter hpet0 frequency 14318180 Hz quality 2000
pckbc1 at acpi0 (PS2K, PNP0303) (kbd port): io 0x60,0x64 irq 1
pckbc2 at acpi0 (PS2M, PNP0F03) (aux port): irq 12
LPT (PNP0400) at acpi0 not configured
attimer1 at acpi0 (TIMR, PNP0100): io 0x40-0x43,0x50-0x53
SRL0 (PNP0501) at acpi0 not configured
acpibat0 at acpi0 (BAT0, PNP0C0A-0): ACPI Battery
acpibat0: innotek VBOX rechargeable battery
acpibat0: granularity: low-warn 0.001 Wh, warn-full 0.001 Wh
acpiacad0 at acpi0 (AC, ACPI0003-0): ACPI AC Adapter
pckbd0 at pckbc1 (kbd slot)
pckbc1: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
rnd: pckbd0 attached as an entropy source (collecting)
pms0 at pckbc1 (aux slot)
pckbc1: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
rnd: pms0 attached as an entropy source (collecting)
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
vga0 at pci0 dev 2 function 0: vendor 0x80ee product 0xbeef (rev. 0x00)
wsdisplay0 at vga0 kbdmux 1: console (80x25, vt100 emulation), using wskbd0
wsmux1: connecting to wsdisplay0
drm at vga0 not configured
wm0 at pci0 dev 3 function 0: Intel i82540EM 1000BASE-T Ethernet, rev. 2
wm0: interrupting at ioapic0 pin 19
wm0: 32-bit 33MHz PCI bus
wm0: 64 word (6 address bits) MicroWire EEPROM
wm0: Ethernet address 08:00:27:06:ea:f6
wm0: MDIC read timed out: phy 0 reg 1
makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 4
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rnd: wm0 attached as an entropy source (off)
vendor 0x80ee product 0xcafe (miscellaneous system) at pci0 dev 4 function 0 
not configured
auich0 at pci0 dev 5 function 0: i82801AA (ICH) AC-97 Audio
auich0: interrupting at ioapic0 pin 21
auich0: ac97: SigmaTel STAC9700 codec; no 3D stereo
auich0: ac97: ext id 0x809AC97_23,VRM,VRA
piixpm0 at pci0 dev 7 function 0
piixpm0: vendor 0x8086 product 0x7113 (rev. 0x08)
piixpm0: SMBus disabled
mpt has not been converted to device_t
mpt0 at pci0 dev 20 function 0: vendor 0x1000 product 0x0030
mpt0: applying 1030 quirk
mpt0: interrupting at ioapic0 pin 20
scsibus0 at mpt0: 16 targets, 8 luns per target
ppb0 at pci0 dev 24 function 0: vendor 0x8086 product 0x2448 (rev. 0xf2)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
ppb1 at pci0 dev 25 function 0: vendor 0x8086 product 0x2448 (rev. 0xf2)
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled
ichlpcib0 at pci0 dev 31 function 0: vendor 0x8086 product 0x27b9 (rev. 0x02)
timecounter: Timecounter ichlpcib0 frequency 3579545 Hz quality 1000
ichlpcib0: 24-bit timer
ichlpcib0: TCO (watchdog) timer configured.
ahcisata0 at pci0 dev 31 function 2: vendor 0x8086 product 0x2829
ahcisata0: interrupting at ioapic0 pin 23
ahcisata0: 64-bit DMA
ahcisata0: AHCI revision 1.10, 1 ports, 32 slots, CAP 
0xc8241f80CCCS,SAM,ISS=0x2=Gen2,SSS,SNCQ,S64A
atabus0 at ahcisata0 channel 0
ohci0 at pci0 dev 31 function 4: vendor 0x106b product 0x003f (rev. 0x00)
ohci0: 

Re: CVS commit: src/sys/arch

2011-12-07 Thread Thor Lancelot Simon
On Sun, Dec 04, 2011 at 04:24:13PM +, Chuck Silvers wrote:
 Module Name:  src
 Committed By: chs
 Date: Sun Dec  4 16:24:13 UTC 2011
 
 Modified Files:
   src/sys/arch/amd64/amd64: locore.S machdep.c
   src/sys/arch/amd64/include: types.h
   src/sys/arch/x86/include: pmap.h
   src/sys/arch/x86/x86: pmap.c
 
 Log Message:
 map all of physical memory using large pages.
 ported from openbsd years ago by Murray Armfield,
 updated for changes since then by me.

This change causes my 2-cpu, x86-64 VirtualBox VM to hang hard
immediately on boot, regardless of how I build the kernel.  If I build
the kernel with DEBUG, it hangs immediately after printing
kernel text is mapped with 6 large pages and 442 normal pages.

In fact, it hangs so hard that it makes VirtualBox throw a fatal error
when I try to reset the VM.

I understand this change is _not_ causing trouble for some others, but
it sure does for me.  Can we back it out temporarily, and what can I
offer by way of help in debugging it?

Thor


Re: CVS commit: src

2011-11-29 Thread Thor Lancelot Simon
On Tue, Nov 29, 2011 at 01:06:17AM -0800, John Nemeth wrote:
 
  Currently, a kmod is an ELF binary.  However, I had a look at the
 code and I think without too much effort we can achieve what you want.

Already done, but thanks!

There's no point to loading entropy from modload; rndctl can already
load it.  I'll have to poke around at what it takes to do this with
multiboot, since I want it for Xen kernels too.

Thor


Re: CVS commit: src

2011-11-23 Thread Thor Lancelot Simon
On Wed, Nov 23, 2011 at 12:06:26PM +0100, Christoph Egger wrote:
 On 11/23/11 11:47, Thor Lancelot Simon wrote:
 Module Name: src
 Committed By:tls
 Date:Wed Nov 23 10:47:50 UTC 2011
 
 Modified Files:
  src/distrib/sets/lists/etc: mi
  src/etc/defaults: rc.conf
  src/etc/rc.d: Makefile
  src/sbin/rndctl: rndctl.8 rndctl.c
  src/sys/dev: rnd.c
  src/sys/secmodel/securelevel: secmodel_securelevel.c
  src/sys/secmodel/suser: secmodel_suser.c
  src/sys/sys: kauth.h rnd.h
 Added Files:
  src/etc/rc.d: random_seed
 
 Log Message:
 Load entropy at system boot (only works at securelevel  1); save
 at system shutdown.  Disable with random_seed=NO in rc.conf if desired.
 
 Goes to some trouble to never load or save to network filesystems.
 
 Entropy should really be loaded by the boot loader but I am still
 sorting out how to pass it to the kernel.
 
 How about passing it as a module similar to the multiboot technique?

Can't make one of those without an ELF toolchain, right?  The basic
idea's about right, but I actually need something less sophisticated in its
packaging -- a way to just give the kernel the address of blob-of-stuff
the bootloader's dropped into place for it, so the entropy pool code
can just take it and prime itself.

Thor


Re: CVS commit: src/sys/ddb

2010-08-31 Thread Thor Lancelot Simon
On Wed, Sep 01, 2010 at 06:01:23AM +1000, Simon Burge wrote:
 Thor Lancelot Simon wrote:
 
  Module Name:src
  Committed By:   tls
  Date:   Mon Aug 30 19:23:26 UTC 2010
  
  Modified Files:
  
  src/sys/ddb: db_input.c
 
 Was there any particular reason for removing the lint directives in this
 change?  Eg:
 
 -   } while (/*CONSTCOND*/ 0)
 +} while (0)

No.

Thor


Re: CVS commit: src/external/bsd/liblzf/dist

2010-02-03 Thread Thor Lancelot Simon
On Wed, Feb 03, 2010 at 04:55:07PM +0100, Adam Hoka wrote:
 
  It's a tiny, very very fast compression library. ?Christos wants it
  for bootblocks, among other things (or so he says -- ask him).
 
 U-boot and grub2 can boot lzma compressed images, and its also more
 efficient, what about lza? I think this needs some rethinking. :-)

I think you're talking about a different kind of efficiency.  lzf
decompresses at nearly the speed of memcpy, and the decompressor is
900 bytes.

The public domain C implementation of lzma is well over half a megabyte
and appears to require threads.  It seems extremely silly to me that
you'd pick that particular straw man while complaining about system bloat.

In any event, I'm certainly not going to back out lzf on the basis of
such silliness.  So, personally, I consider this conversation closed.
If you have a problem with that, take it up with core.

Thor


Re: CVS commit: src/external/bsd/liblzf/dist

2010-02-03 Thread Thor Lancelot Simon
On Wed, Feb 03, 2010 at 04:33:59PM +0100, Joerg Sonnenberger wrote:
 On Wed, Feb 03, 2010 at 12:44:39AM -0500, Thor Lancelot Simon wrote:
  It's a tiny, very very fast compression library.  Christos wants it
  for bootblocks, among other things (or so he says -- ask him).
 
 For boot blocks or boot loaders?

Ask Christos.  I don't use it for that purpose myself (I use it for
a number of other things a very fast compressor is useful for, like
network data streams and backups).  I am guessing what he was after
was small size and fast decompression (for some resource-constrained
platform?).

The bottom line is that I had the code in my local tree, someone whose
technical opinion I respect sufficiently to simply check it in on his
request made the request, and so I did.

As I said earlier, if you want to remove 3-5000 bytes from the system,
I can recommend some much better candidates.

Thor


Re: CVS commit: src/external/bsd/liblzf/dist

2010-02-03 Thread Thor Lancelot Simon
On Wed, Feb 03, 2010 at 10:19:57PM +0100, Joerg Sonnenberger wrote:
 
 so that the size advantage of the decoder doesn't matter too much. I
 don't mind a fast compressor for streaming applications, but those
 should exist and named as such.

Well, certainly I named some of mine.  I can't check my existing main
use of lzf into the tree, which is why I didn't check lzf itself in
until another developer asked me to.

 Even than it should at least be somewhat
 justified to use a non-standard, ad-hoc algorithm (compared to deflate).

By no means is deflate a fast compressor for streaming applications,
because it's not fast.  It's about 1/4 the speed of lzf at best.

My personal opinion is that the threshold for adding something very small
(5K max on any platform I can find) to our tree should be correspondingly
low, while the threshold for adding something larger should be, of course,
correspondingly high.  If this were 150K, 300K, etc. I'd proceed with a
lot more caution.

Thor


CVS commit: src/external/bsd/liblzf/dist

2010-02-02 Thread Thor Lancelot Simon
Module Name:src
Committed By:   tls
Date:   Wed Feb  3 03:50:45 UTC 2010

Update of /cvsroot/src/external/bsd/liblzf/dist
In directory ivanova.netbsd.org:/tmp/cvs-serv22361

Log Message:
Import liblzf version 3.5 (2-clause BSD licensed by Marc Lehmann)

Status:

Vendor Tag: schmorpforge
Release Tags:   liblzf-3-5

N src/external/bsd/liblzf/dist/Changes
N src/external/bsd/liblzf/dist/LICENSE
N src/external/bsd/liblzf/dist/Makefile.in
N src/external/bsd/liblzf/dist/README
N src/external/bsd/liblzf/dist/config.h.in
N src/external/bsd/liblzf/dist/configure
N src/external/bsd/liblzf/dist/configure.ac
N src/external/bsd/liblzf/dist/crc32.h
N src/external/bsd/liblzf/dist/install-sh
N src/external/bsd/liblzf/dist/lzf.c
N src/external/bsd/liblzf/dist/lzf.h
N src/external/bsd/liblzf/dist/lzfP.h
N src/external/bsd/liblzf/dist/lzf_c.c
N src/external/bsd/liblzf/dist/lzf_d.c
N src/external/bsd/liblzf/dist/cs/README
N src/external/bsd/liblzf/dist/cs/CLZF.cs

No conflicts created by this import



Re: CVS commit: src/external/bsd/liblzf/dist

2010-02-02 Thread Thor Lancelot Simon
On Wed, Feb 03, 2010 at 04:04:22PM +1100, matthew green wrote:

Update of /cvsroot/src/external/bsd/liblzf/dist
In directory ivanova.netbsd.org:/tmp/cvs-serv22361

Log Message:
Import liblzf version 3.5 (2-clause BSD licensed by Marc Lehmann)
 
 
 what's liblzf, and why do we want it in base?

It's a tiny, very very fast compression library.  Christos wants it
for bootblocks, among other things (or so he says -- ask him).

The other obvious choice would be LZO but there's no suitably-licensed
implementation which performs well.

We've had it in our tree at Coyote Point for a long time for a completely
different purpose, so I just grabbed it across.  As time permits, I will
plug it into things like ssh, libarchive, and ipcomp.



CVS commit: src

2010-02-02 Thread Thor Lancelot Simon
Module Name:src
Committed By:   tls
Date:   Wed Feb  3 06:25:55 UTC 2010

Modified Files:
src/distrib/sets/lists/base: shl.elf shl.mi
src/distrib/sets/lists/comp: mi shl.mi
src/external/bsd: Makefile
Added Files:
src/external/bsd/liblzf: Makefile Makefile.inc
src/external/bsd/liblzf/lib: Makefile

Log Message:
Plug liblzf into build.  Liblzf is a small (3504 byte shared library on i386),
very very fast, 2-clause BSD-licensed compressor.  We provide the LZF_STATE
version of the API, which is not the default for generic liblzf.


To generate a diff of this commit:
cvs rdiff -u -r1.198 -r1.199 src/distrib/sets/lists/base/shl.elf
cvs rdiff -u -r1.517 -r1.518 src/distrib/sets/lists/base/shl.mi
cvs rdiff -u -r1.1380 -r1.1381 src/distrib/sets/lists/comp/mi
cvs rdiff -u -r1.100 -r1.101 src/distrib/sets/lists/comp/shl.mi
cvs rdiff -u -r1.20 -r1.21 src/external/bsd/Makefile
cvs rdiff -u -r0 -r1.1 src/external/bsd/liblzf/Makefile \
src/external/bsd/liblzf/Makefile.inc
cvs rdiff -u -r0 -r1.1 src/external/bsd/liblzf/lib/Makefile

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



Re: CVS commit: src/external/bsd/liblzf/dist

2010-02-02 Thread Thor Lancelot Simon
On Wed, Feb 03, 2010 at 08:39:29AM +0100, Adam Hamsik wrote:
 
 Where was this change discussed ? Why were are importing sourcecode without
 any users in base system(look at Lua case). If time permits is quite
 vague, if you will not have a time who will plug this lib to base ?

I imported it because the member of core with the longest tenure in the
history of the project asked me to do so.  That's good enough for me.

Please don't interpret this as a suggestion that I agree with what has
(not!) been done with Lua.

If you find a 3000-byte shared library objectionable, I can point you
at at least 3000 bytes elsewhere in the system which are better candidates
for removal.

Thor


Re: CVS commit: src/sys

2009-09-02 Thread Thor Lancelot Simon
On Wed, Sep 02, 2009 at 08:11:35PM +0200, Thomas Klausner wrote:

  Log Message:
  Add a direction argument to socket upcalls, so they can tell why they've
  been called when, for example, they're waiting for space to write.  From
  Ritesh Agrawal at Coyote Point.
 
 Do you think this might be related to:
 --- dependall-nfsserver ---
 /archive/build/tools/bin/x86_64--netbsd-gcc -O2 -mcmodel=kernel -Wall 
 -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-sign-compare 
 -Wno-traditional -Wa,--fatal-warnings  -Werror  -ffreestanding  
 -fno-strict-aliasing -Wno-pointer-sign  -I/archive/cvs/src/common/include 
 -DNFS -DNFSSERVER -I/archive/cvs/src/common/include  -nostdinc -I. 
 -I/archive/cvs/src/sys/modules/nfsserver -isystem /archive/cvs/src/sys 
 -isystem /archive/cvs/src/sys/arch -isystem 
 /archive/cvs/src/sys/../common/include -D_KERNEL -D_LKM -D_MODULE -c
 /archive/cvs/src/sys/nfs/nfs_syscalls.c
 cc1: warnings being treated as errors
 /archive/cvs/src/sys/nfs/nfs_syscalls.c: In function 'nfssvc_addsock':
 /archive/cvs/src/sys/nfs/nfs_syscalls.c:410: warning: assignment from 
 incompatible pointer type

Not really -- I did a full build of GENERIC with this change in... let me
check I didn't miss anything in the NFS code.

Thor


Re: CVS commit: src/sys

2009-03-25 Thread Thor Lancelot Simon
On Wed, Mar 25, 2009 at 06:14:13PM +0900, Izumi Tsutsui wrote:
 dar...@netbsd.org wrote:
 
  Added Files:
  src/sys/lib/libkern: crc32.c crc32.h
  :
  Adds the fast version of crc32 from zlib to libkern. This should be 
  generally
  useful and provide a place to start normalizing the various crc32 routines
  in the kernel.  The crc32 routine is used in this patch to support GZIP.
 
 Isn't it better to simply build and link libz
 rather than having a dumb copies in libkern?

What was added to libkern is much simpler than what's in libz (not just
because the libz sources are stylistically gross).  It's not only meant
for use by libz.

Thor