Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Crystal Kolipe]

On Wed, Feb 15, 2023 at 03:10:08PM +0100, Why 42? The lists account. wrote:
> However, I also tried testing the same two filesystems using the
> "Flexible IO Tester" or fio (it's available as a package). When I used it
> to do random 4K reads and writes, I appear to have the opposite result:

...

> I wonder why that would be?

For a start, I would test using something other than /dev/zero as the data
source.

It's entirely possible that the firmware on an SSD would special case writing
a block that contains only 0x00 bytes.

In that case, and assuming that the filesystem block boundaries align with
the SSD's own internal flash block layout, the SSD would only need to update
it's metadata to point those LBA blocks to an internal 'zero' block.

This would virtually eliminate the overhead of actually writing to the flash,
and allow it to accept data from the host at a much faster speed.

As soon as you write a single non-0x00 byte, the drive would have to do a
propper write to the main flash memory and not just the area which contains
it's internal LBA to flash block mapping, (which may also be write-cached).

Depending on the state of the SSD, (recently secerased, used with another
OS which supports TRIM, alignment of the filesystem blocks with the raw
flash blocks, etc, etc), this could mean either a write, or a
read-erase-write cycle.

Using /dev/zero as the source definitely makes it a synthetic benchmark.

Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Why 42? The lists account.]

On Mon, Feb 13, 2023 at 01:50:13PM -, Stuart Henderson wrote:
> ...
> It maybe worth checking whether mfs is actually helping -
> it's easy to assume that because it's in RAM it must be fast,
> but I've had machines where mfs was slower than SSD
> (https://marc.info/?l=openbsd-misc=164942119618029=2),
> also it's taking memory that could otherwise be used by
> buffer cache.

Hi All,

Since you mentioned it, I thought I would retry your dd test ...

# mount | grep /tmp
mfs:15266 on /tmp type mfs (asynchronous, local, nodev, nosuid, size=16777216 
512-blocks)

% cd !$ ; for i in `jot 5`; do dd if=/dev/zero of=mfs bs=1m count=990 2>&1 | 
grep bytes; done
cd /tmp/dd_test ; for i in `jot 5`; do dd if=/dev/zero of=mfs bs=1m count=990 
2>&1 | grep bytes; done
1038090240 bytes transferred in 1.376 secs (754215208 bytes/sec)
1038090240 bytes transferred in 1.189 secs (872536649 bytes/sec)
1038090240 bytes transferred in 1.227 secs (845718432 bytes/sec)
1038090240 bytes transferred in 1.186 secs (874866632 bytes/sec)
1038090240 bytes transferred in 1.254 secs (827186370 bytes/sec)

# mount | grep /fast
/dev/sd1l on /fast type ffs (local, nodev, nosuid, softdep)
# dmesg | grep sd1
sd1 at scsibus2 targ 1 lun 0: 
...

% cd /fast/dd_test ; for i in `jot 5`; do dd if=/dev/zero of=fast bs=1m 
count=990 2>&1 | grep bytes; done 
1038090240 bytes transferred in 0.871 secs (1191076597 bytes/sec)
1038090240 bytes transferred in 0.635 secs (1633246669 bytes/sec)
1038090240 bytes transferred in 0.615 secs (1685529408 bytes/sec)
1038090240 bytes transferred in 0.605 secs (1714639562 bytes/sec)
1038090240 bytes transferred in 0.612 secs (1694489764 bytes/sec)


So it seems that the Samsung NVMe device is much faster ...

However, I also tried testing the same two filesystems using the
"Flexible IO Tester" or fio (it's available as a package). When I used it
to do random 4K reads and writes, I appear to have the opposite result:

fio --name=rand_mmap_r+w --directory=/tmp/fio_test --rw=randrw --blocksize=4k 
--size=6g --io_size=60g --runtime=600 --ioengine=psync --fsync=1 --thread 
--numjobs=1 --group_reporting
...
Run status group 0 (all jobs):
   READ: bw=130MiB/s (136MB/s), 130MiB/s-130MiB/s (136MB/s-136MB/s), io=30.0GiB 
(32.2GB), run=236394-236394msec
  WRITE: bw=130MiB/s (136MB/s), 130MiB/s-130MiB/s (136MB/s-136MB/s), io=30.0GiB 
(32.2GB), run=236394-236394msec

% fio --name=rand_mmap_r+w --directory=/fast/fio_test --rw=randrw 
--blocksize=4k --size=6g --io_size=60g --runtime=600 --ioengine=psync --fsync=1 
--thread --numjobs=1 --group_reporting
...
Run status group 0 (all jobs):
   READ: bw=34.8MiB/s (36.5MB/s), 34.8MiB/s-34.8MiB/s (36.5MB/s-36.5MB/s), 
io=20.4GiB (21.9GB), run=60-60msec
  WRITE: bw=34.8MiB/s (36.4MB/s), 34.8MiB/s-34.8MiB/s (36.4MB/s-36.4MB/s), 
io=20.4GiB (21.9GB), run=60-60msec

I wonder why that would be?

Disclaimer: I know almost nothing about fio, I've never used it before.
In particular, it isn't clear to me what the correct/best choice is for
the "ioengine" option. (I played around with a few different settings,
that's why you can see that "mmap" in the (test)name argument.)

This is on a 8th generation i5 Intel NUC running a recent snapshot: 7.2
GENERIC.MP#1049

The CPU has 4 cores, hyperthreading is off. The underlying device for
"/fast" is a Samsung M.2 NVMe "stick":
nvme0: Samsung SSD 970 EVO Plus 500GB, firmware 1B2QEXM7 ...

The full output from fio is included below for anyone who might be
interested ...

Cheers,
Robb.


fio --name=rand_mmap_r+w --directory=/tmp/fio_test --rw=randrw --blocksize=4k 
--size=6g --io_size=60g --runtime=600 --ioengine=psync --fsync=1 --thread 
--numjobs=1 --group_reporting
rand_mmap_r+w: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 
4096B-4096B, ioengine=psync, iodepth=1
fio-3.33
Starting 1 thread
rand_mmap_r+w: Laying out IO file (1 file / 6144MiB)
Jobs: 1 (f=1): [m(1)][100.0%][r=134MiB/s,w=134MiB/s][r=34.3k,w=34.2k IOPS][eta 
00m:00s]
rand_mmap_r+w: (groupid=0, jobs=1): err= 0: pid=669956672: Wed Feb 15 13:52:03 
2023
  read: IOPS=33.3k, BW=130MiB/s (136MB/s)(30.0GiB/236394msec)
clat (nsec): min=1523, max=1504.6k, avg=5387.11, stdev=1201.82
 lat (nsec): min=1580, max=1504.7k, avg=5450.15, stdev=1203.46
clat percentiles (nsec):
 |  1.00th=[ 3632],  5.00th=[ 4576], 10.00th=[ 4832], 20.00th=[ 5024],
 | 30.00th=[ 5152], 40.00th=[ 5280], 50.00th=[ 5344], 60.00th=[ 5472],
 | 70.00th=[ 5600], 80.00th=[ 5792], 90.00th=[ 5984], 95.00th=[ 6176],
 | 99.00th=[ 6496], 99.50th=[ 6688], 99.90th=[13376], 99.95th=[18048],
 | 99.99th=[26240]
   bw (  KiB/s): min=126573, max=144312, per=100.00%, avg=133298.71, 
stdev=2476.36, samples=472
   iops: min=31643, max=36078, avg=33324.48, stdev=619.06, samples=472
  write: IOPS=33.2k, BW=130MiB/s (136MB/s)(30.0GiB/236394msec); 0 zone resets
clat (usec): min=3, max=1549, avg=13.84, stdev= 2.06
 lat (usec): min=3, max=1549, avg=13.92, stdev= 2.07
clat 

Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Stuart Henderson]

On 2023-02-12, Why 42? The lists account.  wrote:
>
> You're exactly right. With this entry in fstab:
>> swap /tmp mfs rw,nodev,nosuid,-s=4194304 0 0 
>
> I now have this /tmp space:
>> mjoelnir:~ 12.02 13:15:07 % df -h
>> Filesystem SizeUsed   Avail Capacity  Mounted on
>> /dev/sd1a 1005M537M418M57%/
>> mfs:67535  1.9G   29.0K1.8G 1%/tmp
>> ...
>
> That's right after a reboot. I'll start Chrome now and it can really chow
> down on some /tmp space :-)

It maybe worth checking whether mfs is actually helping -
it's easy to assume that because it's in RAM it must be fast,
but I've had machines where mfs was slower than SSD
(https://marc.info/?l=openbsd-misc=164942119618029=2),
also it's taking memory that could otherwise be used by
buffer cache.

The main benefit to me from mfs is for things which I explicitly
don't want to hit permanent storage.

Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Crystal Kolipe]

On Sun, Feb 12, 2023 at 01:28:04PM +0100, Why 42? The lists account. wrote:
> 
> On Sun, Feb 05, 2023 at 02:50:44PM -0300, Crystal Kolipe wrote:
> > On Sun, Feb 05, 2023 at 06:05:22PM +0100, Why 42? The lists account. wrote:
> > ...
> > > The fstab file contains this mount entry for tmp:
> > > swap /tmp mfs rw,nodev,nosuid,-s=16777216 0 0
> > 
> > This is 8 Gb, which exceeds the default value for datasize for the daemon
> > class in /etc/login.conf.
> > 
> > Have you changed /etc/login.conf from the default?
> > 
> > > Did MFS filesystems go away, or have I screwed something up?
> > 
> > You've screwed something up :).
> 
> You're exactly right. With this entry in fstab:
> > swap /tmp mfs rw,nodev,nosuid,-s=4194304 0 0 
> 
> I now have this /tmp space:
> > mjoelnir:~ 12.02 13:15:07 % df -h
> > Filesystem SizeUsed   Avail Capacity  Mounted on
> > /dev/sd1a 1005M537M418M57%/
> > mfs:67535  1.9G   29.0K1.8G 1%/tmp
> > ...

If you've got plenty of physical RAM, you can always increase the datasize in
login.conf and keep your original 8 Gb mfs ramdisk rather than reducing it.

Not sure if that was clear from my original reply :).

Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Why 42? The lists account.]

On Sun, Feb 05, 2023 at 02:50:44PM -0300, Crystal Kolipe wrote:
> On Sun, Feb 05, 2023 at 06:05:22PM +0100, Why 42? The lists account. wrote:
> ...
> > The fstab file contains this mount entry for tmp:
> > swap /tmp mfs rw,nodev,nosuid,-s=16777216 0 0
> 
> This is 8 Gb, which exceeds the default value for datasize for the daemon
> class in /etc/login.conf.
> 
> Have you changed /etc/login.conf from the default?
> 
> > Did MFS filesystems go away, or have I screwed something up?
> 
> You've screwed something up :).

You're exactly right. With this entry in fstab:
> swap /tmp mfs rw,nodev,nosuid,-s=4194304 0 0 

I now have this /tmp space:
> mjoelnir:~ 12.02 13:15:07 % df -h
> Filesystem SizeUsed   Avail Capacity  Mounted on
> /dev/sd1a 1005M537M418M57%/
> mfs:67535  1.9G   29.0K1.8G 1%/tmp
> ...

That's right after a reboot. I'll start Chrome now and it can really chow
down on some /tmp space :-)

Thanks!

Cheers,
Robb.

Re: Q: Error: mount_mfs: mmap: Cannot allocate memory

[Crystal Kolipe]

On Sun, Feb 05, 2023 at 06:05:22PM +0100, Why 42? The lists account. wrote:
> mount_mfs: mmap: Cannot allocate memory

...

> The fstab file contains this mount entry for tmp:
> swap /tmp mfs rw,nodev,nosuid,-s=16777216 0 0

This is 8 Gb, which exceeds the default value for datasize for the daemon
class in /etc/login.conf.

Have you changed /etc/login.conf from the default?

> Did MFS filesystems go away, or have I screwed something up?

You've screwed something up :).

Q: Error: mount_mfs: mmap: Cannot allocate memory

[Why 42? The lists account.]

Hi All,

After an update to a recent snapshot on my desktop system, I noticed
these mount_mfs messages at boot time:

/dev/sd0h (7a1775fef773535e.h): file system is clean; not checking /dev/sd1j
(281ef747da03afe7.j): file system is clean; not checking
/dev/sd1k (281ef747da03afe7.k): file system is clean; not checking
/dev/sd1l (281ef747da03afe7.l): file system is clean; not checking
/dev/sd2c (67c92dad63883338.c): file system is clean; not checking
mount_mfs: mmap: Cannot allocate memory
kbd: keyboard mapping set to de.nodead
keyboard.encoding -> de.nodead
pf enabled
kern.maxproc: 1310 -> 4000
kern.maxthread: 2620 -> 8000
kern.maxfiles: 7030 -> 16000
ddb.panic: 1 -> 0
kern.allowdt: 0 -> 1
starting network
reordering: ld.so libc libcrypto sshd.
starting early daemons: syslogd pflogd ntpd.
starting RPC daemons: portmap mountd nfsd lockd statd.
mount_mfs: mmap: Cannot allocate memory
savecore: no core dump
checking quotas: done.
clearing /tmp
kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
running rc.sysmerge
starting network daemons: sshd sndiod.
running rc.firsttime
fw_update: added none; updated none; kept intel,inteldrm,vmm
starting package daemons: messagebus postfix smartd pcscd avahi_daemon.
starting local daemons: sensorsd cron xenodm.

The fstab file contains this mount entry for tmp:
swap /tmp mfs rw,nodev,nosuid,-s=16777216 0 0

I don't know when this first occurred. I first noticed it when I was
investigating why chrome had started to log "filesystem full" messages:
e.g. "/: write failed, file system is full.".

Since the mfs mount of /tmp failed, it's now using the root fs as /tmp
space, which doesn't have much free space.

I'm currently running: OpenBSD mjoelnir.fritz.box 7.2 GENERIC.MP#1012 amd64

Did MFS filesystems go away, or have I screwed something up?

Cheers,
Robb.

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 04/04/2022 à 15:43, Claudio Jeker a écrit :


You should really use as-set for this:

as-set ru-set { 2148 2585 2587 ... }

And also not match any (at least I think you don't really want that to
match on ibgp sessions):

match from ebgp AS as-set ru-set set { localpref 250 nexthop blackhole }

If done right you can replace all your rules by one single one.



Hi Claudio,

I followed your advice and everything is stable now. No need to increase 
memory limits.


Thanks for the hint.

Laurent

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 04/04/2022 à 15:43, Claudio Jeker a écrit :

On Tue, Mar 29, 2022 at 09:53:56AM +0200, Laurent CARON wrote:

Hi,

I'm happily running several OpenBGPd routers (Openbsd 7.0).

After having applied the folloxing filters (to blackhole traffic from
certain countries):

include "/etc/bgpd/deny-asn.ru.bgpd"
include "/etc/bgpd/deny-asn.by.bgpd"
include "/etc/bgpd/deny-asn.ua.bgpd"


# head /etc/bgpd/deny-asn.ru.bgpd
match from any AS 2148 set { localpref 250 nexthop blackhole }
match from any AS 2585 set { localpref 250 nexthop blackhole }
match from any AS 2587 set { localpref 250 nexthop blackhole }
match from any AS 2599 set { localpref 250 nexthop blackhole }
match from any AS 2766 set { localpref 250 nexthop blackhole }
match from any AS 2848 set { localpref 250 nexthop blackhole }
match from any AS 2854 set { localpref 250 nexthop blackhole }
match from any AS 2875 set { localpref 250 nexthop blackhole }
match from any AS 2878 set { localpref 250 nexthop blackhole }
match from any AS 2895 set { localpref 250 nexthop blackhole }


You should really use as-set for this:

as-set ru-set { 2148 2585 2587 ... }

And also not match any (at least I think you don't really want that to
match on ibgp sessions):

match from ebgp AS as-set ru-set set { localpref 250 nexthop blackhole }

If done right you can replace all your rules by one single one.


Hi Claudio,

Thanks for the hints.

Will change the config accordingly and report back.

Cheers,

Laurent

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Claudio Jeker]

On Tue, Mar 29, 2022 at 09:53:56AM +0200, Laurent CARON wrote:
> Hi,
> 
> I'm happily running several OpenBGPd routers (Openbsd 7.0).
> 
> After having applied the folloxing filters (to blackhole traffic from
> certain countries):
> 
> include "/etc/bgpd/deny-asn.ru.bgpd"
> include "/etc/bgpd/deny-asn.by.bgpd"
> include "/etc/bgpd/deny-asn.ua.bgpd"
> 
> 
> # head /etc/bgpd/deny-asn.ru.bgpd
> match from any AS 2148 set { localpref 250 nexthop blackhole }
> match from any AS 2585 set { localpref 250 nexthop blackhole }
> match from any AS 2587 set { localpref 250 nexthop blackhole }
> match from any AS 2599 set { localpref 250 nexthop blackhole }
> match from any AS 2766 set { localpref 250 nexthop blackhole }
> match from any AS 2848 set { localpref 250 nexthop blackhole }
> match from any AS 2854 set { localpref 250 nexthop blackhole }
> match from any AS 2875 set { localpref 250 nexthop blackhole }
> match from any AS 2878 set { localpref 250 nexthop blackhole }
> match from any AS 2895 set { localpref 250 nexthop blackhole }
> 

You should really use as-set for this:

as-set ru-set { 2148 2585 2587 ... }

And also not match any (at least I think you don't really want that to
match on ibgp sessions):

match from ebgp AS as-set ru-set set { localpref 250 nexthop blackhole }

If done right you can replace all your rules by one single one.

-- 
:wq Claudio

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Claudio Jeker]

On Mon, Apr 04, 2022 at 03:14:35PM +0200, Laurent CARON wrote:
> 
> Le 01/04/2022 à 14:38, Claudio Jeker a écrit :
> > 
> > The numbers look reasonable with maybe the exception of prefix and BGP
> > path attrs. Unless this system is pushing or pulling lots of full feeds to
> > peers I would not expect such a high number of prefixes. Also the number
> > of path attributes is high but that could again be reasonable if many
> > different full feeds are involved.
> 
> Hi Claudio,
> 
> This box is terminating 3 full IPv4 + 3 full IPv6 feeds + a few dozen IX
> sessions in addition to 5 IPv4 + 5 IPv6 iBGP connections.

3G is not enough for such a busy system. You need to increase your limit,
5GB is probably enough.
 
> > > I'm not sure why the processes gets killed at around 3GB. Feels like you
> > > hit the ulimit. See Stuart's mail about how to look into that.
> > > So looking at this output I feel like you somehow created a BGP update
> > > loop where one or more systems are constantly sending UPDATEs to each
> > > other because the moment the update is processed the route decision
> > > changes and flaps back resulting in a withdraw or update.
> 
> I sincerely think it is not related to a BGP update loop because the issue
> is only triggered when adding the following filters:
> 
> include "/etc/bgpd/deny-asn.ru.bgpd"
> include "/etc/bgpd/deny-asn.by.bgpd"
> include "/etc/bgpd/deny-asn.ua.bgpd"
> 
> for a total of 8265 rules
> 
> I'll try to dig further.

If you deny asns then please use an as-set instead of individual rules.

-- 
:wq Claudio

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 01/04/2022 à 14:38, Claudio Jeker a écrit :


The numbers look reasonable with maybe the exception of prefix and BGP
path attrs. Unless this system is pushing or pulling lots of full feeds to
peers I would not expect such a high number of prefixes. Also the number
of path attributes is high but that could again be reasonable if many
different full feeds are involved.


Hi Claudio,

This box is terminating 3 full IPv4 + 3 full IPv6 feeds + a few dozen IX 
sessions in addition to 5 IPv4 + 5 IPv6 iBGP connections.



I'm not sure why the processes gets killed at around 3GB. Feels like you
hit the ulimit. See Stuart's mail about how to look into that.
  
So looking at this output I feel like you somehow created a BGP update

loop where one or more systems are constantly sending UPDATEs to each
other because the moment the update is processed the route decision
changes and flaps back resulting in a withdraw or update.


I sincerely think it is not related to a BGP update loop because the 
issue is only triggered when adding the following filters:


include "/etc/bgpd/deny-asn.ru.bgpd"
include "/etc/bgpd/deny-asn.by.bgpd"
include "/etc/bgpd/deny-asn.ua.bgpd"

for a total of 8265 rules

I'll try to dig further.


Thanks

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 29/03/2022 à 14:50, Stuart Henderson a écrit :


Also: check the values for bgpd's login class (as root, "su -c bgpd -"
then "ulimit -a"), and are you starting bgpd from the rc-script or by hand?




Hi Stuart,

# ulimit -a
time(cpu-seconds)    unlimited
file(blocks) unlimited
coredump(blocks) unlimited
data(kbytes) 33554432
stack(kbytes)    8192
lockedmem(kbytes)    21502949
memory(kbytes)   64498548
nofiles(descriptors) 512
processes    1310


I'm starting bgpd through "rcctl start bgpd"


Thanks

Laurent

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Claudio Jeker]

On Thu, Mar 31, 2022 at 09:06:05PM +0200, Laurent CARON wrote:
> Le 29/03/2022 à 12:10, Claudio Jeker a écrit :
> > I doubt it is the filters. You run into some sort of memory leak. Please
> > monitor 'bgpctl show rib mem' output. Also check ps aux | grep bgpd output
> > to see why and when the memory starts to go up.
> > With that information it may be possible to figure out where this leak
> > sits and how to fix it.
> > 
> > Cheers
> 
> 
> Hi Claudio,
> 
> Please find the output of 'bgpctl show rib mem' just 1 minute before the
> crash:
> 
> cat 2022-03-30::15:07:01.mem
> RDE memory statistics
> 909685 IPv4 unicast network entries using 34.7M of memory
> 272248 IPv6 unicast network entries using 14.5M of memory
>2363169 rib entries using 144M of memory
>   14616410 prefix entries using 1.7G of memory
>1539060 BGP path attribute entries using 106M of memory
>and holding 14616410 references
> 635275 BGP AS-PATH attribute entries using 33.7M of memory
>and holding 1539060 references
>  47399 entries for 681150 BGP communities using 15.1M of memory
>and holding 14616410 references
>  22139 BGP attributes entries using 865K of memory
>and holding 3436885 references
>  22138 BGP attributes using 175K of memory
> 270121 as-set elements in 249193 tables using 9.7M of memory
> 452138 prefix-set elements using 19.0M of memory
> RIB using 2.1G of memory
> Sets using 28.7M of memory
> 
> RDE hash statistics
> path hash: size 131072, 1539060 entries
> min 0 max 31 avg/std-dev = 11.742/3.623
> aspath hash: size 131072, 635275 entries
> min 0 max 16 avg/std-dev = 4.847/2.123
> comm hash: size 16384, 47399 entries
> min 0 max 12 avg/std-dev = 2.893/1.622
> attr hash: size 16384, 22139 entries
> min 0 max 8 avg/std-dev = 1.351/1.084

The numbers look reasonable with maybe the exception of prefix and BGP
path attrs. Unless this system is pushing or pulling lots of full feeds to
peers I would not expect such a high number of prefixes. Also the number
of path attributes is high but that could again be reasonable if many
different full feeds are involved.
 
> Here is the output of 'ps aux | grep bgp' one minute before the crash:
> 
> _bgpd25479 100.1 40.1 33547416 33620192 ??  Rp/2   Tue09AM 1755:38.49
> bgpd: route
> _bgpd 8696 31.6  0.0 15800 13240 ??  Sp Tue09AM  626:35.66 bgpd:
> sessio
> _bgpd46603  0.0  0.0 22728 25876 ??  Ip Tue09AM1:29.11 bgpd: rtr
> en
> root 94644  0.0  0.0   196   916 ??  Rp/33:07PM0:00.00 grep bgpd
 
Interesting, the size is around 3GB which is somewhat reasonable.
What surprises me is the high CPU load and time spent in both the RDE and
SE. One of my core routers running since last September has about the same
CPU usage that your box collected in a few days. It seems that there is a
lot of churn.

I'm not sure why the processes gets killed at around 3GB. Feels like you
hit the ulimit. See Stuart's mail about how to look into that.
 
So looking at this output I feel like you somehow created a BGP update
loop where one or more systems are constantly sending UPDATEs to each
other because the moment the update is processed the route decision
changes and flaps back resulting in a withdraw or update.

You can check the 'bgpctl show' and 'bgpctl show nei ' output to
see between which peers many messages are sent. From there on you need to
see which prefixes cause this update storm. Probably some filter rule
causes this.

My assumption is that because of this UPDATE loop the systems slowly kill
each other by pushing more and more updates into various buffers along the
way.

> During the crash, bgpctl show rib mem doesn't work.
> Here is the ps aux | grep bgp output during the crash:
> 
> _bgpd25479  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
> _bgpd46603  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
> _bgpd 8696  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
> root 76428  0.0  0.0   180   772 ??  R/2 3:08PM0:00.00 grep bgpd
> 
> 
> Please note /var/log/messages output:
> 
> Mar 30 15:07:27 bgpgw-004 bgpd[17103]: peer closed imsg connection
> Mar 30 15:07:27 bgpgw-004 bgpd[17103]: main: Lost connection to RDE
> Mar 30 15:07:27 bgpgw-004 bgpd[46603]: peer closed imsg connection
> Mar 30 15:07:27 bgpgw-004 bgpd[46603]: RTR: Lost connection to RDE
> Mar 30 15:07:27 bgpgw-004 bgpd[46603]: peer closed imsg connection
> Mar 30 15:07:27 bgpgw-004 bgpd[46603]: fatal in RTR: Lost connection to
> parent
> Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
> Mar 30 15:07:27 bgpgw-004 bgpd[8696]: SE: Lost connection to RDE
> Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
> Mar 30 15:07:27 bgpgw-004 bgpd[8696]: SE: Lost connection to RDE control
> Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
> Mar 

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 29/03/2022 à 12:10, Claudio Jeker a écrit :

I doubt it is the filters. You run into some sort of memory leak. Please
monitor 'bgpctl show rib mem' output. Also check ps aux | grep bgpd output
to see why and when the memory starts to go up.
With that information it may be possible to figure out where this leak
sits and how to fix it.

Cheers



Hi Claudio,

Please find the output of 'bgpctl show rib mem' just 1 minute before the 
crash:


cat 2022-03-30::15:07:01.mem
RDE memory statistics
909685 IPv4 unicast network entries using 34.7M of memory
272248 IPv6 unicast network entries using 14.5M of memory
   2363169 rib entries using 144M of memory
  14616410 prefix entries using 1.7G of memory
   1539060 BGP path attribute entries using 106M of memory
   and holding 14616410 references
635275 BGP AS-PATH attribute entries using 33.7M of memory
   and holding 1539060 references
 47399 entries for 681150 BGP communities using 15.1M of memory
   and holding 14616410 references
 22139 BGP attributes entries using 865K of memory
   and holding 3436885 references
 22138 BGP attributes using 175K of memory
270121 as-set elements in 249193 tables using 9.7M of memory
452138 prefix-set elements using 19.0M of memory
RIB using 2.1G of memory
Sets using 28.7M of memory

RDE hash statistics
path hash: size 131072, 1539060 entries
min 0 max 31 avg/std-dev = 11.742/3.623
aspath hash: size 131072, 635275 entries
min 0 max 16 avg/std-dev = 4.847/2.123
comm hash: size 16384, 47399 entries
min 0 max 12 avg/std-dev = 2.893/1.622
attr hash: size 16384, 22139 entries
min 0 max 8 avg/std-dev = 1.351/1.084


Here is the output of 'ps aux | grep bgp' one minute before the crash:

_bgpd25479 100.1 40.1 33547416 33620192 ??  Rp/2   Tue09AM 
1755:38.49 bgpd: route
_bgpd 8696 31.6  0.0 15800 13240 ??  Sp Tue09AM  626:35.66 bgpd: 
sessio
_bgpd46603  0.0  0.0 22728 25876 ??  Ip Tue09AM1:29.11 bgpd: 
rtr en

root 94644  0.0  0.0   196   916 ??  Rp/33:07PM0:00.00 grep bgpd


During the crash, bgpctl show rib mem doesn't work.
Here is the ps aux | grep bgp output during the crash:

_bgpd25479  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
_bgpd46603  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
_bgpd 8696  0.0  0.0 0 0 ??  Zp -  0:00.00 (bgpd)
root 76428  0.0  0.0   180   772 ??  R/2 3:08PM0:00.00 grep bgpd


Please note /var/log/messages output:

Mar 30 15:07:27 bgpgw-004 bgpd[17103]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[17103]: main: Lost connection to RDE
Mar 30 15:07:27 bgpgw-004 bgpd[46603]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[46603]: RTR: Lost connection to RDE
Mar 30 15:07:27 bgpgw-004 bgpd[46603]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[46603]: fatal in RTR: Lost connection to 
parent

Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[8696]: SE: Lost connection to RDE
Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[8696]: SE: Lost connection to RDE control
Mar 30 15:07:27 bgpgw-004 bgpd[8696]: peer closed imsg connection
Mar 30 15:07:27 bgpgw-004 bgpd[8696]: SE: Lost connection to parent


Thanks,

Laurent

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Le 29/03/2022 à 12:10, Claudio Jeker a écrit :

I doubt it is the filters. You run into some sort of memory leak. Please
monitor 'bgpctl show rib mem' output. Also check ps aux | grep bgpd output
to see why and when the memory starts to go up.
With that information it may be possible to figure out where this leak
sits and how to fix it.

Cheers


Thanks Claudio, will do and report.

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Stuart Henderson]

On 2022-03-29, Claudio Jeker  wrote:
> On Tue, Mar 29, 2022 at 09:53:56AM +0200, Laurent CARON wrote:
>> Hi,
>> 
>> I'm happily running several OpenBGPd routers (Openbsd 7.0).
>> 
>> After having applied the folloxing filters (to blackhole traffic from
>> certain countries):
>> 
>> include "/etc/bgpd/deny-asn.ru.bgpd"
>> include "/etc/bgpd/deny-asn.by.bgpd"
>> include "/etc/bgpd/deny-asn.ua.bgpd"
>> 
>> 
>> # head /etc/bgpd/deny-asn.ru.bgpd
>> match from any AS 2148 set { localpref 250 nexthop blackhole }
>> match from any AS 2585 set { localpref 250 nexthop blackhole }
>> match from any AS 2587 set { localpref 250 nexthop blackhole }
>> match from any AS 2599 set { localpref 250 nexthop blackhole }
>> match from any AS 2766 set { localpref 250 nexthop blackhole }
>> match from any AS 2848 set { localpref 250 nexthop blackhole }
>> match from any AS 2854 set { localpref 250 nexthop blackhole }
>> match from any AS 2875 set { localpref 250 nexthop blackhole }
>> match from any AS 2878 set { localpref 250 nexthop blackhole }
>> match from any AS 2895 set { localpref 250 nexthop blackhole }
>> 
>> The bgpd daemon crashes every few days with the following:
>> 
>> Mar 21 11:36:54 bgpgw-004 bgpd[76476]: 338 roa-set entries expired
>> Mar 21 12:06:54 bgpgw-004 bgpd[76476]: 36 roa-set entries expired
>> Mar 21 12:11:54 bgpgw-004 bgpd[76476]: 82 roa-set entries expired
>> Mar 21 12:22:36 bgpgw-004 bgpd[99215]: fatal in RDE: prefix_alloc: Cannot
>> allocate memory
>> Mar 21 12:22:36 bgpgw-004 bgpd[65049]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[65049]: main: Lost connection to RDE
>> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: RTR: Lost connection to RDE
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE control
>> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: fatal in RTR: Lost connection to
>> parent
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: Can't send message 61 to RDE, pipe
>> closed
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
>> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to parent
>> ...
>> 
>> Mar 24 06:34:17 bgpgw-004 bgpd[83062]: 17 roa-set entries expired
>> Mar 24 06:54:47 bgpgw-004 bgpd[82782]: fatal in RDE: communities_copy:
>> Cannot allocate memory
>> Mar 24 06:54:47 bgpgw-004 bgpd[99753]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[99753]: main: Lost connection to RDE
>> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: RTR: Lost connection to RDE
>> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: fatal in RTR: Lost connection to
>> parent
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE control
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: Can't send message 61 to RDE, pipe
>> closed
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
>> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to parent
>> ...
>> 
>> Mar 27 13:07:56 bgpgw-004 bgpd[95001]: fatal in RDE: aspath_get: Cannot
>> allocate memory
>> Mar 27 13:07:56 bgpgw-004 bgpd[84816]: peer closed imsg connection
>> Mar 27 13:07:56 bgpgw-004 bgpd[84816]: main: Lost connection to RDE
>> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
>> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: RTR: Lost connection to RDE
>> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
>> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: fatal in RTR: Lost connection to
>> parent
>> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
>> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE
>> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
>> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE control
>> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
>> Mar 27 1

Re: OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Claudio Jeker]

On Tue, Mar 29, 2022 at 09:53:56AM +0200, Laurent CARON wrote:
> Hi,
> 
> I'm happily running several OpenBGPd routers (Openbsd 7.0).
> 
> After having applied the folloxing filters (to blackhole traffic from
> certain countries):
> 
> include "/etc/bgpd/deny-asn.ru.bgpd"
> include "/etc/bgpd/deny-asn.by.bgpd"
> include "/etc/bgpd/deny-asn.ua.bgpd"
> 
> 
> # head /etc/bgpd/deny-asn.ru.bgpd
> match from any AS 2148 set { localpref 250 nexthop blackhole }
> match from any AS 2585 set { localpref 250 nexthop blackhole }
> match from any AS 2587 set { localpref 250 nexthop blackhole }
> match from any AS 2599 set { localpref 250 nexthop blackhole }
> match from any AS 2766 set { localpref 250 nexthop blackhole }
> match from any AS 2848 set { localpref 250 nexthop blackhole }
> match from any AS 2854 set { localpref 250 nexthop blackhole }
> match from any AS 2875 set { localpref 250 nexthop blackhole }
> match from any AS 2878 set { localpref 250 nexthop blackhole }
> match from any AS 2895 set { localpref 250 nexthop blackhole }
> 
> The bgpd daemon crashes every few days with the following:
> 
> Mar 21 11:36:54 bgpgw-004 bgpd[76476]: 338 roa-set entries expired
> Mar 21 12:06:54 bgpgw-004 bgpd[76476]: 36 roa-set entries expired
> Mar 21 12:11:54 bgpgw-004 bgpd[76476]: 82 roa-set entries expired
> Mar 21 12:22:36 bgpgw-004 bgpd[99215]: fatal in RDE: prefix_alloc: Cannot
> allocate memory
> Mar 21 12:22:36 bgpgw-004 bgpd[65049]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[65049]: main: Lost connection to RDE
> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: RTR: Lost connection to RDE
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE control
> Mar 21 12:22:36 bgpgw-004 bgpd[76476]: fatal in RTR: Lost connection to
> parent
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: Can't send message 61 to RDE, pipe
> closed
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
> Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to parent
> ...
> 
> Mar 24 06:34:17 bgpgw-004 bgpd[83062]: 17 roa-set entries expired
> Mar 24 06:54:47 bgpgw-004 bgpd[82782]: fatal in RDE: communities_copy:
> Cannot allocate memory
> Mar 24 06:54:47 bgpgw-004 bgpd[99753]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[99753]: main: Lost connection to RDE
> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: RTR: Lost connection to RDE
> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[83062]: fatal in RTR: Lost connection to
> parent
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE control
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: Can't send message 61 to RDE, pipe
> closed
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
> Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to parent
> ...
> 
> Mar 27 13:07:56 bgpgw-004 bgpd[95001]: fatal in RDE: aspath_get: Cannot
> allocate memory
> Mar 27 13:07:56 bgpgw-004 bgpd[84816]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[84816]: main: Lost connection to RDE
> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: RTR: Lost connection to RDE
> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[3118]: fatal in RTR: Lost connection to
> parent
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE control
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
> Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to parent
> 
> Is my filter too aggressive for bgpd ? Is there a more efficient way to
> write it ?
 
I doubt it is the filters. You run into some sort of memory leak. Please
monitor 'bgpctl show rib mem' output. Also check ps aux | grep bgpd output 
to see why and when the memory starts to go up.
With that information it may be possible to figure out where this leak
sits and how to fix it.

Cheers
-- 
:wq Claudio

OpenBGPd: fatal in RDE: aspath_get: Cannot allocate memory

[Laurent CARON]

Hi,

I'm happily running several OpenBGPd routers (Openbsd 7.0).

After having applied the folloxing filters (to blackhole traffic from 
certain countries):


include "/etc/bgpd/deny-asn.ru.bgpd"
include "/etc/bgpd/deny-asn.by.bgpd"
include "/etc/bgpd/deny-asn.ua.bgpd"


# head /etc/bgpd/deny-asn.ru.bgpd
match from any AS 2148 set { localpref 250 nexthop blackhole }
match from any AS 2585 set { localpref 250 nexthop blackhole }
match from any AS 2587 set { localpref 250 nexthop blackhole }
match from any AS 2599 set { localpref 250 nexthop blackhole }
match from any AS 2766 set { localpref 250 nexthop blackhole }
match from any AS 2848 set { localpref 250 nexthop blackhole }
match from any AS 2854 set { localpref 250 nexthop blackhole }
match from any AS 2875 set { localpref 250 nexthop blackhole }
match from any AS 2878 set { localpref 250 nexthop blackhole }
match from any AS 2895 set { localpref 250 nexthop blackhole }

The bgpd daemon crashes every few days with the following:

Mar 21 11:36:54 bgpgw-004 bgpd[76476]: 338 roa-set entries expired
Mar 21 12:06:54 bgpgw-004 bgpd[76476]: 36 roa-set entries expired
Mar 21 12:11:54 bgpgw-004 bgpd[76476]: 82 roa-set entries expired
Mar 21 12:22:36 bgpgw-004 bgpd[99215]: fatal in RDE: prefix_alloc: 
Cannot allocate memory

Mar 21 12:22:36 bgpgw-004 bgpd[65049]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[65049]: main: Lost connection to RDE
Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[76476]: RTR: Lost connection to RDE
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[76476]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to RDE control
Mar 21 12:22:36 bgpgw-004 bgpd[76476]: fatal in RTR: Lost connection to 
parent
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: Can't send message 61 to RDE, 
pipe closed

Mar 21 12:22:36 bgpgw-004 bgpd[58155]: peer closed imsg connection
Mar 21 12:22:36 bgpgw-004 bgpd[58155]: SE: Lost connection to parent
...

Mar 24 06:34:17 bgpgw-004 bgpd[83062]: 17 roa-set entries expired
Mar 24 06:54:47 bgpgw-004 bgpd[82782]: fatal in RDE: communities_copy: 
Cannot allocate memory

Mar 24 06:54:47 bgpgw-004 bgpd[99753]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[99753]: main: Lost connection to RDE
Mar 24 06:54:47 bgpgw-004 bgpd[83062]: RTR: Lost connection to RDE
Mar 24 06:54:47 bgpgw-004 bgpd[83062]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[83062]: fatal in RTR: Lost connection to 
parent

Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE
Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to RDE control
Mar 24 06:54:47 bgpgw-004 bgpd[40748]: Can't send message 61 to RDE, 
pipe closed

Mar 24 06:54:47 bgpgw-004 bgpd[40748]: peer closed imsg connection
Mar 24 06:54:47 bgpgw-004 bgpd[40748]: SE: Lost connection to parent
...

Mar 27 13:07:56 bgpgw-004 bgpd[95001]: fatal in RDE: aspath_get: Cannot 
allocate memory

Mar 27 13:07:56 bgpgw-004 bgpd[84816]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[84816]: main: Lost connection to RDE
Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[3118]: RTR: Lost connection to RDE
Mar 27 13:07:56 bgpgw-004 bgpd[3118]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[3118]: fatal in RTR: Lost connection to 
parent

Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE
Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to RDE control
Mar 27 13:07:56 bgpgw-004 bgpd[60695]: peer closed imsg connection
Mar 27 13:07:56 bgpgw-004 bgpd[60695]: SE: Lost connection to parent

Is my filter too aggressive for bgpd ? Is there a more efficient way to 
write it ?


Thanks

Laurent

Re: OpenBGPD fatal in RDE: rde_dispatch_imsg_session: imsg_get error: Cannot allocate memory

[Laurent CARON]

Le 30/06/2020 à 11:56, Claudio Jeker a écrit :

Can you check and monitor with ps aux | grep bgpd and or top the VSZ and
RSS of the RDE process. What is the maximum you notice. Also how do you
start bgpd? Make sure the limits from login.conf are actually applied
(using rcctl start should do that while doas bgpd would not).



Hi Claudio,

After restarting bgpd on 2 affected boxed, RAM usage is back to normal.

root 23427  0.0  0.1 79700 88548 ??  S  12:47PM 1:24.88 
/usr/sbin/bgpd
_bgpd    35700  0.8  2.5 2052496 2061292 ??  Sp 12:47PM 24:15.01 
bgpd: route decision engine (bgpd)
_bgpd    29969  0.8  0.0 21536 10684 ??  Sp 12:47PM    6:56.40 bgpd: 
session engine (bgpd)


What else apart from ps aux | grep bgpd can I give you to help 
troubleshoot this issue ?



Thanks

Re: OpenBGPD fatal in RDE: rde_dispatch_imsg_session: imsg_get error: Cannot allocate memory

[Claudio Jeker]

On Tue, Jun 30, 2020 at 10:23:07AM +0200, Laurent CARON wrote:
> Hi,
> 
> 
> I'm running a pretty busy OpenBGPd router (~250 bgp sessions) with 4 IPv4
> and 4 IPv6 full views, plus a few IX sessions.
> 
> 
> # bgpctl show rib mem
> RDE memory statistics
>     820983 IPv4 unicast network entries using 31.3M of memory
>     203228 IPv6 unicast network entries using 10.9M of memory
>    1935802 rib entries using 118M of memory
>    6348318 prefix entries using 775M of memory
>     728103 BGP path attribute entries using 50.0M of memory
>    and holding 6348318 references
>     464633 BGP AS-PATH attribute entries using 22.3M of memory
>    and holding 728103 references
>  29055 entries for 371905 BGP communities using 8.6M of memory
>    and holding 6348318 references
>  18541 BGP attributes entries using 724K of memory
>    and holding 1618379 references
>  18540 BGP attributes using 145K of memory
>  0 as-set elements in 0 tables using 0B of memory
>     64 prefix-set elements using 3.0K of memory
> RIB using 1008M of memory
> Sets using 3.0K of memory
> 
> RDE hash statistics
>     path hash: size 131072, 728103 entries
>     min 0 max 19 avg/std-dev = 5.555/2.268
>     aspath hash: size 131072, 464633 entries
>     min 0 max 17 avg/std-dev = 3.545/1.853
>     comm hash: size 16384, 29055 entries
>     min 0 max 8 avg/std-dev = 1.773/0.925
>     attr hash: size 16384, 18541 entries
>     min 0 max 8 avg/std-dev = 1.132/0.848
> 
> 
> More often than not the BGPd daemon is crashing (although having plenty of
> RAM (80G) on the server) with: /var/log/messages
> 
> fatal in RDE: rde_dispatch_imsg_session: imsg_get error: Cannot allocate
> memory
> 
> fatal in RDE: prefix_alloc: Cannot allocate memory
> 
> fatal in RDE: communities_copy: Cannot allocate memory
> 
> peer closed imsg connection
> main: Lost connection to RDE
> peer closed imsg connection
> SE: Lost connection to RDE
> peer closed imsg connection
> SE: Lost connection to RDE control
> Can't send message 57 to RDE, pipe closed
> last message repeated 12 times
> peer closed imsg connection
> SE: Lost connection to parent
> neighbor A.B.C.D (sas-v4-001): sending notification: Cease, administratively
> down
> 
> 
> :/etc/login.conf:
> 
> default:\
>     :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin
> /usr/local/sbin:\
>     :umask=022:\
>     :datasize-max=768M:\
>     :datasize-cur=768M:\
>     :maxproc-max=256:\
>     :maxproc-cur=128:\
>     :openfiles-max=1024:\
>     :openfiles-cur=512:\
>     :stacksize-cur=4M:\
>     :localcipher=blowfish,a:\
>     :tc=auth-defaults:\
>     :tc=auth-ftp-defaults:
> 
> daemon:\
>     :ignorenologin:\
>     :datasize=infinity:\
>     :maxproc=infinity:\
>     :openfiles-max=1024:\
>     :openfiles-cur=128:\
>     :stacksize-cur=8M:\
>     :localcipher=blowfish,a:\
>     :tc=default:
> 
> bgpd:\
>     :openfiles=512:\
>     :tc=daemon:
> 
> How can I pinpoint the source of the problem ?
> 

Can you check and monitor with ps aux | grep bgpd and or top the VSZ and
RSS of the RDE process. What is the maximum you notice. Also how do you
start bgpd? Make sure the limits from login.conf are actually applied
(using rcctl start should do that while doas bgpd would not).

Cheers
-- 
:wq Claudio

OpenBGPD fatal in RDE: rde_dispatch_imsg_session: imsg_get error: Cannot allocate memory

[Laurent CARON]

Hi,


I'm running a pretty busy OpenBGPd router (~250 bgp sessions) with 4 
IPv4 and 4 IPv6 full views, plus a few IX sessions.



# bgpctl show rib mem
RDE memory statistics
    820983 IPv4 unicast network entries using 31.3M of memory
    203228 IPv6 unicast network entries using 10.9M of memory
   1935802 rib entries using 118M of memory
   6348318 prefix entries using 775M of memory
    728103 BGP path attribute entries using 50.0M of memory
   and holding 6348318 references
    464633 BGP AS-PATH attribute entries using 22.3M of memory
   and holding 728103 references
 29055 entries for 371905 BGP communities using 8.6M of memory
   and holding 6348318 references
 18541 BGP attributes entries using 724K of memory
   and holding 1618379 references
 18540 BGP attributes using 145K of memory
 0 as-set elements in 0 tables using 0B of memory
    64 prefix-set elements using 3.0K of memory
RIB using 1008M of memory
Sets using 3.0K of memory

RDE hash statistics
    path hash: size 131072, 728103 entries
    min 0 max 19 avg/std-dev = 5.555/2.268
    aspath hash: size 131072, 464633 entries
    min 0 max 17 avg/std-dev = 3.545/1.853
    comm hash: size 16384, 29055 entries
    min 0 max 8 avg/std-dev = 1.773/0.925
    attr hash: size 16384, 18541 entries
    min 0 max 8 avg/std-dev = 1.132/0.848


More often than not the BGPd daemon is crashing (although having plenty 
of RAM (80G) on the server) with: /var/log/messages


fatal in RDE: rde_dispatch_imsg_session: imsg_get error: Cannot allocate 
memory


fatal in RDE: prefix_alloc: Cannot allocate memory

fatal in RDE: communities_copy: Cannot allocate memory

peer closed imsg connection
main: Lost connection to RDE
peer closed imsg connection
SE: Lost connection to RDE
peer closed imsg connection
SE: Lost connection to RDE control
Can't send message 57 to RDE, pipe closed
last message repeated 12 times
peer closed imsg connection
SE: Lost connection to parent
neighbor A.B.C.D (sas-v4-001): sending notification: Cease, 
administratively down



:/etc/login.conf:

default:\
    :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin 
/usr/local/bin /usr/local/sbin:\

    :umask=022:\
    :datasize-max=768M:\
    :datasize-cur=768M:\
    :maxproc-max=256:\
    :maxproc-cur=128:\
    :openfiles-max=1024:\
    :openfiles-cur=512:\
    :stacksize-cur=4M:\
    :localcipher=blowfish,a:\
    :tc=auth-defaults:\
    :tc=auth-ftp-defaults:

daemon:\
    :ignorenologin:\
    :datasize=infinity:\
    :maxproc=infinity:\
    :openfiles-max=1024:\
    :openfiles-cur=128:\
    :stacksize-cur=8M:\
    :localcipher=blowfish,a:\
    :tc=default:

bgpd:\
    :openfiles=512:\
    :tc=daemon:

How can I pinpoint the source of the problem ?


Thanks

Re: "Cannot allocate memory" error when memory is enough

[Stuart Henderson]

On 2018-07-09, Nan Xiao  wrote:
> Hi Stuart,
>
> This is my kernel information:
>
> $ sysctl -n kern.version | head -1
> OpenBSD 6.3 (GENERIC.MP) #4: Sun Jun 17 11:22:20 CEST 2018
>
> I remember I used to build -current code before. So is it possible
> that will mess up my system?

Anything that you installed from packages while running -current has a
higher chance than usual of being broken.

I would suggest reinstalling all packages and forcing to make sure you
have the correct version:

PKG_PATH=http://some.mirror.example/pub/OpenBSD/6.3/packages/amd64/ pkg_add -D 
downgrade -D installed -u

You may also have newer libraries in /usr/lib or /usr/X11R6/lib that could
cause problems too, I would remove files with dates that don't match the
release.

Generally going backwards from -current to release is not recommended
unless you know how to clean things up.

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi Stuart,

This is my kernel information:

$ sysctl -n kern.version | head -1
OpenBSD 6.3 (GENERIC.MP) #4: Sun Jun 17 11:22:20 CEST 2018

I remember I used to build -current code before. So is it possible
that will mess up my system?

Thanks!
Best Regards
Nan Xiao


On Sun, Jul 8, 2018 at 10:17 PM, Stuart Henderson  wrote:
> On 2018/07/08 21:56, Nan Xiao wrote:
>> Hi Stuart,
>>
>> Thanks for your reply!
>>
>> $ cat /etc/installurl
>> https://ftp.openbsd.org/pub/OpenBSD
>>
>> Does my /etc/installurl will always fetch -current?
>>
>> Thanks in advance!
>> Best Regards
>> Nan Xiao
>>
>>
>> On Sun, Jul 8, 2018 at 7:33 AM, Stuart Henderson  
>> wrote:
>> > On 2018-07-07, Tom Smyth  wrote:
>> >> Hi otto
>> >>
>> >> I must check that last time i read man page  it was related it was for
>> >> syspatch
>> >
>> > /etc/installurl is used if present, pkg_add selects /snapshots/ or /6.3/
>> > etc depending on the kernel version string.
>> >
>> > If set, PKG_PATH overrides /etc/installurl, and this takes either a short
>> > hostname (in which case the same rules apply for version string), or a
>> > full path (either as a static string or with placeholders replaced by
>> > arch/version).
>> >
>> > That is why I asked if PKG_PATH was set.
>> >
>> >
>
> It depends on what kernel you are running.
>
> $ sysctl -n kern.version | head -1
> OpenBSD 6.3-current (GENERIC.MP) #6: Thu Jul  5 21:40:47 BST 2018
>
> -> fetches -current packages
>
> $ sysctl -n kern.version | head -1
> OpenBSD 6.3 (GENERIC.MP) #107: Sat Mar 24 14:21:59 MDT 2018
>
> -> fetches release packages
>
> Also I would recommend against using ftp.openbsd.org, pick a local mirror
> or if using releases one of the CDNs instead ..
>

Re: "Cannot allocate memory" error when memory is enough

[Stuart Henderson]

On 2018/07/08 21:56, Nan Xiao wrote:
> Hi Stuart,
> 
> Thanks for your reply!
> 
> $ cat /etc/installurl
> https://ftp.openbsd.org/pub/OpenBSD
> 
> Does my /etc/installurl will always fetch -current?
> 
> Thanks in advance!
> Best Regards
> Nan Xiao
> 
> 
> On Sun, Jul 8, 2018 at 7:33 AM, Stuart Henderson  wrote:
> > On 2018-07-07, Tom Smyth  wrote:
> >> Hi otto
> >>
> >> I must check that last time i read man page  it was related it was for
> >> syspatch
> >
> > /etc/installurl is used if present, pkg_add selects /snapshots/ or /6.3/
> > etc depending on the kernel version string.
> >
> > If set, PKG_PATH overrides /etc/installurl, and this takes either a short
> > hostname (in which case the same rules apply for version string), or a
> > full path (either as a static string or with placeholders replaced by
> > arch/version).
> >
> > That is why I asked if PKG_PATH was set.
> >
> >

It depends on what kernel you are running.

$ sysctl -n kern.version | head -1
OpenBSD 6.3-current (GENERIC.MP) #6: Thu Jul  5 21:40:47 BST 2018

-> fetches -current packages

$ sysctl -n kern.version | head -1
OpenBSD 6.3 (GENERIC.MP) #107: Sat Mar 24 14:21:59 MDT 2018

-> fetches release packages

Also I would recommend against using ftp.openbsd.org, pick a local mirror
or if using releases one of the CDNs instead ..

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi Stuart,

Thanks for your reply!

$ cat /etc/installurl
https://ftp.openbsd.org/pub/OpenBSD

Does my /etc/installurl will always fetch -current?

Thanks in advance!
Best Regards
Nan Xiao


On Sun, Jul 8, 2018 at 7:33 AM, Stuart Henderson  wrote:
> On 2018-07-07, Tom Smyth  wrote:
>> Hi otto
>>
>> I must check that last time i read man page  it was related it was for
>> syspatch
>
> /etc/installurl is used if present, pkg_add selects /snapshots/ or /6.3/
> etc depending on the kernel version string.
>
> If set, PKG_PATH overrides /etc/installurl, and this takes either a short
> hostname (in which case the same rules apply for version string), or a
> full path (either as a static string or with placeholders replaced by
> arch/version).
>
> That is why I asked if PKG_PATH was set.
>
>

Re: "Cannot allocate memory" error when memory is enough

[Stuart Henderson]

On 2018-07-07, Tom Smyth  wrote:
> Hi otto
>
> I must check that last time i read man page  it was related it was for
> syspatch

/etc/installurl is used if present, pkg_add selects /snapshots/ or /6.3/
etc depending on the kernel version string.

If set, PKG_PATH overrides /etc/installurl, and this takes either a short
hostname (in which case the same rules apply for version string), or a
full path (either as a static string or with placeholders replaced by
arch/version).

That is why I asked if PKG_PATH was set.

Re: "Cannot allocate memory" error when memory is enough

[Tom Smyth]

Hello
yeah the installurl functionality appears to have been added to
after 6.1

thanks Otto

On 7 July 2018 at 17:10, Tom Smyth  wrote:
> Hi otto
>
> I must check that last time i read man page  it was related it was for
> syspatch
>
>
> Thanks
>
>
>
> On Sat 7 Jul 2018, 17:07 Otto Moerbeek,  wrote:
>>
>> On Sat, Jul 07, 2018 at 03:52:44PM +0100, Tom Smyth wrote:
>>
>> > Hello Nan,
>> > you need to set the PKG_path as Stuart suggested to install packages
>> > using
>> >
>> > pkg_add command
>> >
>> > for example to use fastly cdn  mirror you would set your pkg_path
>> > variable as
>> > shown below
>> >
>> > export PKG_PATH=https://fastly.cdn.openbsd.org/pub/OpenBSD/$(uname
>> > -r)/packages/$(uname -p)/
>> >
>> > /etc/installurl  file is for the syspatch utility to install binary
>> > patches for the release
>>
>> Wrong. installurl *is* used by pkg_add. Read the man page.
>>
>> -Otto
>> >
>> >
>> >
>> > if you are using current ... Packages  are less straight forward as
>> > packages and current
>> > are modified regularly updated
>> > so if you go to install a package from today on a current snapshot from
>> > 1 week
>> > ago it is not guaranteed to work
>> >
>> > i believe there are few ways around this
>> > 1) install the packages that you expect to use on current as soon as
>> > possible
>> > after install
>> > 2) if you must use that version of current for some reason ... when
>> > downloading
>> > the current install iso / install image, download the contents of the
>> > entire package
>> > directory for current also. (on the same day )  your millage on this
>> > will vary  depending
>> > on what work is being done on the tree at the time
>> >
>> > and if anyone else uses a  better way id love to know also
>> >
>> >
>> > I hope this helps
>> >
>> >
>> > On 7 July 2018 at 15:26, Nan Xiao  wrote:
>> > > HI Stuart,
>> > >
>> > > Thanks for your reply!
>> > >
>> > > I don't set PKG_PATH environment.
>> > >
>> > > Yes, it seems I installed the current package:
>> > > $ ls -l /usr/local/bin/egdb
>> > > -rwxr-xr-x  1 root  bin  10040410 Jun 28 19:03 /usr/local/bin/egdb
>> > >
>> > > But I can't remember what I have done. I can only remember I modified
>> > > /etc/installurl before.
>> > >
>> > > Thanks!
>> > > Best Regards
>> > > Nan Xiao
>> > >
>> > >
>> > > On Sat, Jul 7, 2018 at 8:23 PM, Stuart Henderson 
>> > > wrote:
>> > >> On 2018-07-07, Nan Xiao  wrote:
>> > >>> Hi Philip,
>> > >>>
>> > >>> Thanks very much for your time and effort!
>> > >>>
>> > >>> The following is the command output:
>> > >>>
>> > >>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print
>> > >>> ($5+0)/1024}'
>> > >>> 88.4844
>> > >>>
>> > >>> The same as yours.
>> > >>
>> > >> That is from -current.
>> > >>
>> > >>> Honestly, I am not sure whether the package is for -stable or
>> > >>> -current.
>> > >>>
>> > >>> After installation, I just add following config in /etc/installurl:
>> > >>> $ cat /etc/installurl
>> > >>> https://ftp.openbsd.org/pub/OpenBSD
>> > >>>
>> > >>> Then I use "pkg_add -u" sometimes. Doesn't
>> > >>> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
>> > >>> packages? Or I miss something?
>> > >>
>> > >> Do you have PKG_PATH set in the environment?
>> > >>
>> > >> Whatever the reason, it looks like you have a -current version of the
>> > >> gdb package installed.
>> > >>
>> > >> The one in 6.3 should look like this:
>> > >>
>> > >> $ ls -l /usr/local/bin/egdb
>> > >> -rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb
>> > >>
>> > >> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
>> > >> 0.0078125
>> > >>
>> > >>
>> > >
>> >
>> >
>> >
>> > --
>> > Kindest regards,
>> > Tom Smyth
>> >
>> > Mobile: +353 87 6193172
>> > The information contained in this E-mail is intended only for the
>> > confidential use of the named recipient. If the reader of this message
>> > is not the intended recipient or the person responsible for
>> > delivering it to the recipient, you are hereby notified that you have
>> > received this communication in error and that any review,
>> > dissemination or copying of this communication is strictly prohibited.
>> > If you have received this in error, please notify the sender
>> > immediately by telephone at the number above and erase the message
>> > You are requested to carry out your own virus check before
>> > opening any attachment.



-- 
Kindest regards,
Tom Smyth

Mobile: +353 87 6193172
The information contained in this E-mail is intended only for the
confidential use of the named recipient. If the reader of this message
is not the intended recipient or the person responsible for
delivering it to the recipient, you are hereby notified that you have
received this communication in error and that any review,
dissemination or copying of this communication is strictly prohibited.
If you have received this in error, please notify the sender
immediately by telephone at the number above and erase the message
You are 

Re: "Cannot allocate memory" error when memory is enough

[Tom Smyth]

Hi otto

I must check that last time i read man page  it was related it was for
syspatch


Thanks



On Sat 7 Jul 2018, 17:07 Otto Moerbeek,  wrote:

> On Sat, Jul 07, 2018 at 03:52:44PM +0100, Tom Smyth wrote:
>
> > Hello Nan,
> > you need to set the PKG_path as Stuart suggested to install packages
> using
> >
> > pkg_add command
> >
> > for example to use fastly cdn  mirror you would set your pkg_path
> variable as
> > shown below
> >
> > export PKG_PATH=https://fastly.cdn.openbsd.org/pub/OpenBSD/$(uname
> > -r)/packages/$(uname -p)/
> >
> > /etc/installurl  file is for the syspatch utility to install binary
> > patches for the release
>
> Wrong. installurl *is* used by pkg_add. Read the man page.
>
> -Otto
> >
> >
> >
> > if you are using current ... Packages  are less straight forward as
> > packages and current
> > are modified regularly updated
> > so if you go to install a package from today on a current snapshot from
> 1 week
> > ago it is not guaranteed to work
> >
> > i believe there are few ways around this
> > 1) install the packages that you expect to use on current as soon as
> possible
> > after install
> > 2) if you must use that version of current for some reason ... when
> downloading
> > the current install iso / install image, download the contents of the
> > entire package
> > directory for current also. (on the same day )  your millage on this
> > will vary  depending
> > on what work is being done on the tree at the time
> >
> > and if anyone else uses a  better way id love to know also
> >
> >
> > I hope this helps
> >
> >
> > On 7 July 2018 at 15:26, Nan Xiao  wrote:
> > > HI Stuart,
> > >
> > > Thanks for your reply!
> > >
> > > I don't set PKG_PATH environment.
> > >
> > > Yes, it seems I installed the current package:
> > > $ ls -l /usr/local/bin/egdb
> > > -rwxr-xr-x  1 root  bin  10040410 Jun 28 19:03 /usr/local/bin/egdb
> > >
> > > But I can't remember what I have done. I can only remember I modified
> > > /etc/installurl before.
> > >
> > > Thanks!
> > > Best Regards
> > > Nan Xiao
> > >
> > >
> > > On Sat, Jul 7, 2018 at 8:23 PM, Stuart Henderson 
> wrote:
> > >> On 2018-07-07, Nan Xiao  wrote:
> > >>> Hi Philip,
> > >>>
> > >>> Thanks very much for your time and effort!
> > >>>
> > >>> The following is the command output:
> > >>>
> > >>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
> > >>> 88.4844
> > >>>
> > >>> The same as yours.
> > >>
> > >> That is from -current.
> > >>
> > >>> Honestly, I am not sure whether the package is for -stable or
> -current.
> > >>>
> > >>> After installation, I just add following config in /etc/installurl:
> > >>> $ cat /etc/installurl
> > >>> https://ftp.openbsd.org/pub/OpenBSD
> > >>>
> > >>> Then I use "pkg_add -u" sometimes. Doesn't
> > >>> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
> > >>> packages? Or I miss something?
> > >>
> > >> Do you have PKG_PATH set in the environment?
> > >>
> > >> Whatever the reason, it looks like you have a -current version of the
> > >> gdb package installed.
> > >>
> > >> The one in 6.3 should look like this:
> > >>
> > >> $ ls -l /usr/local/bin/egdb
> > >> -rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb
> > >>
> > >> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
> > >> 0.0078125
> > >>
> > >>
> > >
> >
> >
> >
> > --
> > Kindest regards,
> > Tom Smyth
> >
> > Mobile: +353 87 6193172
> > The information contained in this E-mail is intended only for the
> > confidential use of the named recipient. If the reader of this message
> > is not the intended recipient or the person responsible for
> > delivering it to the recipient, you are hereby notified that you have
> > received this communication in error and that any review,
> > dissemination or copying of this communication is strictly prohibited.
> > If you have received this in error, please notify the sender
> > immediately by telephone at the number above and erase the message
> > You are requested to carry out your own virus check before
> > opening any attachment.
>

Re: "Cannot allocate memory" error when memory is enough

[Otto Moerbeek]

On Sat, Jul 07, 2018 at 03:52:44PM +0100, Tom Smyth wrote:

> Hello Nan,
> you need to set the PKG_path as Stuart suggested to install packages using
> 
> pkg_add command
> 
> for example to use fastly cdn  mirror you would set your pkg_path variable as
> shown below
> 
> export PKG_PATH=https://fastly.cdn.openbsd.org/pub/OpenBSD/$(uname
> -r)/packages/$(uname -p)/
> 
> /etc/installurl  file is for the syspatch utility to install binary
> patches for the release

Wrong. installurl *is* used by pkg_add. Read the man page.

-Otto
> 
> 
> 
> if you are using current ... Packages  are less straight forward as
> packages and current
> are modified regularly updated
> so if you go to install a package from today on a current snapshot from 1 week
> ago it is not guaranteed to work
> 
> i believe there are few ways around this
> 1) install the packages that you expect to use on current as soon as possible
> after install
> 2) if you must use that version of current for some reason ... when 
> downloading
> the current install iso / install image, download the contents of the
> entire package
> directory for current also. (on the same day )  your millage on this
> will vary  depending
> on what work is being done on the tree at the time
> 
> and if anyone else uses a  better way id love to know also
> 
> 
> I hope this helps
> 
> 
> On 7 July 2018 at 15:26, Nan Xiao  wrote:
> > HI Stuart,
> >
> > Thanks for your reply!
> >
> > I don't set PKG_PATH environment.
> >
> > Yes, it seems I installed the current package:
> > $ ls -l /usr/local/bin/egdb
> > -rwxr-xr-x  1 root  bin  10040410 Jun 28 19:03 /usr/local/bin/egdb
> >
> > But I can't remember what I have done. I can only remember I modified
> > /etc/installurl before.
> >
> > Thanks!
> > Best Regards
> > Nan Xiao
> >
> >
> > On Sat, Jul 7, 2018 at 8:23 PM, Stuart Henderson  
> > wrote:
> >> On 2018-07-07, Nan Xiao  wrote:
> >>> Hi Philip,
> >>>
> >>> Thanks very much for your time and effort!
> >>>
> >>> The following is the command output:
> >>>
> >>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
> >>> 88.4844
> >>>
> >>> The same as yours.
> >>
> >> That is from -current.
> >>
> >>> Honestly, I am not sure whether the package is for -stable or -current.
> >>>
> >>> After installation, I just add following config in /etc/installurl:
> >>> $ cat /etc/installurl
> >>> https://ftp.openbsd.org/pub/OpenBSD
> >>>
> >>> Then I use "pkg_add -u" sometimes. Doesn't
> >>> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
> >>> packages? Or I miss something?
> >>
> >> Do you have PKG_PATH set in the environment?
> >>
> >> Whatever the reason, it looks like you have a -current version of the
> >> gdb package installed.
> >>
> >> The one in 6.3 should look like this:
> >>
> >> $ ls -l /usr/local/bin/egdb
> >> -rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb
> >>
> >> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
> >> 0.0078125
> >>
> >>
> >
> 
> 
> 
> -- 
> Kindest regards,
> Tom Smyth
> 
> Mobile: +353 87 6193172
> The information contained in this E-mail is intended only for the
> confidential use of the named recipient. If the reader of this message
> is not the intended recipient or the person responsible for
> delivering it to the recipient, you are hereby notified that you have
> received this communication in error and that any review,
> dissemination or copying of this communication is strictly prohibited.
> If you have received this in error, please notify the sender
> immediately by telephone at the number above and erase the message
> You are requested to carry out your own virus check before
> opening any attachment.

Re: "Cannot allocate memory" error when memory is enough

[Tom Smyth]

Hello Nan,
you need to set the PKG_path as Stuart suggested to install packages using

pkg_add command

for example to use fastly cdn  mirror you would set your pkg_path variable as
shown below

export PKG_PATH=https://fastly.cdn.openbsd.org/pub/OpenBSD/$(uname
-r)/packages/$(uname -p)/

/etc/installurl  file is for the syspatch utility to install binary
patches for the release



if you are using current ... Packages  are less straight forward as
packages and current
are modified regularly updated
so if you go to install a package from today on a current snapshot from 1 week
ago it is not guaranteed to work

i believe there are few ways around this
1) install the packages that you expect to use on current as soon as possible
after install
2) if you must use that version of current for some reason ... when downloading
the current install iso / install image, download the contents of the
entire package
directory for current also. (on the same day )  your millage on this
will vary  depending
on what work is being done on the tree at the time

and if anyone else uses a  better way id love to know also


I hope this helps


On 7 July 2018 at 15:26, Nan Xiao  wrote:
> HI Stuart,
>
> Thanks for your reply!
>
> I don't set PKG_PATH environment.
>
> Yes, it seems I installed the current package:
> $ ls -l /usr/local/bin/egdb
> -rwxr-xr-x  1 root  bin  10040410 Jun 28 19:03 /usr/local/bin/egdb
>
> But I can't remember what I have done. I can only remember I modified
> /etc/installurl before.
>
> Thanks!
> Best Regards
> Nan Xiao
>
>
> On Sat, Jul 7, 2018 at 8:23 PM, Stuart Henderson  wrote:
>> On 2018-07-07, Nan Xiao  wrote:
>>> Hi Philip,
>>>
>>> Thanks very much for your time and effort!
>>>
>>> The following is the command output:
>>>
>>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
>>> 88.4844
>>>
>>> The same as yours.
>>
>> That is from -current.
>>
>>> Honestly, I am not sure whether the package is for -stable or -current.
>>>
>>> After installation, I just add following config in /etc/installurl:
>>> $ cat /etc/installurl
>>> https://ftp.openbsd.org/pub/OpenBSD
>>>
>>> Then I use "pkg_add -u" sometimes. Doesn't
>>> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
>>> packages? Or I miss something?
>>
>> Do you have PKG_PATH set in the environment?
>>
>> Whatever the reason, it looks like you have a -current version of the
>> gdb package installed.
>>
>> The one in 6.3 should look like this:
>>
>> $ ls -l /usr/local/bin/egdb
>> -rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb
>>
>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
>> 0.0078125
>>
>>
>



-- 
Kindest regards,
Tom Smyth

Mobile: +353 87 6193172
The information contained in this E-mail is intended only for the
confidential use of the named recipient. If the reader of this message
is not the intended recipient or the person responsible for
delivering it to the recipient, you are hereby notified that you have
received this communication in error and that any review,
dissemination or copying of this communication is strictly prohibited.
If you have received this in error, please notify the sender
immediately by telephone at the number above and erase the message
You are requested to carry out your own virus check before
opening any attachment.

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

HI Stuart,

Thanks for your reply!

I don't set PKG_PATH environment.

Yes, it seems I installed the current package:
$ ls -l /usr/local/bin/egdb
-rwxr-xr-x  1 root  bin  10040410 Jun 28 19:03 /usr/local/bin/egdb

But I can't remember what I have done. I can only remember I modified
/etc/installurl before.

Thanks!
Best Regards
Nan Xiao


On Sat, Jul 7, 2018 at 8:23 PM, Stuart Henderson  wrote:
> On 2018-07-07, Nan Xiao  wrote:
>> Hi Philip,
>>
>> Thanks very much for your time and effort!
>>
>> The following is the command output:
>>
>> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
>> 88.4844
>>
>> The same as yours.
>
> That is from -current.
>
>> Honestly, I am not sure whether the package is for -stable or -current.
>>
>> After installation, I just add following config in /etc/installurl:
>> $ cat /etc/installurl
>> https://ftp.openbsd.org/pub/OpenBSD
>>
>> Then I use "pkg_add -u" sometimes. Doesn't
>> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
>> packages? Or I miss something?
>
> Do you have PKG_PATH set in the environment?
>
> Whatever the reason, it looks like you have a -current version of the
> gdb package installed.
>
> The one in 6.3 should look like this:
>
> $ ls -l /usr/local/bin/egdb
> -rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb
>
> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
> 0.0078125
>
>

Re: "Cannot allocate memory" error when memory is enough

[Stuart Henderson]

On 2018-07-07, Nan Xiao  wrote:
> Hi Philip,
>
> Thanks very much for your time and effort!
>
> The following is the command output:
>
> $ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
> 88.4844
>
> The same as yours.

That is from -current.

> Honestly, I am not sure whether the package is for -stable or -current.
>
> After installation, I just add following config in /etc/installurl:
> $ cat /etc/installurl
> https://ftp.openbsd.org/pub/OpenBSD
>
> Then I use "pkg_add -u" sometimes. Doesn't
> https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
> packages? Or I miss something?

Do you have PKG_PATH set in the environment?

Whatever the reason, it looks like you have a -current version of the
gdb package installed.

The one in 6.3 should look like this:

$ ls -l /usr/local/bin/egdb 
   
-rwxr-xr-x  1 root  bin  9452688 Mar 28 00:40 /usr/local/bin/egdb

$ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print($5+0)/1024}'
0.0078125

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi Philip,

Thanks very much for your time and effort!

The following is the command output:

$ readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print ($5+0)/1024}'
88.4844

The same as yours.

Honestly, I am not sure whether the package is for -stable or -current.

After installation, I just add following config in /etc/installurl:
$ cat /etc/installurl
https://ftp.openbsd.org/pub/OpenBSD

Then I use "pkg_add -u" sometimes. Doesn't
https://ftp.openbsd.org/pub/OpenBSD guarantee it selects correct
packages? Or I miss something?

Thanks very much in advance!
Best Regards
Nan Xiao


On Sat, Jul 7, 2018 at 1:08 PM, Philip Guenther  wrote:
> On Wed, Jul 4, 2018 at 6:31 PM Nan Xiao  wrote:
>>
>> Thanks very much for your time and patience. I run "syspatch" command
>> regularly, so it should be 6.3-stable.
>>
>>
>> My full dmesg output is here:
>
> ...
> Okay, nothing weird in there.
>
>
>> And full ouput of "vmstat -m":
>
>
> Nothing stands out in that output either, with nothing showing failures or
> consuming much more than might be expected.
>
> So, I'm back to my theory that the programs that are failing to run for you
> are from packages built for -current and not -stable and have
> PT_OPENBSD_RANDOMIZE segments larger than are permitted by -stable.
>
> For example, the gdb-7.12.1p2 package in -current has an 88kB
> PT_OPENBSD_RANDOMIZE segment:
>
> : morgaine; readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print
> ($5+0)/1024}
> 88.4844
> : morgaine;
>
> That's bigger than what a -stable kernel will permit.
>
> So, what's the output of that command for the egdb binary that fails for
> you, and how confident are you that it's from a -stable package and not a
> -current package?
>
>
> Philip Guenther
>

Re: "Cannot allocate memory" error when memory is enough

[Philip Guenther]

On Wed, Jul 4, 2018 at 6:31 PM Nan Xiao  wrote:

> Thanks very much for your time and patience. I run "syspatch" command
> regularly, so it should be 6.3-stable.
>

> My full dmesg output is here:
>
...
Okay, nothing weird in there.


And full ouput of "vmstat -m":
>

Nothing stands out in that output either, with nothing showing failures or
consuming much more than might be expected.

So, I'm back to my theory that the programs that are failing to run for you
are from packages built for -current and not -stable and have
PT_OPENBSD_RANDOMIZE segments larger than are permitted by -stable.

For example, the gdb-7.12.1p2 package in -current has an 88kB
PT_OPENBSD_RANDOMIZE segment:

: morgaine; readelf -Wl /usr/local/bin/egdb | awk '/RANDOM/{print
($5+0)/1024}
88.4844
: morgaine;

That's bigger than what a -stable kernel will permit.

So, what's the output of that command for the egdb binary that fails for
you, and how confident are you that it's from a -stable package and not a
-current package?


Philip Guenther

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi Philip,

Thanks very much for your time and patience. I run "syspatch" command
regularly, so it should be 6.3-stable.

My full dmesg output is here:

OpenBSD 6.3 (RAMDISK_CD) #98: Sat Mar 24 14:26:39 MDT 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 4228214784 (4032MB)
avail mem = 4096286720 (3906MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe1840 (41 entries)
bios0: vendor FUJITSU // Phoenix Technologies Ltd. version "Version
1.06" date 01/16/2009
bios0: FUJITSU LifeBook T5010
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP HPET MCFG SSDT SSDT APIC BOOT SLIC SSDT
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz, 2527.30 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,XSAVE,NXE,LONG,LAHF,PERF,SENSOR,MELTDOWN
cpu0: 3MB 64b/line 8-way L2 cache
cpu0: apic clock running at 266MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2.1.3, IBE
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P2)
acpiprt2 at acpi0: bus 8 (RP01)
acpiprt3 at acpi0: bus 16 (RP02)
acpiprt4 at acpi0: bus 24 (RP03)
acpiprt5 at acpi0: bus -1 (RP05)
acpiprt6 at acpi0: bus 56 (PCIB)
acpiec0 at acpi0
acpicpu at acpi0 not configured
acpitz at acpi0 not configured
acpitz at acpi0 not configured
"FUJ02BF" at acpi0 not configured
"FUJ02E5" at acpi0 not configured
"FUJ02B1" at acpi0 not configured
"SYN1F01" at acpi0 not configured
"FUJ02E3" at acpi0 not configured
"ACPI0003" at acpi0 not configured
"PNP0C0A" at acpi0 not configured
"PNP0C0A" at acpi0 not configured
"PNP0C0D" at acpi0 not configured
"PNP0C0C" at acpi0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel GM45 Host" rev 0x07
vga1 at pci0 dev 2 function 0 "Intel GM45 Video" rev 0x07
wsdisplay1 at vga1 mux 1: console (80x25, vt100 emulation)
"Intel GM45 Video" rev 0x07 at pci0 dev 2 function 1 not configured
em0 at pci0 dev 25 function 0 "Intel ICH9 IGP M AMT" rev 0x03: msi,
address 00:23:26:5e:36:bc
uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x03: apic 2 int 16
uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x03: apic 2 int 17
uhci2 at pci0 dev 26 function 2 "Intel 82801I USB" rev 0x03: apic 2 int 18
ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x03: apic 2 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
"Intel 82801I HD Audio" rev 0x03 at pci0 dev 27 function 0 not configured
ppb0 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x03: msi
pci1 at ppb0 bus 8
ppb1 at pci0 dev 28 function 1 "Intel 82801I PCIE" rev 0x03: msi
pci2 at ppb1 bus 16
ppb2 at pci0 dev 28 function 2 "Intel 82801I PCIE" rev 0x03: msi
pci3 at ppb2 bus 24
iwn0 at pci3 dev 0 function 0 "Intel WiFi Link 5300" rev 0x00: msi,
MIMO 3T3R, MoW, address 00:21:6a:4f:20:5a
uhci3 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x03: apic 2 int 23
uhci4 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x03: apic 2 int 19
uhci5 at pci0 dev 29 function 2 "Intel 82801I USB" rev 0x03: apic 2 int 18
ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x03: apic 2 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev
2.00/1.00 addr 1
ppb3 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0x93
pci4 at ppb3 bus 56
cbb0 at pci4 dev 3 function 0 "O2 Micro OZ711SP1 CardBus" rev 0x01:
apic 2 int 17
cbb1 at pci4 dev 3 function 1 "O2 Micro OZ711SP1 CardBus" rev 0x01:
apic 2 int 17
sdhc0 at pci4 dev 3 function 2 "O2 Micro OZ711MP1 SDHC" rev 0x02: apic 2 int 17
sdhc0: SDHC 3.0, 33 MHz base clock
sdmmc0 at sdhc0: 4-bit
"O2 Micro OZ711MP1 XDHC" rev 0x01 at pci4 dev 3 function 3 not configured
"O2 Micro Firewire" rev 0x02 at pci4 dev 3 function 4 not configured
cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 57 device 0 cacheline 0x0, lattimer 0x20
pcmcia0 at cardslot0
cardslot1 at cbb1 slot 1 flags 0
cardbus1 at cardslot1: bus 57 device 0 cacheline 0x0, lattimer 0x20
pcmcia1 at cardslot1
"Intel 82801IEM LPC" rev 0x03 at pci0 dev 31 function 0 not configured
ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x03: msi, AHCI 1.2
ahci0: port 0: 3.0Gb/s
ahci0: port 1: 1.5Gb/s
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0:  SCSI3
0/direct fixed naa.50e043a28e8c
sd0: 305245MB, 512 bytes/sector, 625142448 sectors
cd0 at scsibus0 targ 1 lun 0:  ATAPI
5/cdrom removable
"Intel 82801I SMBus" rev 0x03 at pci0 dev 31 function 3 not configured
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 configuration 1 interface 0 "Intel UHCI root hub" rev
1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 configuration 1 interface 0 

Re: "Cannot allocate memory" error when memory is enough

[Philip Guenther]

On Wed, Jul 4, 2018 at 12:57 AM Nan Xiao  wrote:

> My OS is 6.3. I already use "pkg_add -u" to upgrade all installed
> packages. cmake and egdb are are installed by "pkg_add", not compiled
> by me.
>

You don't mention -release, or -stable, or -current, which is utterly
critical: 6.3-release and 6.3-stable are not guaranteed to run -current
packages, and ditto for even merely an older -current kernel+base vs fresh
-current packages.  Your messages continue to lack these critical details,
which is why we tell everyone to *include your full dmesg output*.  That
would have instantly answered what version you were running and how out of
date (or not) it is!



>  "vmstat -m" gives some information:
>
> $ vmstat -m
>

...but you trimmed out most of what would show failures or odd consumption
patterns.


It seems kernel dynamic memory is run out, and devbuf and temp consume
> most of the space.
>

What I saw in the output doesn't indicate that.


Philip Guenther

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

HI Philip,

Thanks very much for your detailed explanation!

My OS is 6.3. I already use "pkg_add -u" to upgrade all installed
packages. cmake and egdb are are installed by "pkg_add", not compiled
by me.

 "vmstat -m" gives some information:

$ vmstat -m
Memory statistics by bucket size
Size   In Use   Free   Requests  HighWater  Couldfree
  16  752   283226592131280  3
  32  482   1054 691465 640 99
  64  622   41141023682 320 191519
 128 4496560   27265523 1603596104
 256  164364 107121  80  19059
 512  387197  63372  40  18454
1024 1507  5 125086  20  0
2048   36  4   2193  10  0
4096  555  1  80673   5  0
8192  207  1450   5  0
   16384   10  0 15   5  0
   327689  0  29163   5  0
   655369  0   21946371   5  0
  2621443  0  3   5  0
  5242882  0  2   5  0

..

Memory statistics by type   Type  Kern
  Type InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
devbuf  2991  7220K   7220K 78644K   1463470 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,262144,524288
..
   dirhash   678   130K239K 78644K183870 0
16,32,64,128,256,512
  ..
  ttys   408  1724K   1724K 78644K  4080 0
512,1024,4096,8192
  ..
   VM swap 7   299K299K 78644K70 0
16,64,2048,262144
  UVM amap   29512K441K 78644K  43028130 0
16,32,64,128,256,512
  ..
  temp54  2082K   2211K 78644K 228087350 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,524288
 ..
   DRM   275   114K116K 78644K 14100 0
16,32,64,128,256,512,1024,2048,4096,16384

Memory Totals:  In UseFreeRequests
12378K619K53994333
Memory resource pool statistics
NameSize Requests FailInUse Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
phpool   112849530 4236   125 1   124   124 0 80
extentpl  40  1260   48 1 0 1 1 0 80
pmappl   192662210   34   105   103 2 3 0 80
..
In use 34552K, total allocated 40384K; utilization 85.6%


It seems kernel dynamic memory is run out, and devbuf and temp consume
most of the space.

Could you give some suggestions? Thanks very much in advance!
Best Regards
Nan Xiao


On Wed, Jul 4, 2018 at 10:57 AM, Philip Guenther  wrote:
> On Tue, 3 Jul 2018, Philip Guenther wrote:
> 
>
> Flakey button on my mouse; time to clean it again and throw it out if it
> keeps glitching.  Sorry about that.
>
>
>> On Tue, Jul 3, 2018 at 4:53 PM Nan Xiao  wrote:
>> > Thanks for your reply! The "ulimit -a" outputs following:
>> >
>> > $ ulimit -a
>> > time(cpu-seconds)unlimited
>> > file(blocks) unlimited
>> > coredump(blocks) unlimited
>> > data(kbytes) 33554432
>> > stack(kbytes)8192
>> > lockedmem(kbytes)1332328
>> > memory(kbytes)   3978716
>> > nofiles(descriptors) 128
>> > processes1310
>> >
>> > It seems should be enough to launch cmake or egdb.
>
> But it wasn't and the kernel can only indicate that with a single error
> code, so now you have to actually dig into what's going on.  There are
> many possibilities, as a search for ENOMEM in /usr/src/sys/kern/*exec*.c
> will show.
> 1) the ELF interpreter (normal ld.so) could be too large
> 2) the PT_OPENBSD_RANDOMIZE segment could be larger than permitted by the
>kernel
> 3) program's text segment could exceed the maximum for the arch, MAXTSIZ
> 4) the program's vnode couldn't be mmaped for some reason
> 5) the argument list and environment were together too big for the stack
> 6) the signal trampoline couldn't be mapped into the process VM
> 7) other random memory allocation problems
>
> Of those, (1), (4), and (6) are *really* unlikely.  (3) is possible if
> you're building a debugging binary that's *huge* as a result.  (5) would
> result in _all_ programs failing in that shell.  I think (7) would show up
> in a close examination of the "vmstat -m" output.
>
> (2) is perhaps the most likely, as recent compiler changes have increased
> the expected size of the PT_OPENBSD_RANDOMIZE segment and while the kernel
> limit on that was also increased recently, you didn't provide any
> information about your setup: are your kernel, userland, and ports all in
> sync?
>
>
> Philip Guenther

Re: "Cannot allocate memory" error when memory is enough

[Philip Guenther]

On Tue, 3 Jul 2018, Philip Guenther wrote:


Flakey button on my mouse; time to clean it again and throw it out if it 
keeps glitching.  Sorry about that.


> On Tue, Jul 3, 2018 at 4:53 PM Nan Xiao  wrote:
> > Thanks for your reply! The "ulimit -a" outputs following:
> >
> > $ ulimit -a
> > time(cpu-seconds)unlimited
> > file(blocks) unlimited
> > coredump(blocks) unlimited
> > data(kbytes) 33554432
> > stack(kbytes)8192
> > lockedmem(kbytes)1332328
> > memory(kbytes)   3978716
> > nofiles(descriptors) 128
> > processes1310
> >
> > It seems should be enough to launch cmake or egdb.

But it wasn't and the kernel can only indicate that with a single error 
code, so now you have to actually dig into what's going on.  There are 
many possibilities, as a search for ENOMEM in /usr/src/sys/kern/*exec*.c 
will show.
1) the ELF interpreter (normal ld.so) could be too large
2) the PT_OPENBSD_RANDOMIZE segment could be larger than permitted by the 
   kernel
3) program's text segment could exceed the maximum for the arch, MAXTSIZ
4) the program's vnode couldn't be mmaped for some reason
5) the argument list and environment were together too big for the stack
6) the signal trampoline couldn't be mapped into the process VM
7) other random memory allocation problems

Of those, (1), (4), and (6) are *really* unlikely.  (3) is possible if 
you're building a debugging binary that's *huge* as a result.  (5) would 
result in _all_ programs failing in that shell.  I think (7) would show up 
in a close examination of the "vmstat -m" output.

(2) is perhaps the most likely, as recent compiler changes have increased 
the expected size of the PT_OPENBSD_RANDOMIZE segment and while the kernel 
limit on that was also increased recently, you didn't provide any 
information about your setup: are your kernel, userland, and ports all in 
sync?


Philip Guenther

Re: "Cannot allocate memory" error when memory is enough

[Philip Guenther]

On Tue, Jul 3, 2018 at 4:53 PM Nan Xiao  wrote:

> Thanks for your reply! The "ulimit -a" outputs following:
>
> $ ulimit -a
> time(cpu-seconds)unlimited
> file(blocks) unlimited
> coredump(blocks) unlimited
> data(kbytes) 33554432
> stack(kbytes)8192
> lockedmem(kbytes)1332328
> memory(kbytes)   3978716
> nofiles(descriptors) 128
> processes1310
>
> It seems should be enough to launch cmake or egdb.
>
> Thanks!
> Best Regards
> Nan Xiao
>
>
> On Tue, Jul 3, 2018 at 9:37 PM, Marc Espie  wrote:
> > On Tue, Jul 03, 2018 at 05:31:22PM +0800, Nan Xiao wrote:
> >> Hi all,
> >>
> >> Greeting from me!
> >>
> >> I am running OpenBSD 6.3, and don't know from when, loading some
> >> binary will prompt "Cannot allocate memory":
> >>
> >> $ egdb
> >> ksh: egdb: Cannot allocate memory
> >>
> >> $ cmake
> >> ksh: cmake: Cannot allocate memory
> >>
> >> But the memory seems enough:
> >> $top
> >> ..
> >> Memory: Real: 57M/1365M act/tot Free: 2546M Cache: 925M Swap: 0K/4103M
> >> ..
> >>
> >> I try to use "ktrace/kdump" tool, but can't find something special:
> >> ..
> >>  21881 ktrace   NAMI  "/usr/local/bin/egdb"
> >>  21881 ktrace   RET   execve -1 errno 12 Cannot allocate memory
> >> ..
> >>
> >> Could anyone give some clues? Thanks very much in advance!
> >> Best Regards
> >> Nan Xiao
> >
> > Check your limits.
> >
> > ulimit -a
> >
> > from the shell will tell you what's wrong.
> >
> > you might also need to brush up on login.conf  and get your user into
> > a different class.
>
>

Re: "Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi Marc,

Thanks for your reply! The "ulimit -a" outputs following:

$ ulimit -a
time(cpu-seconds)unlimited
file(blocks) unlimited
coredump(blocks) unlimited
data(kbytes) 33554432
stack(kbytes)8192
lockedmem(kbytes)1332328
memory(kbytes)   3978716
nofiles(descriptors) 128
processes1310

It seems should be enough to launch cmake or egdb.

Thanks!
Best Regards
Nan Xiao


On Tue, Jul 3, 2018 at 9:37 PM, Marc Espie  wrote:
> On Tue, Jul 03, 2018 at 05:31:22PM +0800, Nan Xiao wrote:
>> Hi all,
>>
>> Greeting from me!
>>
>> I am running OpenBSD 6.3, and don't know from when, loading some
>> binary will prompt "Cannot allocate memory":
>>
>> $ egdb
>> ksh: egdb: Cannot allocate memory
>>
>> $ cmake
>> ksh: cmake: Cannot allocate memory
>>
>> But the memory seems enough:
>> $top
>> ..
>> Memory: Real: 57M/1365M act/tot Free: 2546M Cache: 925M Swap: 0K/4103M
>> ..
>>
>> I try to use "ktrace/kdump" tool, but can't find something special:
>> ..
>>  21881 ktrace   NAMI  "/usr/local/bin/egdb"
>>  21881 ktrace   RET   execve -1 errno 12 Cannot allocate memory
>> ..
>>
>> Could anyone give some clues? Thanks very much in advance!
>> Best Regards
>> Nan Xiao
>
> Check your limits.
>
> ulimit -a
>
> from the shell will tell you what's wrong.
>
> you might also need to brush up on login.conf  and get your user into
> a different class.

Re: "Cannot allocate memory" error when memory is enough

[Marc Espie]

On Tue, Jul 03, 2018 at 05:31:22PM +0800, Nan Xiao wrote:
> Hi all,
> 
> Greeting from me!
> 
> I am running OpenBSD 6.3, and don't know from when, loading some
> binary will prompt "Cannot allocate memory":
> 
> $ egdb
> ksh: egdb: Cannot allocate memory
> 
> $ cmake
> ksh: cmake: Cannot allocate memory
> 
> But the memory seems enough:
> $top
> ..
> Memory: Real: 57M/1365M act/tot Free: 2546M Cache: 925M Swap: 0K/4103M
> ..
> 
> I try to use "ktrace/kdump" tool, but can't find something special:
> ..
>  21881 ktrace   NAMI  "/usr/local/bin/egdb"
>  21881 ktrace   RET   execve -1 errno 12 Cannot allocate memory
> ..
> 
> Could anyone give some clues? Thanks very much in advance!
> Best Regards
> Nan Xiao

Check your limits.

ulimit -a

from the shell will tell you what's wrong.

you might also need to brush up on login.conf  and get your user into
a different class.

"Cannot allocate memory" error when memory is enough

[Nan Xiao]

Hi all,

Greeting from me!

I am running OpenBSD 6.3, and don't know from when, loading some
binary will prompt "Cannot allocate memory":

$ egdb
ksh: egdb: Cannot allocate memory

$ cmake
ksh: cmake: Cannot allocate memory

But the memory seems enough:
$top
..
Memory: Real: 57M/1365M act/tot Free: 2546M Cache: 925M Swap: 0K/4103M
..

I try to use "ktrace/kdump" tool, but can't find something special:
..
 21881 ktrace   NAMI  "/usr/local/bin/egdb"
 21881 ktrace   RET   execve -1 errno 12 Cannot allocate memory
..

Could anyone give some clues? Thanks very much in advance!
Best Regards
Nan Xiao

Re: node: Cannot allocate memory

[Ken M]

So just to eliminate the off variable I updated my snapshot. Updated packages,
etc, etc

And now node works fine...

Ken

On Wed, Jun 06, 2018 at 09:28:39PM -0400, Ken M wrote:
> On Wed, Jun 06, 2018 at 09:10:59PM -0400, Thomas Frohwein wrote:
> > 
> > I run ksh. Doubt that bash is the cause though...
> > Might wanna check if you have the same problem with ksh.
> > 
> 
> I tried in sh before submitting and got the same problem, I just tried ksh and
> the same. Sorry for omitting that I tried to eliminate bash from the equation
> first.
> 
> > 
> > ... still, you provided rather little information to understand what might 
> > be
> > particular about your system. In most cases, including a dmesg is MVP to
> > understand this better.
> > 
> 
> Sorry, I wasn't thinking a dmesg would be useful in this case. I will put at 
> the
> end of this as well as the kdump from a ktrace on this.
> 
> > 
> > Yes? what about your ulimit?
> 
> I put myself in the staff user group so...
> 
> $ ulimit -a
> core file size  (blocks, -c) unlimited
> data seg size   (kbytes, -d) 2097152
> file size   (blocks, -f) unlimited
> max locked memory   (kbytes, -l) 1244372
> max memory size (kbytes, -m) 3710004
> open files  (-n) 512
> pipe size(512 bytes, -p) 1
> stack size  (kbytes, -s) 4096
> cpu time   (seconds, -t) unlimited
> max user processes  (-u) 256
> virtual memory  (kbytes, -v) 2101248
> 
> I also tried setting ulimit -d to my total memory size and no luck there.
> 
> kdump of ktrace:
> 
> $ kdump -f kt-node.out
>   7097 ktrace   RET   ktrace 0
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/bin/node"
>   7097 ktrace   RET   execve -1 errno 2 No such file or directory
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/usr/bin/node"
>   7097 ktrace   RET   execve -1 errno 2 No such file or directory
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/sbin/node"
>   7097 ktrace   RET   execve -1 errno 2 No such file or directory
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/usr/sbin/node"
>   7097 ktrace   RET   execve -1 errno 2 No such file or directory
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/usr/X11R6/bin/node"
>   7097 ktrace   RET   execve -1 errno 2 No such file or directory
>   7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
>   7097 ktrace   NAMI  "/usr/local/bin/node"
>   7097 ktrace   RET   execve -1 errno 12 Cannot allocate memory
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  write(2,0x7f7eb1e0,0x8)
>   7097 ktrace   GIO   fd 2 wrote 8 bytes
>"ktrace: "
>   7097 ktrace   RET   write 8
>   7097 ktrace   CALL  write(2,0x7f7eb2c0,0x15)
>   7097 ktrace   GIO   fd 2 wrote 21 bytes
>    "exec of 'node' failed"
>   7097 ktrace   RET   write 21/0x15
>   7097 ktrace   CALL  write(2,0x7f7eb1e0,0x2)
>   7097 ktrace   GIO   fd 2 wrote 2 bytes
>": "
>   7097 ktrace   RET   write 2
>   7097 ktrace   CALL  write(2,0x7f7eb1e0,0x17)
>   7097 ktrace   GIO   fd 2 wrote 23 bytes
>"Cannot allocate memory
>"
>   7097 ktrace   RET   write 23/0x17
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
>   7097 ktrace   RET   mprotect 0
>   7097 ktrace   CALL  munmap(0xe36158b1000,0x1000)
>   7097 ktrace   RET   munmap 0
>   7097 ktrace   CALL  exit(1)
> 
> 
> dmesg:
> 
> $ dmesg
> OpenBSD 6.3-current (GENERIC.MP) #57: Thu May 31 15:57:20 MDT 2018
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 3950682112 (3767MB)
> avail mem = 3822796800 (3645MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdccb3000 (61 entries)
> bios0: vendor LENOVO version "GJET98WW (2.48 )" dat

Re: node: Cannot allocate memory

[Ken M]

On Wed, Jun 06, 2018 at 09:10:59PM -0400, Thomas Frohwein wrote:
> 
> I run ksh. Doubt that bash is the cause though...
> Might wanna check if you have the same problem with ksh.
> 

I tried in sh before submitting and got the same problem, I just tried ksh and
the same. Sorry for omitting that I tried to eliminate bash from the equation
first.

> 
> ... still, you provided rather little information to understand what might be
> particular about your system. In most cases, including a dmesg is MVP to
> understand this better.
> 

Sorry, I wasn't thinking a dmesg would be useful in this case. I will put at the
end of this as well as the kdump from a ktrace on this.

> 
> Yes? what about your ulimit?

I put myself in the staff user group so...

$ ulimit -a
core file size  (blocks, -c) unlimited
data seg size   (kbytes, -d) 2097152
file size   (blocks, -f) unlimited
max locked memory   (kbytes, -l) 1244372
max memory size (kbytes, -m) 3710004
open files  (-n) 512
pipe size(512 bytes, -p) 1
stack size  (kbytes, -s) 4096
cpu time   (seconds, -t) unlimited
max user processes  (-u) 256
virtual memory  (kbytes, -v) 2101248

I also tried setting ulimit -d to my total memory size and no luck there.

kdump of ktrace:

$ kdump -f kt-node.out
  7097 ktrace   RET   ktrace 0
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/bin/node"
  7097 ktrace   RET   execve -1 errno 2 No such file or directory
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/usr/bin/node"
  7097 ktrace   RET   execve -1 errno 2 No such file or directory
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/sbin/node"
  7097 ktrace   RET   execve -1 errno 2 No such file or directory
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/usr/sbin/node"
  7097 ktrace   RET   execve -1 errno 2 No such file or directory
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/usr/X11R6/bin/node"
  7097 ktrace   RET   execve -1 errno 2 No such file or directory
  7097 ktrace   CALL  execve(0x7f7eb5d0,0x7f7ebb60,0x7f7ebb78)
  7097 ktrace   NAMI  "/usr/local/bin/node"
  7097 ktrace   RET   execve -1 errno 12 Cannot allocate memory
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  write(2,0x7f7eb1e0,0x8)
  7097 ktrace   GIO   fd 2 wrote 8 bytes
   "ktrace: "
  7097 ktrace   RET   write 8
  7097 ktrace   CALL  write(2,0x7f7eb2c0,0x15)
  7097 ktrace   GIO   fd 2 wrote 21 bytes
   "exec of 'node' failed"
  7097 ktrace   RET   write 21/0x15
  7097 ktrace   CALL  write(2,0x7f7eb1e0,0x2)
  7097 ktrace   GIO   fd 2 wrote 2 bytes
   ": "
  7097 ktrace   RET   write 2
  7097 ktrace   CALL  write(2,0x7f7eb1e0,0x17)
  7097 ktrace   GIO   fd 2 wrote 23 bytes
   "Cannot allocate memory
   "
  7097 ktrace   RET   write 23/0x17
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x3)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  mprotect(0xe36158b1000,0x1000,0x1)
  7097 ktrace   RET   mprotect 0
  7097 ktrace   CALL  munmap(0xe36158b1000,0x1000)
  7097 ktrace   RET   munmap 0
  7097 ktrace   CALL  exit(1)


dmesg:

$ dmesg
OpenBSD 6.3-current (GENERIC.MP) #57: Thu May 31 15:57:20 MDT 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 3950682112 (3767MB)
avail mem = 3822796800 (3645MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdccb3000 (61 entries)
bios0: vendor LENOVO version "GJET98WW (2.48 )" date 03/20/2018
bios0: LENOVO 20B7S41700
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC DBGP ECDT HPET APIC MCFG SSDT SSDT SSDT SSDT SSDT 
SSDT SSDT SSDT PCCT SSDT UEFI MSDM ASF! BATB FPDT UEFI DMAR
acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP2(S4) XHCI(S3) EHC1(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-4300U CPU @ 1.90GHz, 798.30 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE

Re: node: Cannot allocate memory

[Thomas Frohwein]

I can't reproduce this on -current amd64, neither with a snapshot from last
week, nor after updating today.

thfr@e5570:~$ node -v
v8.11.1
thfr@e5570:~$ node
> console.log('test log');
test log
undefined
> .exit
thfr@e5570:~$

> node -v
> bash: /usr/local/bin/node: Cannot allocate memory

I run ksh. Doubt that bash is the cause though...
Might wanna check if you have the same problem with ksh.

> I am on current, last grabbed the snapshot last Friday I think.

... still, you provided rather little information to understand what might be
particular about your system. In most cases, including a dmesg is MVP to
understand this better.

> Plenty of swap and memory available

Yes? what about your ulimit?

node: Cannot allocate memory

[Ken M]

The subject is the problem:

node -v
bash: /usr/local/bin/node: Cannot allocate memory

I am on current, last grabbed the snapshot last Friday I think.

Plenty of swap and memory available

vmstat
 procsmemory   pagedisk traps  cpu
 r   s   avm fre  flt  re  pi  po  fr  sr sd0  int   sys   cs us sy id
 1 180  543M   1802M 1263   0   0   0   0   0  38  235  5252  781  3  1 95
swapctl -l
Device  512-blocks UsedAvail Capacity  Priority
/dev/sd0b  82415360  8241536 0%0

Last I checked on a 6.3 release install node was working. Last I checked it was
working in 6.3 so not sure what is going on here. Nothing else is giving me any
problems.

Ken

Re: cannot allocate memory

[Otto Moerbeek]

On Sat, Mar 19, 2016 at 03:09:13AM +0700, Roman Gorelov wrote:

> I have strange issues. I installed go port and figured out that go
> compiler and executables produced by it require 770 MB of datasize.
> I bumped datasize-max and datasize-cur of login class `default' to
> 771 MB. Then programs being run as another user stopped working with
> `cannot allocate memory' error, while there's clearly plenty of
> memory. E.g. useradd -m a7sl4 ; doas -u a7sl4 mpv "$x". Then reboot,
> cannot login as usual user to xdm. Login(1)s on ttys didn't even run.
> Logged in as root to xdm, changed login class of user to `staff', and
> then only relogged as usual user. I don't really know what is going
> on and appreciate any help.

Looks like you got your units wrong in login.conf, at least in the
default login class.

-Otto


> 
> ps auxw
> 
> USER   PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED   TIME COMMAND
> _x11  2566 21.8  1.5 13536 15856 ??  Rs 2:51AM1:39.99 
> /usr/X11R6/bin/X :0 vt05 -auth /etc/X11/xdm/authdir/authfiles/A:0
> root 1  0.0  0.0   480   480 ??  Ss 2:51AM0:01.15 /sbin/init
> root  3899  0.0  0.0   628   396 ??  Is 2:51AM0:00.04 dhclient: 
> alc0 [priv] (dhclient)
> _dhcp14600  0.0  0.0   744   484 ??  Is 2:51AM0:00.01 dhclient: 
> alc0 (dhclient)
> _syslogd 28193  0.0  0.1   944  1296 ??  S  2:51AM0:00.05 
> /usr/sbin/syslogd
> root 15932  0.0  0.1   944  1160 ??  Is 2:51AM0:00.02 syslogd: 
> [priv] (syslogd)
> root 20610  0.0  0.1   624   552 ??  Is 2:51AM0:00.04 pflogd: 
> [priv] (pflogd)
> _pflogd  31696  0.0  0.0   688   364 ??  S  2:51AM0:00.05 pflogd: 
> [running] -s 160 -i pflog0 -f /var/log/pflog (pflogd)
> root  2681  0.0  0.1   660  1316 ??  I [priv] (ntpd)
> _ntp  1120  0.0  0.2  1380  2512 ??  S< 2:51AM0:00.15 ntpd: ntp 
> engine (ntpd)
> _ntp  5306  0.0  0.1   928  1320 ??  I  2:51AM0:00.02 ntpd: dns 
> engine (ntpd)
> _smtpq   30513  0.0  0.2  1444  1992 ??  I  2:51AM0:00.02 smtpd: 
> queue (smtpd)
> root 30736  0.0  0.2  1460  1904 ??  Is 2:51AM0:00.03 smtpd: 
> [priv] (smtpd)
> _smtpd   17934  0.0  0.2  1320  1888 ??  I  2:51AM0:00.02 smtpd: 
> lookup (smtpd)
> _smtpd   16021  0.0  0.2  1380  2280 ??  I  2:51AM0:00.01 smtpd: pony 
> express (smtpd)
> _smtpd   12548  0.0  0.2  1244  1748 ??  I  2:51AM0:00.01 smtpd: 
> klondike (smtpd)
> _smtpd8725  0.0  0.2  1176  1736 ??  I  2:51AM0:00.02 smtpd: 
> scheduler (smtpd)
> _smtpd   31670  0.0  0.2  1448  1988 ??  I  2:51AM0:00.02 smtpd: 
> control (smtpd)
> _sndio   15485  0.0  0.1   368   524 ??  I /usr/bin/sndiod -c 0:3
> _transmission  7888  0.0  0.2  1544  2364 ??  Ss 2:51AM0:00.58 
> /usr/local/bin/transmission-daemon (transmission-dae)
> root  8430  0.0  0.1   356   776 ??  Ss 2:51AM0:00.04 
> /usr/sbin/apmd -A
> root   398  0.0  0.1   356   784 ??  Is 2:51AM0:00.03 
> /usr/sbin/hotplugd
> root 14252  0.0  0.1   660  1084 ??  Is 2:51AM0:00.02 
> /usr/sbin/cron
> root 24072  0.0  0.2   548  1660 ??  Is 2:51AM0:00.23 
> /usr/X11R6/bin/xdm
> root  3450  0.0  0.1  2320  1188 ??  I  2:51AM0:00.31 X: [priv] 
> (Xorg)
> root 16612  0.0  0.5  1332  5516 ??  Is 2:51AM0:00.60 xdm: :0 
> (xdm)
> u 3291  0.0  0.1   644   632 ??  Is 2:51AM0:00.02 /bin/sh 
> /etc/X11/xdm/Xsession
> u   65  0.0  0.1   644   624 ??  I  2:51AM0:00.03 /bin/sh 
> /home/u/.xsession
> u10190  0.0  0.5  1096  4828 ??  S  2:51AM0:00.31 cwm
> u15624  0.0  0.5  1584  5300 ??  Ss 2:52AM0:01.46 st
> u 6913  0.1  0.2  1160  2160 ??  Ss 2:52AM0:02.80 tmux: 
> server (/tmp/tmux-1000/default) (tmux)
> root 15233  0.0  0.0   632   504 ??  S  2:59AM0:00.00 /sbin/init
> root  9263  0.0  0.0   620   500 ??  S  2:59AM0:00.00 /sbin/init
> root  3639  0.0  0.0   620   496 ??  S  2:59AM0:00.00 /sbin/init
> root 30257  0.0  0.0   620   496 ??  S  2:59AM0:00.00 /sbin/init
> root  6873  0.0  0.0   632   504 ??  S  2:59AM0:00.00 /sbin/init
> u 1482  0.0  0.1   648   672 p0  Is 2:52AM0:00.13 /bin/ksh
> u17821  0.0  0.2   744  1588 p0  I+ 2:52AM0:00.04 tmux: 
> client (/tmp/tmux-1000/default) (tmux)
> u23676  0.0  0.1   652   688 p1  Ss 2:59AM0:00.05 -ksh (ksh)
> u 1353  0.0  0.0   392   376 p1  R+ 2:59AM0:00.00 ps -auxw
> u28398  0.0  0.1   652   696 p2  Is 2:53AM0:00.03 -ksh (ksh)
> u20896  0.0  0.5  2256  5272 p2  I+ 2:53AM0:00

Re: cannot allocate memory

[Roman Gorelov]

Solved, thank you. I was imprudent.

cannot allocate memory

[Roman Gorelov]

I have strange issues. I installed go port and figured out that go
compiler and executables produced by it require 770 MB of datasize.
I bumped datasize-max and datasize-cur of login class `default' to
771 MB. Then programs being run as another user stopped working with
`cannot allocate memory' error, while there's clearly plenty of
memory. E.g. useradd -m a7sl4 ; doas -u a7sl4 mpv "$x". Then reboot,
cannot login as usual user to xdm. Login(1)s on ttys didn't even run.
Logged in as root to xdm, changed login class of user to `staff', and
then only relogged as usual user. I don't really know what is going
on and appreciate any help.

ps auxw

USER   PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED   TIME COMMAND
_x11  2566 21.8  1.5 13536 15856 ??  Rs 2:51AM1:39.99 
/usr/X11R6/bin/X :0 vt05 -auth /etc/X11/xdm/authdir/authfiles/A:0
root 1  0.0  0.0   480   480 ??  Ss 2:51AM0:01.15 /sbin/init
root  3899  0.0  0.0   628   396 ??  Is 2:51AM0:00.04 dhclient: 
alc0 [priv] (dhclient)
_dhcp14600  0.0  0.0   744   484 ??  Is 2:51AM0:00.01 dhclient: 
alc0 (dhclient)
_syslogd 28193  0.0  0.1   944  1296 ??  S  2:51AM0:00.05 
/usr/sbin/syslogd
root 15932  0.0  0.1   944  1160 ??  Is 2:51AM0:00.02 syslogd: 
[priv] (syslogd)
root 20610  0.0  0.1   624   552 ??  Is 2:51AM0:00.04 pflogd: 
[priv] (pflogd)
_pflogd  31696  0.0  0.0   688   364 ??  S  2:51AM0:00.05 pflogd: 
[running] -s 160 -i pflog0 -f /var/log/pflog (pflogd)
root  2681  0.0  0.1   660  1316 ??  I rlkok pckbc0
  noram
Namei Sys-cacheProc-cacheNo-cache  42 ndcpy
Calls hits%hits %miss   % fltcp
  256  256  10015 zfod
   29 cow
Disks   sd0  8359 fmin
seeks   11145 ftarg
xfers 1   itarg
speed   10K49 wired
  sec   0.1   pdfre
  pdscn
  pzidle
   26 kmapent












IPKTS
OPKTS


dmesg

OpenBSD 5.8 (GENERIC.MP) #1: Wed Mar 16 10:05:46 CET 2016

r...@stable-58-i386.mtier.org:/binpatchng/work-binpatch58-i386/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Atom(TM) CPU N2600 @ 1.60GHz ("GenuineIntel" 686-class) 1.60 GHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
real mem  = 1060052992 (1010MB)
avail mem = 1026433024 (978MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: date 03/15/11, SMBIOS rev. 2.7 @ 0xe96d0 (31 entries)
bios0: vendor American Megatrends Inc. version "X101CH.1203" date 07/30/2012
bios0: ASUSTeK COMPUTER INC. X101CH
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG RTCF HPET SSDT SLIC BGRT
acpi0: wakeup devices P0P8(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) 
RP03(S4) PXSX(S4) RP04(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 7 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=22785, max=32960
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Atom(TM) CPU N2600 @ 1.60GHz ("GenuineIntel" 686-class) 1.60 GHz
cpu1: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Atom(TM) CPU N2600 @ 1.60GHz ("GenuineIntel" 686-class) 1.60 GHz
cpu2: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Atom(TM) CPU N2600 @ 1.60GHz ("GenuineIntel" 686-class) 1.60 GHz
cpu3: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,DTES64,MWAIT,DS-CPL,EST,TM2,SSSE3,xTPR,PDCM,MOVBE,LAHF,PERF,ITSC
ioapic0 at mainbus0: apid 4 pa 0xfec0, version 20, 24 

Re: xz: (stdin): Cannot allocate memory

[Lampshade]

I figured out that my default is:
ulimit -d

1572864

echo "1572864/1024" | bc
1536

Value, which lets me compress using this setting is
between 1682864 and 1672864 kilobytes.

I have also discovered command line option for xz --memlimit=
Now my command looks that:

cat archive.tar | xz -zf --memlimit=1600MiB  \
--format=xz -9e --threads=2 - > archive.tar.xz

but I get:
xz: Adjusted the number of threads from 2 to 1 to not exceed the memory
usage limit of 1600 MiB

1600 is clearly larger than 674*2=1348

In the end I can compress, but I think that something is wrong.

Od: "Christian Weisgerber" <na...@mips.inka.de>
Do: "Lampshade" <lampsh...@poczta.fm>;
Wysłane: 16:25 Sobota 2016-01-30
Temat: Re: xz: (stdin): Cannot allocate memory

> Lampshade:
>
> > I have following error:
> > cat archive.tar | xz -zf --format=xz -9e --threads=2 - >
archive.tar.xz
> > xz: (stdin): Cannot allocate memory
>
> You are using the most extreme compression setting, which requires
> about 674 MB per thread according to the xz(1) man page.  This
> causes you to bump against the data size limit (ulimit -d, see
> ksh(1)).
>
> You need to raise the limit or use a less greedy compression setting.
>
> --
> Christian "naddy" Weisgerber  na...@mips.inka.de

Re: xz: (stdin): Cannot allocate memory

[Lampshade]

This xz command worked in past so I think something must
have been changed in past. Indeed, this command worked
when I had 4G of DDR3@1333Mhz RAM. Now I have 6GB DDR3
on the same laptop so I have even more.
I will look at ulimit -d this evening. I didn't changed them manually, so they
must have been changed during upgrade from current to current.

Re: xz: (stdin): Cannot allocate memory

[Brian Conway]

Have you read the xz man page, specifically on memory requirements?
9-extreme is probably not what you want - it's fairly insane/placebo. Full
dmesg with memory information is needed beyond that.

Brian Conway


On Jan 30, 2016 7:18 AM, "Lampshade" <lampsh...@poczta.fm> wrote:

> Hello
> I have this OS with packages as of yesterday (Jan 29):
> kern.version=OpenBSD 5.9-beta (GENERIC.MP) #1865: Thu Jan 28 20:18:15 MST
> 2016
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
> and also tested on with packages around Jan 17:
> kern.version=OpenBSD 5.9-beta (GENERIC.MP) #1846: Sun Jan 17 02:34:54 MST
> 2016
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
> I have following error:
> cat archive.tar | xz -zf --format=xz -9e --threads=2 - > archive.tar.xz
> xz: (stdin): Cannot allocate memory

Re: xz: (stdin): Cannot allocate memory

[Christian Weisgerber]

Lampshade:

> I have following error:
> cat archive.tar | xz -zf --format=xz -9e --threads=2 - > archive.tar.xz 
> xz: (stdin): Cannot allocate memory

You are using the most extreme compression setting, which requires
about 674 MB per thread according to the xz(1) man page.  This
causes you to bump against the data size limit (ulimit -d, see
ksh(1)).

You need to raise the limit or use a less greedy compression setting.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

xz: (stdin): Cannot allocate memory

[Lampshade]

Hello
I have this OS with packages as of yesterday (Jan 29):
kern.version=OpenBSD 5.9-beta (GENERIC.MP) #1865: Thu Jan 28 20:18:15 MST 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

and also tested on with packages around Jan 17:
kern.version=OpenBSD 5.9-beta (GENERIC.MP) #1846: Sun Jan 17 02:34:54 MST 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

I have following error:
cat archive.tar | xz -zf --format=xz -9e --threads=2 - > archive.tar.xz 
xz: (stdin): Cannot allocate memory

Re: xz: (stdin): Cannot allocate memory

[Christian Weisgerber]

On 2016-01-30, Lampshade  wrote:

> xz: Adjusted the number of threads from 2 to 1 to not exceed the memory
> usage limit of 1600 MiB
>
> 1600 is clearly larger than 674*2=1348

A closer reading of the man page reveals that memory consumption
is even higher in multi-threaded mode.

In multi-threaded mode about three times _size_ bytes will be
allocated in each thread for buffering input and output.  The
default _size_ is three times the LZMA2 dictionary size or 1 MiB,
whichever is more.

At -9, the dictionary size is 64 MB, so this adds another 576 MB
per thread.  A quick check with top(1) confirms that xz in
multi-threaded mode allocates 1250 MB per thread at compression
level 9.

> In the end I can compress, but I think that something is wrong.

The last relevant change is that we actually enabled multi-threading.
Before that, the -T option was silently ignored.  However that
change was ten months ago and is already in the 5.8 packages.

Anyway, there is nothing wrong, and as I said before:
You need to raise the limit or use a less greedy compression setting.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

[Cannot allocate memory][Qemu][x86 i386] limits ? login.conf ?

[Jan Lambertz]

Hi, had same Problem.the only (poor) workaround i found is running qemu as
root .

Re: [Cannot allocate memory][Qemu][x86 i386] limits ? login.conf ?

[Francois Pussault]

Hello, Thanks,

This didn't correct the problem.
I'm still limitedOf course the host has been rebooted few days ago.
this doesn't fix this case.

I set :datasize=infinity:\ in a specific qemu class, and added the user.

I tried to play datasize-cur or max or both but 

:(



 
 From: Alexis de BRUYN alexis.mailingl...@de-bruyn.fr
 Sent: Thu Jul 17 14:56:11 CEST 2014
 To: misc@openbsd.org
 Subject: Re: [Cannot allocate memory][Qemu][x86  i386] limits ? login.conf ?
 
 
 On 16.07.2014 10:48, Francois Pussault wrote:
  Hi all.
 Hi François,
 
  I experiment qemu problems while playing with emulators/virtual machines ...
  
  When I try to create any virtual machines with x86_64 or i 386 hardware and
  more than 256Mo ram that fail.
  Qemu returns Cannot allocate memory.
 I had the same issue on -current couple weeks ago.
 
  default:\
  :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin
  /usr/local/sbin:\
  :umask=022:\
  :datasize-max=2048M:\
  :datasize-cur=2024M:\
  :maxproc-max=512:\
  :maxproc-cur=256:\
  :vmemoryuse=1024:\
  :openfiles-cur=1024:\
  :stacksize-cur=8M:\
  :localcipher=blowfish,6:\
  :ypcipher=old:\
  :tc=auth-defaults:\
  :tc=auth-ftp-defaults:
  
 I added in my user class:
 :datasize=infinity:\
 
 No more problem for me, I can start VM with +256M RAM.
 
 Be sure that the user which launches qemu has your class (vipw(8)).
 
 Regards,
 
 -- 
 Alexis de BRUYN
 


Cordialement
Francois Pussault
10 chemin de négo saoumos
apt 202 - bat 2
31300 Toulouse
+33 6 17 230 820   +33 5 34 365 269 
fpussa...@contactoffice.fr

Re: [Cannot allocate memory][Qemu][x86 i386] limits ? login.conf ?

[Alexis de BRUYN]

On 16.07.2014 10:48, Francois Pussault wrote:
 Hi all.
Hi François,

 I experiment qemu problems while playing with emulators/virtual machines ...
 
 When I try to create any virtual machines with x86_64 or i 386 hardware and
 more than 256Mo ram that fail.
 Qemu returns Cannot allocate memory.
I had the same issue on -current couple weeks ago.

 default:\
 :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin
 /usr/local/sbin:\
 :umask=022:\
 :datasize-max=2048M:\
 :datasize-cur=2024M:\
 :maxproc-max=512:\
 :maxproc-cur=256:\
 :vmemoryuse=1024:\
 :openfiles-cur=1024:\
 :stacksize-cur=8M:\
 :localcipher=blowfish,6:\
 :ypcipher=old:\
 :tc=auth-defaults:\
 :tc=auth-ftp-defaults:
 
I added in my user class:
:datasize=infinity:\

No more problem for me, I can start VM with +256M RAM.

Be sure that the user which launches qemu has your class (vipw(8)).

Regards,

-- 
Alexis de BRUYN

[Cannot allocate memory][Qemu][x86 i386] limits ? login.conf ?

[Francois Pussault]

Hi all.

I experiment qemu problems while playing with emulators/virtual machines ...

When I try to create any virtual machines with x86_64 or i 386 hardware and
more than 256Mo ram that fail.
Qemu returns Cannot allocate memory.

I tryied to modify login.conf  ulimits values, but this still fail.

I randomly ... YES RANDOMLY  discover ram values are limited to exact
values between : 256  512Mo ram allocable ...no more ... no less... on a test
setup with the laptop.
How I discoverd that ?
like this :
 $ M=2048
 $ while : ; do qeum-system-i386 -m $M ; M=$(( $M - 1 )) ; echo $M ; done ;
unset M
So I let this run until VM starts, then I look the last $M value.

Qemu documentation says ram limit is 2047Mo for both i386  x86_64

Anyway Qemu with ram values between 512  846 are just working very fine
...

Can check my actual setup in attached file. On this machine Qemu machines ram
is limited to 256Mo (with no modifications of ulimits neither in login.conf

I don't understand where is the problem. This doesn't seem to cause any hangs,
bugs or warning in system log files...

Any help or Ideas about that ?


FYI : any other hardware emulated seems to work with its own maximum supported
ram
for example :
$ qemu-system-sparc  -machine SS-20 -smp 4 -m 768
 just works fine on same machine...



Max values I have tried in login.conf were :



default:\
:path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin
/usr/local/sbin:\
:umask=022:\
:datasize-max=2048M:\
:datasize-cur=2024M:\
:maxproc-max=512:\
:maxproc-cur=256:\
:vmemoryuse=1024:\
:openfiles-cur=1024:\
:stacksize-cur=8M:\
:localcipher=blowfish,6:\
:ypcipher=old:\
:tc=auth-defaults:\
:tc=auth-ftp-defaults:








Cordialement
Francois Pussault
10 chemin de négo saoumos
apt 202 - bat 2
31300 Toulouse
+33 6 17 230 820   +33 5 34 365 269
fpussa...@contactoffice.fr
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar  5 09:37:46 MST 2014
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17161342976 (16366MB)
avail mem = 16695902208 (15922MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xee000 (71 entries)
bios0: vendor HP version P56 date 11/13/2007
bios0: HP ProLiant DL380 G5
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP SPCR MCFG HPET SPMI ERST APIC  BERT HEST
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2333.67 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu0: 4MB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 333MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.0, IBE
cpu1 at mainbus0: apid 4 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2332.51 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu1: 4MB 64b/line 16-way L2 cache
cpu1: smt 0, core 0, package 1
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2332.51 MHz
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu2: 4MB 64b/line 16-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2332.51 MHz
cpu3: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu3: 4MB 64b/line 16-way L2 cache
cpu3: smt 0, core 2, package 1
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2332.51 MHz
cpu4: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu4: 4MB 64b/line 16-way L2 cache
cpu4: smt 0, core 1, package 0
cpu5 at mainbus0: apid 5 (application processor)
cpu5: Intel(R) Xeon(R) CPU E5345 @ 2.33GHz, 2332.51 MHz
cpu5: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,LONG,LAHF,PERF
cpu5: 4MB 64b/line 16-way L2 cache
cpu5: smt 0, core 1

Relayd -- Downloading large files fails with Cannot allocate memory

[Andrew Klettke]

Running 4.9, downloading large files (250MB) from a website behind a 
firewall clustered with relayd fails with the following error in our logs:


Dec  6 12:14:15 fw01 relayd[5615]: relay httpproxy, session 768464 (23 
active), 0, * - 192.168.15.101:80, Cannot allocate memory



Here are the applicable relayd.conf directives:


relay httpproxy {
listen on $relayd_addr port $relayd_port
protocol httpfilter
forward to web_parents port 80 mode loadbalance check http / 
code 200

}

http protocol httpfilter {
tcp { nodelay, sack, socket buffer 65536, backlog 50 }
return error

header append $REMOTE_ADDR to X-Forwarded-For
header change Keep-Alive to $TIMEOUT

}

Any thoughts? Has anyone seen anything like this before? vmstat shows 
plenty of free RAM.


--
Thanks,

Andrew Klettke
Systems Admin
Optic Fusion
253-830-2943

Re: mount:mfs: mmap: Cannot allocate memory

[annathemermaid]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 3 May 2011 21:25, Stuart Henderson s...@spacehopper.org wrote:
 On 2011-05-03, annathemerm...@hush.com annathemerm...@hush.com
wrote:

 So, I'm trying to create put my /tmp partition on my swap.

 So I have a line like this in my fstab:
 swap /tmp mfs rw,nodev,nosuid,-s=2097152
 I also tried changing swap to /dev/wd0b, and changing the n in -
s=n
 to smaller numbers.

 [assuming this is still macppc]

 2097152 x 512 (bytes per sector) = 1GB

 Up to OpenBSD 4.9, MAXDSIZ on macppc is 512MB so you will be
 limited to -s=1048576 or slightly below.

 In -current it has been bumped to 2GB so you'll be able to use
 more there.

A bit below, but now it works. I'm quite happy with stable. Unless
current fixes my X11 keyboard issues, I'd rather not deal with
current unless I have a second hard disk to play with. I suppose if
I ever temporarily need more space in /tmp I'll just have to set up
a softlink to /var/tmp or somewhere.

However, is there any way to get mfs to use swap first rather than
memory first? I'm just trying to make it take advantage of the swap
encryption (random keys unlike a single key I have to remember the
password for); I don't really need speed improvements and certain
excessively bloated applications like Firefox and Gimp, which I
might occasionally want to run, could really use the memory.

 Also, is there any way to switch consoles? On i386 it was
 Ctrl+Alt+F2, Ctrl+Alt+F3, and so on (in fact I think the Ctrl was
 optional unless leaving X11), but it doesn't seem to work on
 macppc, even with the Fn key held down too.

 No separate consoles, you will need to use tmux or similar
 (or X of course).

tmux works fine. Not as convenient as separate consoles, but it
does the job. And, with one tmux terminal displaying the output of
pdisk and another running disklabel, I was able to manually tell
disklabel where the Linux partitions were so OpenBSD and Linux
could share swap and ext2/3 home. For some reason, OpenBSD did not
seem able to find the Linux partitions automatically like it could
on x86, even when I tried re-installing OpenBSD after Debian.

Unfortunately, once I start X11, I don't know how to get out, short
of rebooting the computer. This is very inconvenient when X
crashes, or I can't click the exit button in fvwm because the
faulty keyboard driver things I'm holding down the arrow down key
even though I'm not. For the record, X seems to crash in
framebuffer mode, but not in ati/radeon accelerated mode. In
framebuffer  mode, even running some xterm program, like top, might
kill it, although it seems more likely to die on me if Firefox is
running. Changing the driver seemed to fix mouse issues. (Whatever
it had by default, sometimes I would lose control of the cursor and
it would go toward the bottom of the screen.) I haven't figured out
the right driver for the keyboard. If I tell xorg.conf to use kbd
or akbd, it doens't work at all, but if I tell it to use keyboard,
sometimes it randomly decides not to notice that I've pressed
certain keys, and is particularly reluctant to recognize the
backspace key. Sometimes, it becomes convinced I am holding a key
down even when I am not: this is particularly annoying when it
thinks I'm holding the down arrow key. None of these issues occur
when not running X.org. Running X -configure crashes the system,
and not even Ctrl+C or Ctrl+Z was able to uncrash it, so the
configuration had to be done manually.
-BEGIN PGP SIGNATURE-
Charset: UTF8
Version: Hush 3.0
Note: This signature can be verified at https://www.hushtools.com/verify

wsBcBAEBAgAGBQJN1ATLAAoJEKlMTST7VF+omA0H/iRSQ3inWMcin+0d9nB9odznLVTD
nh73vTCqSFpNJSaeu4hW4mJENlzbBA9C2ioy5vg1XKjxTeqXFIl2QHLiCrdGLBEB/C36
bBQd+3WKbisRzpeJOTRh7Oaqo1fEmSPdEOo2a3vlHqWDo1kUgp3maguHlzSHZN+4L0IG
Biuh7bN5jwPAgV7WM5Ne3fQPNM02kS6mP7gw/WJQP1XY6C5H0u+MqHjGNl80X+tKtZuy
b5owR2k5RbciOOK+pZ10t2fcWCumd4rtPMlmI4KsG09IP60Vffmf8ve5WAGOyzcxlZmy
W2zhCEc7Poi/ub7OaxftN829eQSNIBXeF/zwvoObPi0=
=WPkG
-END PGP SIGNATURE-

Re: [Bulk] Re: mount:mfs: mmap: Cannot allocate memory

[Kevin Chadwick]

On Wed, 18 May 2011 17:41:32 +
annathemerm...@hush.com wrote:

 I'm just trying to make it take advantage of the swap
 encryption (random keys unlike a single key I have to remember the
 password for);

bioctl or vnconfig and /dev/urandom maybe useful here

Re: mount:mfs: mmap: Cannot allocate memory

[Stuart Henderson]

On 2011-05-18, annathemerm...@hush.com annathemerm...@hush.com wrote:
 In -current it has been bumped to 2GB so you'll be able to use
 more there.

 A bit below, but now it works. I'm quite happy with stable. Unless
 current fixes my X11 keyboard issues, I'd rather not deal with
 current unless I have a second hard disk to play with. I suppose if
 I ever temporarily need more space in /tmp I'll just have to set up
 a softlink to /var/tmp or somewhere.

 However, is there any way to get mfs to use swap first rather than
 memory first?

I don't think so. (And afaik mfs doesn't give memory back once
you've used it, at least without umounting).

If I tell xorg.conf to use kbd
 or akbd, it doens't work at all, but if I tell it to use keyboard,
 sometimes it randomly decides not to notice that I've pressed
 certain keys, and is particularly reluctant to recognize the
 backspace key. Sometimes, it becomes convinced I am holding a key
 down even when I am not: this is particularly annoying when it
 thinks I'm holding the down arrow key. None of these issues occur
 when not running X.org.

This sounds rather like the behaviour you get when you need to
disable /dev/ttyC0 (see /usr/X11R6/README).

Re: mount:mfs: mmap: Cannot allocate memory

[annathemermaid]

Kevin Chadwick ma1l1i...@yahoo.co.uk wrote:
On Wed, 18 May 2011 17:41:32 +
annathemerm...@hush.com wrote:

 I'm just trying to make it take advantage of the swap
 encryption (random keys unlike a single key I have to remember 
the
 password for);

bioctl or vnconfig and /dev/urandom maybe useful here

So you are suggesting I create a boot-time script to do the 
following every time my computer starts:
1. Create a softraid or svnd encrypted partition with a key from 
/dev/urandom (without overwriting existing data)
2. Open it.
3. Create a new ffs filesystem on it.
4. Mount the new ffs filesystem, async most likely.

Well, it's a horribly convoluted idea, not at all elegant like an 
mfs or a tmpfs, but if one doesn't mind the long boot time while 
OpenBSD creates the new ffs filesystem, it ought to work.

Stuart Henderson s...@spacehopper.org wrote:
On 2011-05-18, annathemerm...@hush.com annathemerm...@hush.com 
wrote:
 In -current it has been bumped to 2GB so you'll be able to use
 more there.

 A bit below, but now it works. I'm quite happy with stable. 
Unless
 current fixes my X11 keyboard issues, I'd rather not deal with
 current unless I have a second hard disk to play with. I suppose 
if
 I ever temporarily need more space in /tmp I'll just have to set 
up
 a softlink to /var/tmp or somewhere.

 However, is there any way to get mfs to use swap first rather 
than
 memory first?

I don't think so. (And afaik mfs doesn't give memory back once
you've used it, at least without umounting).

Oh well.

Stuart Henderson s...@spacehopper.org wrote:
If I tell xorg.conf to use kbd
 or akbd, it doens't work at all, but if I tell it to use 
keyboard,
 sometimes it randomly decides not to notice that I've pressed
 certain keys, and is particularly reluctant to recognize the
 backspace key. Sometimes, it becomes convinced I am holding a key
 down even when I am not: this is particularly annoying when it
 thinks I'm holding the down arrow key. None of these issues occur
 when not running X.org.

This sounds rather like the behaviour you get when you need to
disable /dev/ttyC0 (see /usr/X11R6/README).

It states:
The wsfb driver uses whatever graphics mode the console is set to
(by Mac OS) as a dumb unaccelerated framebuffer in 8 bits per pixel.
You should be able to use 'startx' or xdm to start X without further
configuration.

To use xdm from rc.conf, it is necessary to disable /dev/ttyC0 in
/etc/ttys, change the 'status' of /dev/ttyC0 to 'off'.

I don't want to use xdm from rc.conf. I want to start it from the 
console on an as-needed basis. I don't actually like X.org. I find 
it annoying and bloated, but nonetheless, a necessary evil. Thus, I 
don't want it to start when my computer starts: I want to call on 
it when I have need of graphical programs.

mount:mfs: mmap: Cannot allocate memory

[annathemermaid]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

So, I'm trying to create put my /tmp partition on my swap.

So I have a line like this in my fstab:
swap /tmp mfs rw,nodev,nosuid,-s=2097152
I also tried changing swap to /dev/wd0b, and changing the n in -s=n
to smaller numbers.

Anyway, mount /tmp returns:
mount_mfs: mmap: Cannot allocate memory.

swapctl says neither of my swap partitions are being used.

Computer is a PowerPC, G3 iBook.

Also, is there any way to switch consoles? On i386 it was
Ctrl+Alt+F2, Ctrl+Alt+F3, and so on (in fact I think the Ctrl was
optional unless leaving X11), but it doesn't seem to work on
macppc, even with the Fn key held down too.
-BEGIN PGP SIGNATURE-
Charset: UTF8
Version: Hush 3.0
Note: This signature can be verified at https://www.hushtools.com/verify

wsBcBAEBAgAGBQJNwGKoAAoJEKlMTST7VF+oVUAIAJ9u57uLGR9zvfQzYXPWavKJ1W9a
YPsewbGtW+3iSbG2Ec4gMqqEPSzdUZ9NBW9yzaWhr1uYiwqVvOVTFQyR73Uq5Lwpmla/
dgPrX7x7cXwueRqFEYwxB//znGyIWOXDyl1qCHSf158f7JC/DGyTgnTjd3jByfuIyfxF
MJeJGXpms0B4j4wt23YwaIwhOnSUwv6fBjd9CAdapfYLgT9n462/5uF9Ml90gDE+kIsj
45KY78YKhRnj7HZJl0sh7gMGGlSTNKbeh25upyLHwgZG5pl9wF9t/VBhTMYlialmD2+A
eVwo/zmWdxXjN3zumtZXudxsGf1xxfcHyHfKMknaBJI=
=nIdn
-END PGP SIGNATURE-

Re: mount:mfs: mmap: Cannot allocate memory

[Stuart Henderson]

On 2011-05-03, annathemerm...@hush.com annathemerm...@hush.com wrote:

 So, I'm trying to create put my /tmp partition on my swap.

 So I have a line like this in my fstab:
 swap /tmp mfs rw,nodev,nosuid,-s=2097152
 I also tried changing swap to /dev/wd0b, and changing the n in -s=n
 to smaller numbers.

[assuming this is still macppc]

2097152 x 512 (bytes per sector) = 1GB

Up to OpenBSD 4.9, MAXDSIZ on macppc is 512MB so you will be
limited to -s=1048576 or slightly below.

In -current it has been bumped to 2GB so you'll be able to use
more there.

 Also, is there any way to switch consoles? On i386 it was
 Ctrl+Alt+F2, Ctrl+Alt+F3, and so on (in fact I think the Ctrl was
 optional unless leaving X11), but it doesn't seem to work on
 macppc, even with the Fn key held down too.

No separate consoles, you will need to use tmux or similar
(or X of course).

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Otto Moerbeek]

On Tue, Nov 30, 2010 at 08:35:46AM +0100, Xavier Beaudouin wrote:

 Hello,
 
 I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.
 
 Now I have new instability like this :
 
 Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot allocate
 memory
 Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot allocate
 memory
 
 I have 2Gb on this machine and login.conf like this :
 
 default:\
 :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin:\
 :umask=022:\
 :datasize-max=1512M:\
 :datasize-cur=1024M:\
 :maxproc-max=2048:\
 :maxproc-cur=1024:\
 :openfiles-cur=1024:\
 :stacksize-cur=4M:\
 :localcipher=blowfish,6:\
 :ypcipher=old:\
 :tc=auth-defaults:\
 :tc=auth-ftp-defaults:
 
 This currently make me mad, because this router handle more than 130 peers and
 is still unstable.
 
 What is needed to make openbgpd work as it should and shuttup ?
 
 (I am going to add a monit... because on production day this is not
 acceptable).
 
 Xavier

By default daemons run in the daemon login class.  Check that, also
check if you do not have stale /etc/login.conf.db file lying around. 

AFAIK I know, bgpd does not increase its limits to the max, so it does
not make sense to have different values for -max and -cur.

If these things don't help, analyzing this requires some specific bgpd
knowledge, which I do not have. 

-Otto

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Stuart Henderson]

On 2010-11-30, Xavier Beaudouin k...@oav.net wrote:
 Hello,

 I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.

 Now I have new instability like this :

 Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot allocate
 memory
 Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot allocate
 memory

Is this box acting as a route-reflector?

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Claudio Jeker]

On Tue, Nov 30, 2010 at 10:13:13AM +0100, Otto Moerbeek wrote:
 On Tue, Nov 30, 2010 at 08:35:46AM +0100, Xavier Beaudouin wrote:
 
  Hello,
  
  I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.
  
  Now I have new instability like this :
  
  Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot 
  allocate
  memory
  Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot 
  allocate
  memory
  
  I have 2Gb on this machine and login.conf like this :
  
  default:\
  :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin:\
  :umask=022:\
  :datasize-max=1512M:\
  :datasize-cur=1024M:\
  :maxproc-max=2048:\
  :maxproc-cur=1024:\
  :openfiles-cur=1024:\
  :stacksize-cur=4M:\
  :localcipher=blowfish,6:\
  :ypcipher=old:\
  :tc=auth-defaults:\
  :tc=auth-ftp-defaults:
  
  This currently make me mad, because this router handle more than 130 peers 
  and
  is still unstable.
  
  What is needed to make openbgpd work as it should and shuttup ?
  
  (I am going to add a monit... because on production day this is not
  acceptable).
  
  Xavier
 
 By default daemons run in the daemon login class.  Check that, also
 check if you do not have stale /etc/login.conf.db file lying around. 
 
 AFAIK I know, bgpd does not increase its limits to the max, so it does
 not make sense to have different values for -max and -cur.
 
 If these things don't help, analyzing this requires some specific bgpd
 knowledge, which I do not have. 
 

Maybe it is time to change the default datalimit in the RDE. So maybe
something like this may help.
bgpd needs quite a bit more (temporary) memory when running with
softreconfig. A lot of additional memory is needed on reloads and when
large sessions flap that cause a lot of UPDATE messages.

Side note: bgpd on amd64 needs quite a bit more memory then i386 because
of the 64bit pointers.
-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.302
diff -u -p -r1.302 rde.c
--- rde.c   24 Nov 2010 00:58:10 -  1.302
+++ rde.c   30 Nov 2010 10:12:56 -
@@ -18,6 +18,8 @@
 
 #include sys/types.h
 #include sys/socket.h
+#include sys/time.h
+#include sys/resource.h
 
 #include errno.h
 #include ifaddrs.h
@@ -156,6 +158,7 @@ pid_t
 rde_main(int pipe_m2r[2], int pipe_s2r[2], int pipe_m2s[2], int pipe_s2rctl[2],
 int debug)
 {
+   struct rlimitrl;
pid_tpid;
struct passwd   *pw;
struct pollfd   *pfd = NULL;
@@ -184,6 +187,13 @@ rde_main(int pipe_m2r[2], int pipe_s2r[2
 
setproctitle(route decision engine);
bgpd_process = PROC_RDE;
+
+   if (getrlimit(RLIMIT_DATA, rl) == -1)
+   fatal(getrlimit);
+   rl.rlim_cur = RLIM_INFINITY;
+   rl.rlim_max = RLIM_INFINITY;
+   if (setrlimit(RLIMIT_DATA, rl) == -1)
+   fatal(setrlimit);
 
if (setgroups(1, pw-pw_gid) ||
setresgid(pw-pw_gid, pw-pw_gid, pw-pw_gid) ||

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Otto Moerbeek]

On Tue, Nov 30, 2010 at 11:25:41AM +0100, Claudio Jeker wrote:

 On Tue, Nov 30, 2010 at 10:13:13AM +0100, Otto Moerbeek wrote:
  On Tue, Nov 30, 2010 at 08:35:46AM +0100, Xavier Beaudouin wrote:
  
   Hello,
   
   I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.
   
   Now I have new instability like this :
   
   Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot 
   allocate
   memory
   Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot 
   allocate
   memory
   
   I have 2Gb on this machine and login.conf like this :
   
   default:\
   :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin 
   /usr/local/bin:\
   :umask=022:\
   :datasize-max=1512M:\
   :datasize-cur=1024M:\
   :maxproc-max=2048:\
   :maxproc-cur=1024:\
   :openfiles-cur=1024:\
   :stacksize-cur=4M:\
   :localcipher=blowfish,6:\
   :ypcipher=old:\
   :tc=auth-defaults:\
   :tc=auth-ftp-defaults:
   
   This currently make me mad, because this router handle more than 130 
   peers and
   is still unstable.
   
   What is needed to make openbgpd work as it should and shuttup ?
   
   (I am going to add a monit... because on production day this is not
   acceptable).
   
   Xavier
  
  By default daemons run in the daemon login class.  Check that, also
  check if you do not have stale /etc/login.conf.db file lying around. 
  
  AFAIK I know, bgpd does not increase its limits to the max, so it does
  not make sense to have different values for -max and -cur.
  
  If these things don't help, analyzing this requires some specific bgpd
  knowledge, which I do not have. 
  
 
 Maybe it is time to change the default datalimit in the RDE. So maybe
 something like this may help.
 bgpd needs quite a bit more (temporary) memory when running with
 softreconfig. A lot of additional memory is needed on reloads and when
 large sessions flap that cause a lot of UPDATE messages.
 
 Side note: bgpd on amd64 needs quite a bit more memory then i386 because
 of the 64bit pointers.

Two questions: 

- why the getrlimit() if you are seting both cur and max?

- isn't it better to set cur to max?  Running with no bounds feels not ok.

-Otto



 -- 
 :wq Claudio
 
 Index: rde.c
 ===
 RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
 retrieving revision 1.302
 diff -u -p -r1.302 rde.c
 --- rde.c 24 Nov 2010 00:58:10 -  1.302
 +++ rde.c 30 Nov 2010 10:12:56 -
 @@ -18,6 +18,8 @@
  
  #include sys/types.h
  #include sys/socket.h
 +#include sys/time.h
 +#include sys/resource.h
  
  #include errno.h
  #include ifaddrs.h
 @@ -156,6 +158,7 @@ pid_t
  rde_main(int pipe_m2r[2], int pipe_s2r[2], int pipe_m2s[2], int 
 pipe_s2rctl[2],
  int debug)
  {
 + struct rlimitrl;
   pid_tpid;
   struct passwd   *pw;
   struct pollfd   *pfd = NULL;
 @@ -184,6 +187,13 @@ rde_main(int pipe_m2r[2], int pipe_s2r[2
  
   setproctitle(route decision engine);
   bgpd_process = PROC_RDE;
 +
 + if (getrlimit(RLIMIT_DATA, rl) == -1)
 + fatal(getrlimit);
 + rl.rlim_cur = RLIM_INFINITY;
 + rl.rlim_max = RLIM_INFINITY;
 + if (setrlimit(RLIMIT_DATA, rl) == -1)
 + fatal(setrlimit);
  
   if (setgroups(1, pw-pw_gid) ||
   setresgid(pw-pw_gid, pw-pw_gid, pw-pw_gid) ||

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Xavier Beaudouin]

Hello,

Le 30 nov. 2010 ` 11:03, Stuart Henderson a icrit :

 On 2010-11-30, Xavier Beaudouin k...@oav.net wrote:
 Hello,

 I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.

 Now I have new instability like this :

 Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot
allocate
 memory
 Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot
allocate
 memory

 Is this box acting as a route-reflector?



No route reflector at all.

It is a peering box with 3 IX, on transit and 3 ibgp session (count 6, because
I use IPv6).

Configuration of this box on demand .

Xavier

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Claudio Jeker]

On Tue, Nov 30, 2010 at 12:06:38PM +0100, Otto Moerbeek wrote:
 On Tue, Nov 30, 2010 at 11:25:41AM +0100, Claudio Jeker wrote:
 
  On Tue, Nov 30, 2010 at 10:13:13AM +0100, Otto Moerbeek wrote:
   On Tue, Nov 30, 2010 at 08:35:46AM +0100, Xavier Beaudouin wrote:
   
Hello,

I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.

Now I have new instability like this :

Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot 
allocate
memory
Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot 
allocate
memory

I have 2Gb on this machine and login.conf like this :

default:\
:path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin 
/usr/local/bin:\
:umask=022:\
:datasize-max=1512M:\
:datasize-cur=1024M:\
:maxproc-max=2048:\
:maxproc-cur=1024:\
:openfiles-cur=1024:\
:stacksize-cur=4M:\
:localcipher=blowfish,6:\
:ypcipher=old:\
:tc=auth-defaults:\
:tc=auth-ftp-defaults:

This currently make me mad, because this router handle more than 130 
peers and
is still unstable.

What is needed to make openbgpd work as it should and shuttup ?

(I am going to add a monit... because on production day this is not
acceptable).

Xavier
   
   By default daemons run in the daemon login class.  Check that, also
   check if you do not have stale /etc/login.conf.db file lying around. 
   
   AFAIK I know, bgpd does not increase its limits to the max, so it does
   not make sense to have different values for -max and -cur.
   
   If these things don't help, analyzing this requires some specific bgpd
   knowledge, which I do not have. 
   
  
  Maybe it is time to change the default datalimit in the RDE. So maybe
  something like this may help.
  bgpd needs quite a bit more (temporary) memory when running with
  softreconfig. A lot of additional memory is needed on reloads and when
  large sessions flap that cause a lot of UPDATE messages.
  
  Side note: bgpd on amd64 needs quite a bit more memory then i386 because
  of the 64bit pointers.
 
 Two questions: 
 
 - why the getrlimit() if you are seting both cur and max?
 

Because I first thought of only setting cur.

 - isn't it better to set cur to max?  Running with no bounds feels not ok.
 

First I planned to do the same thing but then decided against it. But
maybe I'm wrong and it would be better to just set cur to max and hope
people have sensible max values.

-- 
:wq Claudio

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Xavier Beaudouin]

Hi Claudio,

 Maybe it is time to change the default datalimit in the RDE. So maybe
 something like this may help.
 bgpd needs quite a bit more (temporary) memory when running with
 softreconfig. A lot of additional memory is needed on reloads and when
 large sessions flap that cause a lot of UPDATE messages.

 Side note: bgpd on amd64 needs quite a bit more memory then i386 because
 of the 64bit pointers.

Yeah... That's why I have 2G on this machine I hope this should be enougth
... 1G on i386 was ok... So...

I will tell you if this fix my problem... (if you don't hear me... so it can
be fixed...) ping me if you need a clear status.

Cheers.
Xavier

 --
 :wq Claudio

 Index: rde.c
 ===
 RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
 retrieving revision 1.302
 diff -u -p -r1.302 rde.c
 --- rde.c 24 Nov 2010 00:58:10 -  1.302
 +++ rde.c 30 Nov 2010 10:12:56 -
 @@ -18,6 +18,8 @@

 #include sys/types.h
 #include sys/socket.h
 +#include sys/time.h
 +#include sys/resource.h

 #include errno.h
 #include ifaddrs.h
 @@ -156,6 +158,7 @@ pid_t
 rde_main(int pipe_m2r[2], int pipe_s2r[2], int pipe_m2s[2], int
pipe_s2rctl[2],
 int debug)
 {
 + struct rlimitrl;
   pid_tpid;
   struct passwd   *pw;
   struct pollfd   *pfd = NULL;
 @@ -184,6 +187,13 @@ rde_main(int pipe_m2r[2], int pipe_s2r[2

   setproctitle(route decision engine);
   bgpd_process = PROC_RDE;
 +
 + if (getrlimit(RLIMIT_DATA, rl) == -1)
 + fatal(getrlimit);
 + rl.rlim_cur = RLIM_INFINITY;
 + rl.rlim_max = RLIM_INFINITY;
 + if (setrlimit(RLIMIT_DATA, rl) == -1)
 + fatal(setrlimit);

   if (setgroups(1, pw-pw_gid) ||
   setresgid(pw-pw_gid, pw-pw_gid, pw-pw_gid) ||

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Xavier Beaudouin]

Hi Claudio,

Le 30 nov. 2010 ` 17:45, Xavier Beaudouin a icrit :

 Hi Claudio,

 Maybe it is time to change the default datalimit in the RDE. So maybe
 something like this may help.
 bgpd needs quite a bit more (temporary) memory when running with
 softreconfig. A lot of additional memory is needed on reloads and when
 large sessions flap that cause a lot of UPDATE messages.

 Side note: bgpd on amd64 needs quite a bit more memory then i386 because
 of the 64bit pointers.

 Yeah... That's why I have 2G on this machine I hope this should be
enougth
 ... 1G on i386 was ok... So...

 I will tell you if this fix my problem... (if you don't hear me... so it
can
 be fixed...) ping me if you need a clear status.


This patch, opens another problem, seems that FIB is not updated at all when
applied.

I reverted to openbgp 4.8 release.

:(
Xavier

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Claudio Jeker]

On Tue, Nov 30, 2010 at 06:24:32PM +0100, Xavier Beaudouin wrote:
 Hi Claudio,
 
 Le 30 nov. 2010 ` 17:45, Xavier Beaudouin a icrit :
 
  Hi Claudio,
  
  Maybe it is time to change the default datalimit in the RDE. So maybe
  something like this may help.
  bgpd needs quite a bit more (temporary) memory when running with
  softreconfig. A lot of additional memory is needed on reloads and when
  large sessions flap that cause a lot of UPDATE messages.
  
  Side note: bgpd on amd64 needs quite a bit more memory then i386 because
  of the 64bit pointers.
  
  Yeah... That's why I have 2G on this machine I hope this should be 
  enougth
  ... 1G on i386 was ok... So...
  
  I will tell you if this fix my problem... (if you don't hear me... so it can
  be fixed...) ping me if you need a clear status.
  
 
 This patch, opens another problem, seems that FIB is not updated at all when 
 applied.
 
 I reverted to openbgp 4.8 release.
 

You sure you have
http://ftp.openbsd.org/pub/OpenBSD/patches/4.8/common/001_bgpd.patch
installed? Since that could be the cause of your problem.

-- 
:wq Claudio

Re: OpenBGPD fatal in RDE : cannot allocate memory

[Xavier Beaudouin]

Hi Claudio,

Le 30 nov. 2010 ` 19:38, Claudio Jeker a icrit :
 This patch, opens another problem, seems that FIB is not updated at all
when applied.

 I reverted to openbgp 4.8 release.


 You sure you have
 http://ftp.openbsd.org/pub/OpenBSD/patches/4.8/common/001_bgpd.patch
 installed? Since that could be the cause of your problem.

Both patch applied... Well I will see if those 2 patches fixes the problem.

Sincerly,
Xavier

OpenBGPD fatal in RDE : cannot allocate memory

[Xavier Beaudouin]

Hello,

I have updated a openbgpd router from OpenBSD 4.7 i386 to 4.8 amd64.

Now I have new instability like this :

Nov 29 21:25:22 core-3 bgpd[28895]: fatal in RDE: path_alloc: Cannot allocate
memory
Nov 30 02:01:47 core-3 bgpd[5522]: fatal in RDE: up_generate: Cannot allocate
memory

I have 2Gb on this machine and login.conf like this :

default:\
:path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin:\
:umask=022:\
:datasize-max=1512M:\
:datasize-cur=1024M:\
:maxproc-max=2048:\
:maxproc-cur=1024:\
:openfiles-cur=1024:\
:stacksize-cur=4M:\
:localcipher=blowfish,6:\
:ypcipher=old:\
:tc=auth-defaults:\
:tc=auth-ftp-defaults:

This currently make me mad, because this router handle more than 130 peers and
is still unstable.

What is needed to make openbgpd work as it should and shuttup ?

(I am going to add a monit... because on production day this is not
acceptable).

Xavier

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Ruy Bento]

On 21-06-2010 22:44, Ruy Bento wrote:


...



My question is: In this small env. (100 MB - RAM) I need to change the
Kernel memory or other sysctl value, which one?



Thank you for all your replys and comments.

In 4.6 everything work perfect, so what happen 4.6 - 4.7, it need more mem?

And if I can:
set limit table-entries 500


And with all daemons load in mem I have:

36 processes:  35 idle, 1 on processor
CPU states:  0.5% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.5% 
idle

Memory: Real: 20M/45M act/tot  Free: 41M  Swap: 0K/161M used/tot

What a perfect world 

So with 41MB free i could load more kernel ...



My other servers: Core 2, i5, i7 with lots of mem (4 or 8 GB).

This and the SUN its to test and see the OpenBSD continue to run happily 
for ever :-) :-)



Thank you for your great effort and work.

Best regards,
Ruy Benton

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Stuart Henderson]

On 2010-06-21, Ruy Bento r...@madeira.dyndns.org wrote:
 spamd_black=YES # set to YES to run spamd without greylisting

you don't want blacklist-only mode if you have limited RAM.

pfctl: Cannot allocate memory and spamd-setup -bd

[Ruy Bento]

Hi,

I have a server with:

OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium II (GenuineIntel 686-class, 512KB L2 cache) 234MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,MMX
real mem  = 100233216 (95MB)
avail mem = 87961600 (83MB)


With sendmail and spamd in blacklist (/etc/rc.conf.local):

spamd_flags=-bv   # for normal use:  and see spamd(8)
spamd_black=YES # set to YES to run spamd without greylisting
spamlogd_flags=-i rl0 # use eg. -i interface and see spamlogd(8)




/etc/mail/spamd.conf:

with:china:korea:  it's ok

with:uatraps:china:korea:  -  pfctl: Cannot allocate memory.

or
:nixspam::china:korea: -  pfctl: Cannot allocate memory.

with the shell /usr/libexec/spamd-setup -bd


vmstat -m
_

pfrke_plain   92196163 1175   429   39435   168 0 
  88


.

In use 2275K, total allocated 3808K; utilization 59.7%




pstat -s
Device  512-blocks UsedAvail Capacity  Priority
swap_device 3299800   329980 0%0


In OpenBSD 4.6 the same hardware and config ... no problem.

I try several setups: Core2 Duo with 1 GB RAM (the same config) 4.6 and 
4.7, works.


But ... 4.6 and 4.7 with 128MB, 4.7 give the same error: pfctl: Cannot 
allocate memory. 


So I change

set limit tables 1
set limit table-entries 500

in pf.conf, but no luck.

My question is: In this small env. (100 MB - RAM) I need to change the 
Kernel memory or other sysctl value, which one?


I work with OpenBSd for more 10 years Intel, AMD, PPC - Webhosting, 
Servers, Firewalls.


Thank you for your great effort and work.

Best regards,
Ruy Benton

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Theo de Raadt]

 avail mem = 87961600 (83MB)

 with:uatraps:china:korea:  -  pfctl: Cannot allocate memory.

Not enough kernel memory.

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Chris Bennett]

Ruy Bento wrote:

Hi,

I have a server with:

OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium II (GenuineIntel 686-class, 512KB L2 cache) 234MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,MMX
real mem  = 100233216 (95MB)
avail mem = 87961600 (83MB)


With sendmail and spamd in blacklist (/etc/rc.conf.local):

spamd_flags=-bv   # for normal use:  and see spamd(8)
spamd_black=YES # set to YES to run spamd without greylisting
spamlogd_flags=-i rl0 # use eg. -i interface and see spamlogd(8)




/etc/mail/spamd.conf:

with:china:korea:  it's ok

with:uatraps:china:korea:  -  pfctl: Cannot allocate memory.

or
:nixspam::china:korea: -  pfctl: Cannot allocate memory.

with the shell /usr/libexec/spamd-setup -bd


vmstat -m
_

pfrke_plain   92196163 1175   429   39435   168 0 
  88


.

In use 2275K, total allocated 3808K; utilization 59.7%




pstat -s
Device  512-blocks UsedAvail Capacity  Priority
swap_device 3299800   329980 0%0


In OpenBSD 4.6 the same hardware and config ... no problem.

I try several setups: Core2 Duo with 1 GB RAM (the same config) 4.6 
and 4.7, works.


But ... 4.6 and 4.7 with 128MB, 4.7 give the same error: pfctl: 
Cannot allocate memory. 


So I change

set limit tables 1
set limit table-entries 500

in pf.conf, but no luck.

My question is: In this small env. (100 MB - RAM) I need to change the 
Kernel memory or other sysctl value, which one?


I work with OpenBSd for more 10 years Intel, AMD, PPC - Webhosting, 
Servers, Firewalls.


Thank you for your great effort and work.

Best regards,
Ruy Benton


OK, I'm game to ask after seeing Theo's response. I actually have some 
equipment like this, not that I use it this way, normally.


So, change a setting or rewrite things to fit better in this small 
memory space?
I was actually using that laptop to make some pretty extensive website 
changes last year, while traveling with little internet access.
Filled those boring hours while I waked up hours before the world back 
then. No regrets having brought that old thing with me! :)


Chris Bennett

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Theo de Raadt]

  OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010
  dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
  cpu0: Intel Pentium II (GenuineIntel 686-class, 512KB L2 cache) 234MHz
  cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,MMX
  real mem  = 100233216 (95MB)
  avail mem = 87961600 (83MB)


  pstat -s
  Device  512-blocks UsedAvail Capacity  Priority
  swap_device 3299800   329980 0%0


 OK, I'm game to ask after seeing Theo's response. I actually have some 
 equipment like this, not that I use it this way, normally.

You're kidding.

 So, change a setting or rewrite things to fit better in this small 
 memory space?

There is no solution.  The tables are in kernel memory.

The kernel isn't going to go out to swap space to check if packets
should flow through.  Would anyone want that?  No, of course not.

 I was actually using that laptop to make some pretty extensive website 
 changes last year, while traveling with little internet access.
 Filled those boring hours while I waked up hours before the world back 
 then. No regrets having brought that old thing with me! :)

Laptops tend to have more than 83MB of available memory.

Re: pfctl: Cannot allocate memory and spamd-setup -bd

[Chris Bennett]

Theo de Raadt wrote:

OpenBSD 4.7 (GENERIC) #558: Wed Mar 17 20:46:15 MDT 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium II (GenuineIntel 686-class, 512KB L2 cache) 234MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,MMX
real mem  = 100233216 (95MB)
avail mem = 87961600 (83MB)
  



  

pstat -s
Device  512-blocks UsedAvail Capacity  Priority
swap_device 3299800   329980 0%0
  



  
OK, I'm game to ask after seeing Theo's response. I actually have some 
equipment like this, not that I use it this way, normally.



You're kidding.

  
So, change a setting or rewrite things to fit better in this small 
memory space?



There is no solution.  The tables are in kernel memory.

The kernel isn't going to go out to swap space to check if packets
should flow through.  Would anyone want that?  No, of course not.

  
I was actually using that laptop to make some pretty extensive website 
changes last year, while traveling with little internet access.
Filled those boring hours while I waked up hours before the world back 
then. No regrets having brought that old thing with me! :)



Laptops tend to have more than 83MB of available memory.


  

No kidding, that sucker only has 128MB, after I added memory.
And I'm serious, I really did feel perfectly safe bringing it to parts 
unknown, where the locals said I really shouldn't be carrying it around 
in the streets.

I still have that antique, right here, working alongside a real computer.

But anyway, thanks for answering the question clearly

Chris Bennett

Re: Error in fsck_ffs: cannot allocate memory for inode cache

[Jan Stary]

On Feb 26 11:23:16, Nicolas Letellier wrote:
 Hello m...@.
 
 My server crashed this night. However, OpenBSD 4.4 does not want to
 boot. There are errors on my /home partition (125 Go, ffs type).
 
 The following file system had unexpected inconsistency
 ffs: /dev/rwd0g (/home)
 
 I must run fsck_ffs manually.
 
 However, when I run it :
 
 fsck_ffs /dev/rwd0g
 
 1824 DUP I=2678
 1825 DUP I=2678
 
 cannot allocate memory for inode cache.

 I tried to reboot, same problem. I didn't find informations about it in
 fsck_ffs manpage.

http://www.openbsd.org/faq/faq14.html#LargeDrive says

A rough guideline is the system should have at least 1M of
available memory for every 1G of disk space to successfully fsck
the disk. 

How much memory do you have?
Do you have a swap partition?

 Do you have some advices to repair this disk,

the disk is probably OK, it's the filesystem that is broken.

 and for booting my OpenBSD?

(1) Read the FAQ
(2) Don't run fsck_ffs, run fsck.
(3) If that doesn't help, either put mor memory into your machine,
or just fsck the disk on another machine.

Jan

Re: Error in fsck_ffs: cannot allocate memory for inode cache

[Jan Stary]

 Thanks for your advice but it did not work. So, we bought another hard
 disks.

What for? If you didn't need the data that was on the broken /home,
you could have just newfs'ed the partition ...

Jan

Error in fsck_ffs: cannot allocate memory for inode cache

[Nicolas Letellier]

Hello m...@.

My server crashed this night. However, OpenBSD 4.4 does not want to
boot. There are errors on my /home partition (125 Go, ffs type).

The following file system had unexpected inconsistency
ffs: /dev/rwd0g (/home)

I must run fsck_ffs manually.

However, when I run it :

fsck_ffs /dev/rwd0g

1824 DUP I=2678
1825 DUP I=2678

cannot allocate memory for inode cache.

I tried to reboot, same problem. I didn't find informations about it in
fsck_ffs manpage.

Do you have some advices to repair this disk, and for booting my
OpenBSD?

Regards,

-- 
Nicolas

Re: Error in fsck_ffs: cannot allocate memory for inode cache

[Otto Moerbeek]

On Thu, Feb 26, 2009 at 11:23:16AM +0100, Nicolas Letellier wrote:

 Hello m...@.
 
 My server crashed this night. However, OpenBSD 4.4 does not want to
 boot. There are errors on my /home partition (125 Go, ffs type).
 
 The following file system had unexpected inconsistency
 ffs: /dev/rwd0g (/home)
 
 I must run fsck_ffs manually.
 
 However, when I run it :
 
 fsck_ffs /dev/rwd0g
 
 1824 DUP I=2678
 1825 DUP I=2678
 
 cannot allocate memory for inode cache.

Run fsck by using the fsck(8) command. It sets up the max resource
limits before calling fsck_ffs(8). That might save your day. 

-Otto
 
 I tried to reboot, same problem. I didn't find informations about it in
 fsck_ffs manpage.
 
 Do you have some advices to repair this disk, and for booting my
 OpenBSD?
 
 Regards,
 
 -- 
 Nicolas

Re: Error in fsck_ffs: cannot allocate memory for inode cache

[Nicolas Letellier]

Le Thu, 26 Feb 2009 13:50:12 +0100,
Otto Moerbeek o...@drijf.net a icrit :

 On Thu, Feb 26, 2009 at 11:23:16AM +0100, Nicolas Letellier wrote:

  Hello m...@.
 
  My server crashed this night. However, OpenBSD 4.4 does not want to
  boot. There are errors on my /home partition (125 Go, ffs type).
 
  The following file system had unexpected inconsistency
  ffs: /dev/rwd0g (/home)
 
  I must run fsck_ffs manually.
 
  However, when I run it :
 
  fsck_ffs /dev/rwd0g
 
  1824 DUP I=2678
  1825 DUP I=2678
 
  cannot allocate memory for inode cache.

 Run fsck by using the fsck(8) command. It sets up the max resource
 limits before calling fsck_ffs(8). That might save your day.

   -Otto

Thanks for your advice but it did not work. So, we bought another hard
disks.

Regards,

--
Nicolas

umount /mnt2: Cannot allocate memory (strange bug)

[bofh]

Is this a bug, or just me running out of ram?  Look at the multiple
mount attempts at the bottom - 3rd one works?!  Why?  Should I
sendbug?  This box is not on the network yet, which is why I'm asking.

dmesg at the bottom

# uname -a
OpenBSD newurd.spidernet.to 4.4 GENERIC#1021 i386

# df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/wd0a 72.4G6.8G   62.0G10%/
/dev/sd0a  186G159G   17.7G90%/mnt

# newfs /dev/rsd1a  [output munched]
# mkdir /mnt2;
# mount /dev/sd1a /mnt2
# rsync -av /mnt/* /mnt2/  [output munched]
# df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/wd0a 72.4G6.8G   62.0G10%/
/dev/sd0a  186G159G   17.7G90%/mnt
/dev/sd1a  229G166G   51.7G76%/mnt2
# umount /mnt2
umount: /mnt2: Cannot allocate memory
# sync
# w
10:14AM  up 15 days, 20:58, 1 user, load averages: 0.21, 0.15, 0.10
USERTTY FROM  LOGIN@  IDLE WHAT
root p4 apple2   10:12AM 0 w
# top
# lsof /mnt2
# umount /mnt2
# df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/wd0a 72.4G6.8G   62.0G10%/
/dev/sd0a  186G159G   17.7G90%/mnt
# fsck /dev/sd1a
** /dev/rsd1a
** File system is clean; not checking
# fsck -f /dev/sd1a
** /dev/rsd1a
** File system is already clean
** Last Mounted on /mnt2
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
450839 files, 87042327 used, 33123735 free (7639 frags, 4139512
blocks, 0.0% fragmentation)
# mount /dev/sd1a /mnt2
mount_ffs: /dev/sd1a on /mnt2: Cannot allocate memory
# top
# mount /dev/sd1a /mnt2
mount_ffs: /dev/sd1a on /mnt2: Cannot allocate memory
# mount /dev/sd1a /mnt2

[Why did it work on the 3rd attempt??!]

# dmesg
OpenBSD 4.4 (GENERIC) #1021: Tue Aug 12 17:16:55 MDT 2008
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: VIA Samuel 2 (CentaurHauls 686-class) 797 MHz
cpu0: FPU,DE,TSC,MSR,MTRR,PGE,MMX
real mem  = 1039691776 (991MB)
avail mem = 996839424 (950MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 08/19/04, BIOS32 rev. 0 @
0xfdad0, SMBIOS rev. 2.3 @ 0xf0630 (24 entries)
bios0: vendor American Megatrends Inc. version Version 07.00T date 04/02/01
bios0: ECS M789CG
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
acpi at bios0 function 0x0 not configured
pcibios0 at bios0: rev 2.1 @ 0xf/0x1
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xf79a0/144 (7 entries)
pcibios0: PCI Interrupt Router at 000:17:0 (VIA VT8235 ISA rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc/0x7e00 0xc8000/0x3000!
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 VIA VT8623 PCI rev 0x00
ppb0 at pci0 dev 1 function 0 VIA VT8633 AGP rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 VIA CLE266 rev 0x03
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
agp0 at vga1: v2, aperture at 0xe000, size 0x1000
drm at vga1 unsupported
uhci0 at pci0 dev 16 function 0 VIA VT83C572 USB rev 0x80: irq 11
uhci1 at pci0 dev 16 function 1 VIA VT83C572 USB rev 0x80: irq 5
uhci2 at pci0 dev 16 function 2 VIA VT83C572 USB rev 0x80: irq 12
ehci0 at pci0 dev 16 function 3 VIA VT6202 USB rev 0x82: irq 10
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 VIA EHCI root hub rev 2.00/1.00 addr 1
viapm0 at pci0 dev 17 function 0 VIA VT8235 ISA rev 0x00
iic0 at viapm0
spdmem0 at iic0 addr 0x50: 1GB DDR SDRAM non-parity PC2100CL2.5
pciide0 at pci0 dev 17 function 1 VIA VT82C571 IDE rev 0x06: ATA133,
channel 0 configured to compatibility, channel 1 configured to
compatibility
wd0 at pciide0 channel 0 drive 0: WDC WD800JB-00JJA0
wd0: 16-sector PIO, LBA, 76319MB, 156301488 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
pciide0: channel 1 disabled (no drives)
auvia0 at pci0 dev 17 function 5 VIA VT8233 AC97 rev 0x50: irq 12
ac97: codec id 0x434d4983 (C-Media Electronics CMI9761A+)
audio0 at auvia0
vr0 at pci0 dev 18 function 0 VIA RhineII-2 rev 0x74: irq 11,
address 00:1b:b9:9f:10:7b
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 10: OUI
0x004063, model 0x0032
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 VIA UHCI root hub rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 VIA UHCI root hub rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 VIA UHCI root hub rev 1.00/1.00 addr 1
isa0 at mainbus0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
it0 at isa0 port 0x2e/2: IT8705F rev 0x03, EC port 0x290
npx0 at isa0 port

fsck_msdos results in No space for FAT (Cannot allocate memory)

[Thilo Pfennig]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I have a FAT32 disk on wd1i as a disk for interchange with other
computers. On every bootup I get the message No space for FAT (Cannot
allocate memory) and then get into a shell where I should check
manually. But I can mount the disk without any problem. I have found
this as it seems to be a bug in NetBSD and FreeBSD but did not find it
mentioned on this list.

Anybody can give me a hint of what could be wrong. My system is rather
current but I had not updated after december 4th.

Regards,
Thilo
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAklag8IACgkQK3WHfZT3wS37rgCfb5mR9suas+8f+UOSVsSyr4XM
TrIAoIOVqLR54fsjPWAAPLfo0/wum3Cw
=y60X
-END PGP SIGNATURE-

Re: fsck_msdos results in No space for FAT (Cannot allocate memory)

[ropers]

2008/12/30 Thilo Pfennig t...@pfennigsolutions.de:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 I have a FAT32 disk on wd1i as a disk for interchange with other
 computers. On every bootup I get the message No space for FAT (Cannot
 allocate memory) and then get into a shell where I should check
 manually.

Okay, so when you're booting OpenBSD -CURRENT, you're getting this
error? What does your disklabel look like? Is the FAT32 disk a
separate physical disk, and/or separate slice, and (disklabel)
partition)? Where does it come in, when does it get mounted? What does
your fstab(5) say?

But I can mount the disk without any problem. I have found
 this as it seems to be a bug in NetBSD and FreeBSD but did not find it
 mentioned on this list.

I don't understand what you mean by that. What is this? Are you
getting the same error message with these other BSDs? What makes you
think it's a bug in Net/FreeBSD? Is this a known/confirmed bug there?
Yadda, yadda, yadda.

 Anybody can give me a hint of what could be wrong. My system is rather
 current but I had not updated after december 4th.

I guess a dmesg also couldn't hurt, but whatever.

Sorry, I don't have answers, only questions.

regards,
--ropers

Re: fsck_msdos results in No space for FAT (Cannot allocate memory)

[Stuart Henderson]

On 2008-12-30, Thilo Pfennig t...@pfennigsolutions.de wrote:

 I have a FAT32 disk on wd1i as a disk for interchange with other
 computers. On every bootup I get the message No space for FAT (Cannot
 allocate memory) and then get into a shell where I should check
 manually.

This is coming from fsck_msdos. Either one of NumClusters, FATsecs
or BytesPerSec stored in the bootblock of the partition is mangled,
or it's just too big a partition for fsck_msdos to handle in the
available memory.

Build fsck_msdos with debugging symbols and run it under gdb, set
a breakpoint after readboot() is called in check.c (line 90 or so
should be ok), then when it's triggered, print boot.

 But I can mount the disk without any problem.

Unless you do it manually, there won't be an fsck then.

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Stuart Henderson]

On 2008-10-07, Henning Brauer [EMAIL PROTECTED] wrote:
 * Christophe Rioux [EMAIL PROTECTED] [2008-10-07 13:15]:
  # pfctl -f pf.conf
  pfctl: failed to create table __automatic_e11ee055_282 in :
  Cannot allocate memory

 so the ruleset optiomizer optimized a large list of addresses into a
 table for you. then allocating memory for that table failed.

it optimized the large lists to _multiple_ tables, one per rule
referencing them. consider how the implementation of port lists
works e.g. { 80 443 25 110 } expands to 4 rules and you'll see
how this can affect some rulesets. (pfctl -nvf pf.conf shows how
this works visually).

here's a smarter workaround that lets you turn the optimizer back on:

identify the macros for frequently used groups of hosts, especially
where large numbers of ports are involved, and convert them to
tables yourself.

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Stuart Henderson]

On 2008-10-07, Christophe Rioux [EMAIL PROTECTED] wrote:
 # pfctl -f pf.conf
 pfctl: failed to create table __automatic_e11ee055_282 in :
 Cannot allocate
 memory
 pfctl in free(): error: chunk is already free
 Abort trap (core dumped)

that's a PF ruleset optimizer bug. I think it was fixed after 4.3
but without a copy of your pf.conf (ideally reduced to the minimum
that shows the fault) we can't test this, so you'd have to try it
with newer pfctl code.

 Which value do I need to change so that it can work without problems ?

workaround:
set ruleset-optimization none

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Henning Brauer]

ok, we have multiple issues here.

* Christophe Rioux [EMAIL PROTECTED] [2008-10-07 13:15]:
 NameSize Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg
 Idle
 ...
 pfrktable   1240 54345 4434   334 0   334   334 0   334
 0
 ...
 The rest seems to be OK.

the 5 failed requests probably map to the error you see. the pfrktable
pool has no hard limit. it looks like you simply request more memory
in one chunk than available in the pool at that time, and the backend
allocator doesn't get to run at that point. I'm not entirely sure why
the allocator doesn't run in that context, will find out.

 I try to increase the memory for the table in pf.conf
 set limit tables  1 # default   1000
 But nothing is changed.

there is no button you can push to fix this.

  # pfctl -f pf.conf
  pfctl: failed to create table __automatic_e11ee055_282 in :
  Cannot allocate memory

so the ruleset optiomizer optimized a large list of addresses into a
table for you. then allocating memory for that table failed.

  pfctl in free(): error: chunk is already free

that should not happen either... I don't see where this is coming from
tho. if you can reproduce this problem at will, please build a pfctl
with debug symols (cd /usr/src/sbin/pfctl; make clean; DEBUG=-g make)
and run it under gdb (gdb /usr/src/sbin/pfctl/obj/pfctl, on the gdb
prompt run -f /etc/pf.conf and when it dies type bt and send me
the entire output from starting gdb to the end of the trace to henning@)

  Somebody an idea ?

only a stupid workaround for the moment... set optimization none

-- 
Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED]
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting - Hamburg  Amsterdam

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Christophe Rioux]

Hi

I check some information on my system:

vmstat -m

NameSize Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg
Idle
...
pfrktable   1240 54345 4434   334 0   334   334 0   334
0
...
The rest seems to be OK.

I try to increase the memory for the table in pf.conf
set limit tables  1 # default   1000

But nothing is changed.

By doing a pfctl -F rules -f /etc/pf.conf the rules are implemented.

Which value do I need to change so that it can work without problems ?

Regards


 -Message d'origine-
 De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
 De la part de Christophe Rioux
 Envoyi : mardi 16 septembre 2008 17:33
 @ : misc@openbsd.org
 Objet : OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

 Hi,

 During a manual migration (installation with 4.3 CD, and renew the
 configuration manually), I get this error message:

 # pfctl -f pf.conf
 pfctl: failed to create table __automatic_e11ee055_282 in :
 Cannot allocate
 memory
 pfctl in free(): error: chunk is already free
 Abort trap (core dumped)

 I check some informations:

 New system:
 pfctl -sm
 stateshard limit1
 src-nodes hard limit1
 frags hard limit 5000
 tableshard limit 1000
 table-entries hard limit   20
 Top:
 27 processes:  26 idle, 1 on processor
 CPU states:  2.7% user,  0.0% nice,  0.1% system,  0.1%
 interrupt, 97.1%
 idle
 Memory: Real: 19M/95M act/tot  Free: 401M  Swap: 0K/518M used/tot

 Old system:
 pfctl -sm
 stateshard limit1
 src-nodes hard limit1
 frags hard limit 5000
 tableshard limit 1000
 table-entries hard limit   10
 Top:
 31 processes:  1 running, 29 idle, 1 on processor
 CPU states:  0.0% user,  0.0% nice,  100% system,  0.0%
 interrupt,  0.0%
 idle
 Memory: Real: 16M/67M act/tot  Free: 179M  Swap: 0K/2002M used/tot

 Somebody an idea ?

 As next, I found another issue. I try to send mail from the
 new serveur: I
 get the error message (in /var/spool/clientmqueue). Actually
 pf is disable.
 T1221580802
 K1221580802
 N1
 P31471
 I0/0/130550
 MDeferred: [127.0.0.1]: Network is unreachable
 Frs
 $_localhost
 $r
 $slocalhost
 ${daemon_flags}c u
 SMAILER-DAEMON
 MDeferred: [127.0.0.1]: Network is unreachable
 C:root

 Thanks for help
 Christophe

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Henning Brauer]

* Stuart Henderson [EMAIL PROTECTED] [2008-10-07 15:16]:
 On 2008-10-07, Christophe Rioux [EMAIL PROTECTED] wrote:
  # pfctl -f pf.conf
  pfctl: failed to create table __automatic_e11ee055_282 in :
  Cannot allocate
  memory
  pfctl in free(): error: chunk is already free
  Abort trap (core dumped)
 
 that's a PF ruleset optimizer bug. I think it was fixed after 4.3
 but without a copy of your pf.conf (ideally reduced to the minimum
 that shows the fault) we can't test this, so you'd have to try it
 with newer pfctl code.

oh I totally missed that this is on 4.3...
on 4.4, the pool_gets are PR_WAITOK and thus the backend allocator
does have a chance to run and this should just work.

despite an optimizer bug, but I don't remember any really (doesn't
have to say much tho)

  Which value do I need to change so that it can work without problems ?
 workaround:
 set ruleset-optimization none

yes, ruleset-optimization, not just optimization as I wrote earlier

-- 
Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED]
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting - Hamburg  Amsterdam

Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

[Christophe Rioux]

The workaround work fine ... thanks

 -Message d'origine-
 De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
 De la part de Stuart Henderson
 Envoyi : mardi 7 octobre 2008 15:10
 @ : misc@openbsd.org
 Objet : Re: OpenBSD 3.9 - 4.3: pfctl Cannot allocate memory

 On 2008-10-07, Christophe Rioux [EMAIL PROTECTED] wrote:
  # pfctl -f pf.conf
  pfctl: failed to create table __automatic_e11ee055_282 in :
  Cannot allocate
  memory
  pfctl in free(): error: chunk is already free
  Abort trap (core dumped)

 that's a PF ruleset optimizer bug. I think it was fixed after 4.3
 but without a copy of your pf.conf (ideally reduced to the minimum
 that shows the fault) we can't test this, so you'd have to try it
 with newer pfctl code.

  Which value do I need to change so that it can work without
 problems ?

 workaround:
 set ruleset-optimization none