Re: Fwd: Booting BSD on a Libreboot system - documentation needed

[Aaron Mason]

Holy frijole, just reading some of the responses from the some people
in GNU - I'm at the point where I'm not entirely convinced that GNU
isn't a cult, with Stallman as the high almighty leader.

On Wed, Oct 5, 2016 at 7:31 PM, Leah Rowe  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Thanks.
>
> On 05/10/16 05:39, Scott Bonds wrote:
>> I've started a stab at it. My x200 is in pieces at the moment and I
>> want to retest my instructions before I submit a PR, so it may be a
>> couple weeks.
>>
>> On 10/05, Leah Rowe wrote:
>>>
>
> - --
> Leah Rowe
>
> Libreboot developer
>
> Use free software. Free as in freedom.
> https://en.wikipedia.org/wiki/Free_software
>
> Use a free operating system, GNU/Linux.
>
> Use a free BIOS.
> https://libreboot.org/
>
> Support computer user freedom.
> https://peers.community/
>
> Minifree Ltd, trading as Ministry of Freedom | Registered in England,
> No. 9361826 | VAT No. GB202190462
> Registered Office: 19 Hilton Road, Canvey Island, Essex SS8 9QA, UK |
> Web: http://minifree.org/
> iQEcBAEBCAAGBQJX9LprAAoJEP9Ft0z50c+UUe4IAJmZ9w5nAQAuQRBF07+vSftI
> 9aW/gz/u0VRbbTbgJJAQYABtgmIOsGUWmV3eg2DvLYz/M5k7uGow4jNkr94e4gtc
> xxrjbsD1Vp5WlDF9MEBI9BFsWmY4ZGGT3wdG7/ioDzek1wKqVfRDgZsjnx5XBJz9
> IhG+BPme1+Z2CFTlml633HVEpT8dPqa2M8TqNozbZpCPLHnaaGAAoN41zgrlRTNN
> coeQyQwvVZvvvz1M6a3A1EGRtwwcaWW5bRQRR9HaESq9YkVkSMrPZh6LgElpp8ei
> l7FWuT6UG1Uq3N1zXfRCusSginE8UazKr0Q+V0XduNquVs3hqvA4Qv+7YfDVk18=
> =9M7A
> -END PGP SIGNATURE-
>



-- 
Aaron Mason - Programmer, open source addict
I've taken my software vows - for beta or for worse

SPARC64: input/output error on softraid 5 with more than 8 disks

[alexmcwhirter]

I'm forwarding this to a few other lists just to see if i can get some 
more input on it. I would like to think this is an arch specific bug as 
i imagine there are people using raid 5 with more than 8 disks on amd64 
/ i386. However i haven't tried this on amd64, so i am not certain.


1. Steps to reproduce (can be done in bsd.rd after issuing MAKEDEV for 
all disks)


Using sd2-sd11 for this example.

disklabel -E sd2(sd3,sd4,etc..) - Create partition "a" with all disk 
space as RAID type all disks are the same model.

bioctl -c 5 -l sd2a,sd3a,etc... softraid0 - comes out as sd12
disklabel -E sd12 - Create partition "a" with all disk space as BSD type
newfs sd12a
mount /dev/sd12a /mnt
cd /mnt
dd if=/dev/zero of=test bs=1m count=128

2. Output Received

dd will fail immediately with "Input/Output Error" with 0 records 
written. dmesg shows no errors. The test file is visible on the 
filesystem however.


3. Dmesg output, will post tomorrow. As stated below, this happens on 
two different sparc64 hosts with completely different hardware, disks, 
HBA's, and disk shelves. There are no dmesg errors added when dd fails.


4. Third party software, can be done in a base install or in bsd.rd. No 
third party software needed.


5. No kernel panic, system keep chugging along, filesystem stays 
mounted, fsck comes back clean, upon reboot raid is still in good 
standing and filesystem checks still pass.


 Original Message 
Subject: Input/Output error on softraid 5 with more that 8 disks
Date: 2016-10-05 19:19
From: alexmcwhir...@triadic.us
To: sp...@openbsd.org

Tested on a Sun E6K and Sun V210. It seems that if i create a raid 5 
array bigger than 8 disks (U320 SCSI) using softraid, i get a rather 
vague "Input / Output" error why attempting to write any significantly 
large data (dd if=/dev/zero of=/mnt/blah bs=1m count=128). newfs creates 
the filesystem fine and fsck comes back clean but trying to put anything 
on the array doesn't seem to do anything.


I was just wondering if this is something that has been reported in the 
past or if I'm the first to uncover it? I have obtained the same results 
from various hardware which seems to point to the softraid driver.


Hardware Tested

HBA's

qlw: isp1000, isp1040, isp10160
esp: fas336

Disk Shelfs

StorEdge D1000
StorEdge 3320

APU2 Internal USB Headers Not Working

[Sam Vaughan]

I'm unable to see USB devices connected to the internal USB headers on
a PC Engines APU2c2 board.  The same devices work as expected when
connected to the external USB ports.

I have a TinyCore Linux USB stick handy that I used to update the
board's firmware.  If I boot it and run `lsusb` then I can see my
devices using all of the internal and external USB headers/ports.

So it would seem that the internal headers are wired up and working,
but for some reason I can't use them via ehci.  Note that that the
external USB ports are USB3, so devices appear via xhci.  It could
therefore be a wider issue with ehci on this board.

I've included as much detail as I know how to find below.  If anyone
has any ideas for things I can try to make this work I'd be very
pleased to hear them.

Sam


Host: PC Engines APU2c2 running OpenBSD 6.0, then TinyCore Linux 6.4.
Test device: FTDI Basic USB to Serial board.

- - - - OpenBSD dmesg - - - -

$ dmesg | egrep -i 'hci|hub|usb'
xhci0 at pci0 dev 16 function 0 "AMD Bolton xHCI" rev 0x11: msi
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 "AMD xHCI root hub" rev 3.00/1.00 addr 1
ahci0 at pci0 dev 17 function 0 "AMD Hudson-2 SATA" rev 0x40: apic 4 int 19,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
scsibus1 at ahci0: 32 targets
ehci0 at pci0 dev 19 function 0 "AMD Hudson-2 USB2" rev 0x39: apic 4 int 18
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 "AMD EHCI root hub" rev 2.00/1.00 addr 1
uhub2 at uhub1 port 1 "Advanced Micro Devices product 0x7900" rev 2.00/0.18
addr 2

- - - - OpenBSD, FTDI on internal J11 USB header (EHCI) - - - -

$ usbdevs -vd
Controller /dev/usb0:
addr 1: super speed, self powered, config 1, xHCI root hub(0x),
AMD(0x1022), rev 1.00
  uhub0
 port 1 disabled
 port 2 disabled
 port 3 disabled
 port 4 disabled
Controller /dev/usb1:
addr 1: high speed, self powered, config 1, EHCI root hub(0x),
AMD(0x1022), rev 1.00
  uhub1
 port 1 addr 2: high speed, self powered, config 1, product 0x7900(0x7900),
Advanced Micro Devices(0x0438), rev 0.18
   uhub2
  port 1 powered
  port 2 powered
  port 3 powered
  port 4 powered
 port 2 powered

- - - - OpenBSD, FTDI on external USB port (xHCI) - - - -

$ usbdevs -vd
Controller /dev/usb0:
addr 1: super speed, self powered, config 1, xHCI root hub(0x),
AMD(0x1022), rev 1.00
  uhub0
 port 1 disabled
 port 2 disabled
 port 3 addr 2: full speed, power 90 mA, config 1, FT232R USB UART(0x6001),
FTDI(0x0403), rev 6.00, iSerialNumber A8004w2k
   uftdi0
 port 4 disabled
Controller /dev/usb1:
addr 1: high speed, self powered, config 1, EHCI root hub(0x),
AMD(0x1022), rev 1.00
  uhub1
 port 1 addr 2: high speed, self powered, config 1, product 0x7900(0x7900),
Advanced Micro Devices(0x0438), rev 0.18
   uhub2
  port 1 powered
  port 2 powered
  port 3 powered
  port 4 powered
 port 2 powered

- - - - OpenBSD pcidumps - - - -

$ doas pcidump -v 0:16:0
 0:16:0: AMD Bolton xHCI
0x: Vendor ID: 1022 Product ID: 7814
0x0004: Command: 0006 Status: 0010
0x0008: Class: 0c Subclass: 03 Interface: 30 Revision: 11
0x000c: BIST: 00 Header Type: 00 Latency Timer: 00 Cache Line Size:
10
0x0010: BAR mem 64bit addr: 0xfeb22000/0x2000
0x0018: BAR empty ()
0x001c: BAR empty ()
0x0020: BAR empty ()
0x0024: BAR empty ()
0x0028: Cardbus CIS: 
0x002c: Subsystem Vendor ID: 1022 Product ID: 1410
0x0030: Expansion ROM Base Address: 
0x0038: 
0x003c: Interrupt Pin: 01 Line: 00 Min Gnt: 00 Max Lat: 00
0x0050: Capability 0x01: Power Management
State: D0
0x0070: Capability 0x05: Message Signaled Interrupts (MSI)
0x0090: Capability 0x11: Extended Message Signaled Interrupts (MSI-X)
0x00a0: Capability 0x10: PCI Express

$ doas pcidump -v 0:19:0
 0:19:0: AMD Hudson-2 USB2
0x: Vendor ID: 1022 Product ID: 7808
0x0004: Command: 0006 Status: 02b0
0x0008: Class: 0c Subclass: 03 Interface: 20 Revision: 39
0x000c: BIST: 00 Header Type: 00 Latency Timer: 40 Cache Line Size:
10
0x0010: BAR mem 32bit addr: 0xfeb25400/0x0100
0x0014: BAR empty ()
0x0018: BAR empty ()
0x001c: BAR empty ()
0x0020: BAR empty ()
0x0024: BAR empty ()
0x0028: Cardbus CIS: 
0x002c: Subsystem Vendor ID: 1022 Product ID: 7808
0x0030: Expansion ROM Base Address: 
0x0038: 
0x003c: Interrupt Pin: 01 Line: 00 Min Gnt: 00 Max Lat: 00
0x00c0: Capability 0x01: Power Management
State: D0
0x00e4: Capability 0x0a: Debug Port

- - - - Linux dmesg - - - -

# dmesg | egrep -i 'hci|hub|usb' | cut -c16-
pci :00:11.0: set SATA to AHCI mode
ACPI: bus type USB registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: 

Re: Setting flags on /dev/audio

[Tobias Brodel]

Somehow sent the inverse of the intended diff, all apologies.

Please find the proper diff attached.

toby/

On 10/06/16 11:07, Tobias Brodel wrote:
> Hi,
>
> I've been working on porting an audio application which uses the OSS
> compatibility layer. It was throwing warnings at runtime about not
> being able to set flags on /dev/audio.
>
> After searching around I found a diff written by Steven McDonald in
> the misc@ archives:
>
> http://openbsd-archive.7691.n7.nabble.com/perl-fctnl-woes-td276032.html
>
> Apparently the issue lies with the removal of the FIOASYNC ioctl(2).
>
> I've applied the diff on -CURRENT and it has solved my issues - please
> find the diff and a test program attached.
>
> toby/

[demime 1.01d removed an attachment of type text/x-patch which had a name of 
sys_dev_audio_c.diff]

Setting flags on /dev/audio

[Tobias Brodel]

Hi,

I've been working on porting an audio application which uses the OSS
compatibility layer. It was throwing warnings at runtime about not
being able to set flags on /dev/audio.

After searching around I found a diff written by Steven McDonald in
the misc@ archives:

http://openbsd-archive.7691.n7.nabble.com/perl-fctnl-woes-td276032.html

Apparently the issue lies with the removal of the FIOASYNC ioctl(2).

I've applied the diff on -CURRENT and it has solved my issues - please
find the diff and a test program attached.

toby/

[demime 1.01d removed an attachment of type text/x-patch which had a name of 
sys_dev_audio_c.diff]

[demime 1.01d removed an attachment of type text/x-csrc which had a name of 
audio-test.c]

Re: ral(4) problems on current/i396 ALIX

[Stefan Sperling]

On Tue, Sep 20, 2016 at 11:53:41AM +0200, Stefan Sperling wrote:
> On Tue, Sep 20, 2016 at 10:46:54AM +0200, Jan Stary wrote:
> > This is ALIX 2C1, just upgraded to current/i386 (dmesg below).
> > It serves as a wifi AP using ral(4). The console gets spammed with
> > 
> > ral0: sending data frame failed 0x02faaafa
> > 
> > This used to work fine since 5.9/i386.
> 
> This is the only known fallout from ieee80211_proto.c r1.68 and seems to
> affect ral 2560 chips only. Apparently this driver has always had a bug
> with RTS/CTS and the wireless stack is now triggering it.

This seems to be a transmission failure mode which Linux rt2x00 simply
ignores. We don't have a name for it, and neither has linux, and
I don't have docs, so I don't know what the reason for the failure
mode could be.

It's curious that this started showing up when we enabled RTS/CTS
for large frames, but RTS/CTS seems to be working fine and was
already being enabled in other cases.
For now, I'd just disable this message and only show it if the
driver is compiled in debug mode.

The only problem with this driver/device I've observed is that transmit
rates are fairly slow. It settles at 1Mbit/s and never recovers, which
means the ral(4) device is retrying frames excessively.
But I don't believe that's a new problem. I remember using ral(4) as my
AP years ago and speed always sucked even back then. There might be other
bugs to fix in here but this message by itself won't help us find them.

Index: rt2560.c
===
RCS file: /cvs/src/sys/dev/ic/rt2560.c,v
retrieving revision 1.80
diff -u -p -r1.80 rt2560.c
--- rt2560.c13 Apr 2016 10:49:26 -  1.80
+++ rt2560.c5 Oct 2016 22:53:37 -
@@ -959,9 +959,10 @@ rt2560_tx_intr(struct rt2560_softc *sc)
case RT2560_TX_FAIL_INVALID:
case RT2560_TX_FAIL_OTHER:
default:
-   printf("%s: sending data frame failed 0x%08x\n",
-   sc->sc_dev.dv_xname, letoh32(desc->flags));
+   DPRINTF(("%s: sending data frame failed 0x%08x\n",
+   sc->sc_dev.dv_xname, letoh32(desc->flags)));
ifp->if_oerrors++;
+   break;
}
 
/* descriptor is no longer valid */
@@ -1031,8 +1032,9 @@ rt2560_prio_intr(struct rt2560_softc *sc
case RT2560_TX_FAIL_INVALID:
case RT2560_TX_FAIL_OTHER:
default:
-   printf("%s: sending mgt frame failed 0x%08x\n",
-   sc->sc_dev.dv_xname, letoh32(desc->flags));
+   DPRINTF(("%s: sending mgt frame failed 0x%08x\n",
+   sc->sc_dev.dv_xname, letoh32(desc->flags)));
+   break;
}
 
/* descriptor is no longer valid */
@@ -1945,6 +1947,8 @@ rt2560_start(struct ifnet *ifp)
break;
 
} else {
+   /* Because RTS/CTS requires an extra frame we need
+* space for 2 frames on the regular Tx queue. */
if (sc->txq.queued >= RT2560_TX_RING_COUNT - 1) {
ifq_set_oactive(>if_snd);
sc->sc_flags |= RT2560_DATA_OACTIVE;

Re: Large datasize - how to limit physical memory?

[Ted Unangst]

If somebody writes a C program that demonstrates the problem, I'm happy to
take a look. I'm not installing erlang.

Re: some more info on pkg_add/fw_update changes

[Marc Espie]

On Tue, Oct 04, 2016 at 03:15:18PM +0200, Marc Espie wrote:
> - the new scheme is slightly more unflexible with respect to unsigned
> data: by default, every .tgz is piped thru signify -Zs, so 
> pkg_add/pkg_info/fw_update WON'T even see any data if it's not signed. 
> Error reporting is inadequate, to say the least. I'm working on fixing
> that, but there is some code I do not like, so there is a great deal of
> rewrite.

There was actually a trivial bug in some code, so that you should now get
error reports that make sense, though slightly verbose. At the very least,
you will now get 'unsigned packages' messages on localhost.
I've also trimmed ftp/signify interactions a bit.



> - I'm working on ways to mix unsigned and signed packages in a sane way.
> I've added TRUSTED_PKG_PATH, and I will have a mechanism that says
> some sources are safe (/usr/ports/packages/%a/all, for instance).
> This is unnecessary if you only install official binary stuff, but it is
> necessary for development or for people who really want to tinker with
> their machines.

/usr/ports/packages/%a/all is surprisingly difficult to normalize every
time (File::Spec->abs2rel tends to yield the "wrong" location for me thx
to /usr/ports being a symlink), so this is likely not to happen.

Re: Multiple web servers behind NAT

[trondd]

On Wed, October 5, 2016 8:43 am, Radek wrote:
> Yes, my servers share the same ext IP.
> It is 5.9. I am trying to configure relayd. I commented out previous
> "rdr-to" rules from /etc/pf.conf and added as below.
> 10.0.30.101, 10.0.30.201 - it is not a mistake - ( 10.0.8.11, 10.0.8.22
> was just an exemplary IP)
> All websites are unreachable now.
>
> #grep relayd /etc/pf.conf
> anchor "relayd/*"
>
> #relayd -n
> configuration OK
>
> #cat /etc/relayd.conf
> ext_addr="msk0"
> host1="10.0.30.101"
> host2="10.0.30.201"
>
> table  { $host1 }
> table  { $host2 }
>
> http protocol "web_one" {
>return error
>pass
>match request header "Host" value "1.domain.com" forward to 

I think you need "pass request header..."

> }
>
> http protocol "web_two" {
>return error
>pass
>match request header "Host" value "4.domain.com" forward to 
> }

You should combine the two protocols into one.  You can have multiple pass
lines.  Last match wins, unless you use "quick".  You can define a default
that way.

>
> relay relay_one {
>listen on $ext_addr port 80
>protocol "web_one"
>forward to  check tcp port 80
> }
>
> relay relay_two {
>listen on $ext_addr port 80
>protocol "web_two"
>forward to  check tcp port 80
> }

You should have only one relay defined, you can't have two things
listening on the same port.  Just put the two "forward to" lines in the
same relay block.


>
> #/etc/rc.d/relayd -df restart
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing rc_check
> relayd
> doing rc_stop
> doing _rc_wait stop
> doing rc_check
> doing rc_check
> doing _rc_rm_runfile
> (ok)
> doing _rc_parse_conf
> doing _rc_quirks
> relayd_flags empty, using default ><
> doing _rc_read_runfile
> doing rc_check
> relayd
> doing rc_pre
> configuration OK
> doing rc_start
> doing _rc_wait start
> doing rc_check
> doing _rc_write_runfile
> (ok)
>

relayctl is your friend here.  See if the relays are actually up:
'relayctl show relays' and 'relayctl show summary'

Re: Large datasize - how to limit physical memory?

[Raimo Niskanen]

On Wed, Oct 05, 2016 at 03:36:20PM +0200, Otto Moerbeek wrote:
> On Wed, Oct 05, 2016 at 03:28:06PM +0200, Raimo Niskanen wrote:
> 
> > On Mon, Oct 03, 2016 at 04:13:58PM +0200, Otto Moerbeek wrote:
> > > On Mon, Oct 03, 2016 at 02:56:05PM +0200, Raimo Niskanen wrote:
> > > 
> > > > On Fri, Sep 30, 2016 at 01:02:10PM +0200, Otto Moerbeek wrote:
> > > > > On Fri, Sep 30, 2016 at 09:10:21AM +0200, Raimo Niskanen wrote:
> > > > > 
> > > > > > On Wed, Sep 28, 2016 at 09:19:51AM +0200, Raimo Niskanen wrote:
> > > > > > > Dear misc@
> > > > > > > 
> > > > > > > I have searched the archives and read the documentation of 
> > > > > > > login.conf(5),
> > > > > > > ksh(1):ulimit and can not find how to limit the amount of 
> > > > > > > physical memory a
> > > > > > > process may use.
> > > > > > > 
> > > > > > > I have the following limits where I have set down ulimit -m and 
> > > > > > > ulimit -l
> > > > > > > to 1 kbytes in an attempt to limit the process I spawn which 
> > > > > > > is
> > > > > > > the Erlang VM.
> > > > > > > 
> > > > > > > $ ulimit -a
> > > > > > > time(cpu-seconds)unlimited
> > > > > > > file(blocks) unlimited
> > > > > > > coredump(blocks) unlimited
> > > > > > > data(kbytes) 33554432
> > > > > > > stack(kbytes)8192
> > > > > > > lockedmem(kbytes)1
> > > > > > > memory(kbytes)   1
> > > > > > > nofiles(descriptors) 1024
> > > > > > > processes1024
> > > > > > > 
> > > > > > > Note that the machine has got 8 GB of physical memory and 8 GB of 
> > > > > > > swap and
> > > > > > > that I have set datasize=infinity in /etc/login.conf. I got
> > > > > > > datasize=33554432 which seems to be the same as 
> > > > > > > kern.shminfo.shmmax.
> > > > > > > The datasize is twice the physical memory + swap.
> > > > > > > 
> > > > > > > Then I start the Erlang VM and tell it to allocate an address 
> > > > > > > block of 3
> > > > > > > MByte for future use where it will store all literal data in the 
> > > > > > > same block
> > > > > > > (this is a garbage collector optimization).  Not much of this 
> > > > > > > data is
> > > > > > > actually used.
> > > > > > > 
> > > > > > >  68196 beam CALL  
> > > > > > > mmap(0,0x75300,0,0x1002,-1,0)
> > > > > > >  68196 beam RET   mmap 11871265173504/0xacbfe8b3000
> > > > > > > 
> > > > > > > Note the protection flags on the block.  No access is allowed.  
> > > > > > > This trick
> > > > > > > works just fine; here is what top says:
> > > > > > > 
> > > > > > > load averages:  0.15,  0.13,  0.09 frerin.otp.ericsson.se 
> > > > > > > 08:49:46
> > > > > > > 48 processes: 47 idle, 1 on processor 
> > > > > > > up 13:49
> > > > > > > CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% 
> > > > > > > interrupt,  100% idle
> > > > > > > CPU1 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% 
> > > > > > > interrupt,  100% idle
> > > > > > > Memory: Real: 43M/636M act/tot Free: 7028M Cache: 508M Swap: 
> > > > > > > 0K/8155M
> > > > > > > 
> > > > > > >   PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIME
> > > > > > > CPU COMMAND
> > > > > > > 68196 raimo  20   29G   15M sleep poll  0:00  
> > > > > > > 1.42% beam
> > > > > > > 
> > > > > > > So I have a process with a data size of 29 GB on a machine with 
> > > > > > > 16 GB
> > > > > > > memory + swap.  I have also tried to start an additional Erlang 
> > > > > > > VM that
> > > > > > > also allocates 29 GB of virtual memory which also works.
> > > > > > > 
> > > > > > > That this is allowed is just fine for me - this trick of 
> > > > > > > allocating a
> > > > > > > "large enough" PROT_NONE memory to get one address range for some 
> > > > > > > special
> > > > > > > data type is very useful for the Erlang VM.  But I wonder how to 
> > > > > > > limit the
> > > > > > > actual memory use?  Setting down ulimit -m and ulimit -l to 1 
> > > > > > > kbytes
> > > > > > > did not prevent this process from getting 15 MByte of "RES" 
> > > > > > > memory...
> > > > > > > 
> > > > > > > Is there some way to limit the actual amount of memory for a 
> > > > > > > process when I
> > > > > > > need to set up the datasize to allow for large unused virtual 
> > > > > > > memory
> > > > > > > blocks?
> > > > > > 
> > > > > > I have found clues in getrlimit,setrlimit(2):
> > > > > > 
> > > > > >  RLIMIT_DATA The maximum size (in bytes) of the data 
> > > > > > segment for a
> > > > > >  process; this includes memory allocated via 
> > > > > > malloc(3)
> > > > > >  and all other anonymous memory mapped via 
> > > > > > mmap(2).
> > > > > > :
> > > > > >  RLIMIT_RSS  The maximum size (in bytes) to which a 
> > > > > > process's
> > > > > >  resident set size may grow.  This imposes a 
> > > > > > limit
> > > > > >  on the amount of physical memory to be given 

Re: Large datasize - how to limit physical memory?

[Otto Moerbeek]

On Wed, Oct 05, 2016 at 03:28:06PM +0200, Raimo Niskanen wrote:

> On Mon, Oct 03, 2016 at 04:13:58PM +0200, Otto Moerbeek wrote:
> > On Mon, Oct 03, 2016 at 02:56:05PM +0200, Raimo Niskanen wrote:
> > 
> > > On Fri, Sep 30, 2016 at 01:02:10PM +0200, Otto Moerbeek wrote:
> > > > On Fri, Sep 30, 2016 at 09:10:21AM +0200, Raimo Niskanen wrote:
> > > > 
> > > > > On Wed, Sep 28, 2016 at 09:19:51AM +0200, Raimo Niskanen wrote:
> > > > > > Dear misc@
> > > > > > 
> > > > > > I have searched the archives and read the documentation of 
> > > > > > login.conf(5),
> > > > > > ksh(1):ulimit and can not find how to limit the amount of physical 
> > > > > > memory a
> > > > > > process may use.
> > > > > > 
> > > > > > I have the following limits where I have set down ulimit -m and 
> > > > > > ulimit -l
> > > > > > to 1 kbytes in an attempt to limit the process I spawn which is
> > > > > > the Erlang VM.
> > > > > > 
> > > > > > $ ulimit -a
> > > > > > time(cpu-seconds)unlimited
> > > > > > file(blocks) unlimited
> > > > > > coredump(blocks) unlimited
> > > > > > data(kbytes) 33554432
> > > > > > stack(kbytes)8192
> > > > > > lockedmem(kbytes)1
> > > > > > memory(kbytes)   1
> > > > > > nofiles(descriptors) 1024
> > > > > > processes1024
> > > > > > 
> > > > > > Note that the machine has got 8 GB of physical memory and 8 GB of 
> > > > > > swap and
> > > > > > that I have set datasize=infinity in /etc/login.conf. I got
> > > > > > datasize=33554432 which seems to be the same as kern.shminfo.shmmax.
> > > > > > The datasize is twice the physical memory + swap.
> > > > > > 
> > > > > > Then I start the Erlang VM and tell it to allocate an address block 
> > > > > > of 3
> > > > > > MByte for future use where it will store all literal data in the 
> > > > > > same block
> > > > > > (this is a garbage collector optimization).  Not much of this data 
> > > > > > is
> > > > > > actually used.
> > > > > > 
> > > > > >  68196 beam CALL  
> > > > > > mmap(0,0x75300,0,0x1002,-1,0)
> > > > > >  68196 beam RET   mmap 11871265173504/0xacbfe8b3000
> > > > > > 
> > > > > > Note the protection flags on the block.  No access is allowed.  
> > > > > > This trick
> > > > > > works just fine; here is what top says:
> > > > > > 
> > > > > > load averages:  0.15,  0.13,  0.09 frerin.otp.ericsson.se 
> > > > > > 08:49:46
> > > > > > 48 processes: 47 idle, 1 on processor 
> > > > > > up 13:49
> > > > > > CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 
> > > > > >  100% idle
> > > > > > CPU1 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt, 
> > > > > >  100% idle
> > > > > > Memory: Real: 43M/636M act/tot Free: 7028M Cache: 508M Swap: 
> > > > > > 0K/8155M
> > > > > > 
> > > > > >   PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU 
> > > > > > COMMAND
> > > > > > 68196 raimo  20   29G   15M sleep poll  0:00  1.42% 
> > > > > > beam
> > > > > > 
> > > > > > So I have a process with a data size of 29 GB on a machine with 16 
> > > > > > GB
> > > > > > memory + swap.  I have also tried to start an additional Erlang VM 
> > > > > > that
> > > > > > also allocates 29 GB of virtual memory which also works.
> > > > > > 
> > > > > > That this is allowed is just fine for me - this trick of allocating 
> > > > > > a
> > > > > > "large enough" PROT_NONE memory to get one address range for some 
> > > > > > special
> > > > > > data type is very useful for the Erlang VM.  But I wonder how to 
> > > > > > limit the
> > > > > > actual memory use?  Setting down ulimit -m and ulimit -l to 1 
> > > > > > kbytes
> > > > > > did not prevent this process from getting 15 MByte of "RES" 
> > > > > > memory...
> > > > > > 
> > > > > > Is there some way to limit the actual amount of memory for a 
> > > > > > process when I
> > > > > > need to set up the datasize to allow for large unused virtual memory
> > > > > > blocks?
> > > > > 
> > > > > I have found clues in getrlimit,setrlimit(2):
> > > > > 
> > > > >  RLIMIT_DATA The maximum size (in bytes) of the data segment 
> > > > > for a
> > > > >  process; this includes memory allocated via 
> > > > > malloc(3)
> > > > >  and all other anonymous memory mapped via 
> > > > > mmap(2).
> > > > > :
> > > > >  RLIMIT_RSS  The maximum size (in bytes) to which a process's
> > > > >  resident set size may grow.  This imposes a limit
> > > > >  on the amount of physical memory to be given to a
> > > > >  process; if memory is tight, the system will 
> > > > > prefer
> > > > >  to take memory from processes that are exceeding
> > > > >  their declared resident set size.
> > > > > 
> > > > > Now I try to figure out the implications of this...  If I set the 
> > > > > 

Re: Large datasize - how to limit physical memory?

[Raimo Niskanen]

On Mon, Oct 03, 2016 at 04:13:58PM +0200, Otto Moerbeek wrote:
> On Mon, Oct 03, 2016 at 02:56:05PM +0200, Raimo Niskanen wrote:
> 
> > On Fri, Sep 30, 2016 at 01:02:10PM +0200, Otto Moerbeek wrote:
> > > On Fri, Sep 30, 2016 at 09:10:21AM +0200, Raimo Niskanen wrote:
> > > 
> > > > On Wed, Sep 28, 2016 at 09:19:51AM +0200, Raimo Niskanen wrote:
> > > > > Dear misc@
> > > > > 
> > > > > I have searched the archives and read the documentation of 
> > > > > login.conf(5),
> > > > > ksh(1):ulimit and can not find how to limit the amount of physical 
> > > > > memory a
> > > > > process may use.
> > > > > 
> > > > > I have the following limits where I have set down ulimit -m and 
> > > > > ulimit -l
> > > > > to 1 kbytes in an attempt to limit the process I spawn which is
> > > > > the Erlang VM.
> > > > > 
> > > > > $ ulimit -a
> > > > > time(cpu-seconds)unlimited
> > > > > file(blocks) unlimited
> > > > > coredump(blocks) unlimited
> > > > > data(kbytes) 33554432
> > > > > stack(kbytes)8192
> > > > > lockedmem(kbytes)1
> > > > > memory(kbytes)   1
> > > > > nofiles(descriptors) 1024
> > > > > processes1024
> > > > > 
> > > > > Note that the machine has got 8 GB of physical memory and 8 GB of 
> > > > > swap and
> > > > > that I have set datasize=infinity in /etc/login.conf. I got
> > > > > datasize=33554432 which seems to be the same as kern.shminfo.shmmax.
> > > > > The datasize is twice the physical memory + swap.
> > > > > 
> > > > > Then I start the Erlang VM and tell it to allocate an address block 
> > > > > of 3
> > > > > MByte for future use where it will store all literal data in the same 
> > > > > block
> > > > > (this is a garbage collector optimization).  Not much of this data is
> > > > > actually used.
> > > > > 
> > > > >  68196 beam CALL  
> > > > > mmap(0,0x75300,0,0x1002,-1,0)
> > > > >  68196 beam RET   mmap 11871265173504/0xacbfe8b3000
> > > > > 
> > > > > Note the protection flags on the block.  No access is allowed.  This 
> > > > > trick
> > > > > works just fine; here is what top says:
> > > > > 
> > > > > load averages:  0.15,  0.13,  0.09 frerin.otp.ericsson.se 
> > > > > 08:49:46
> > > > > 48 processes: 47 idle, 1 on processor up 
> > > > > 13:49
> > > > > CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  
> > > > > 100% idle
> > > > > CPU1 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  
> > > > > 100% idle
> > > > > Memory: Real: 43M/636M act/tot Free: 7028M Cache: 508M Swap: 0K/8155M
> > > > > 
> > > > >   PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU 
> > > > > COMMAND
> > > > > 68196 raimo  20   29G   15M sleep poll  0:00  1.42% 
> > > > > beam
> > > > > 
> > > > > So I have a process with a data size of 29 GB on a machine with 16 GB
> > > > > memory + swap.  I have also tried to start an additional Erlang VM 
> > > > > that
> > > > > also allocates 29 GB of virtual memory which also works.
> > > > > 
> > > > > That this is allowed is just fine for me - this trick of allocating a
> > > > > "large enough" PROT_NONE memory to get one address range for some 
> > > > > special
> > > > > data type is very useful for the Erlang VM.  But I wonder how to 
> > > > > limit the
> > > > > actual memory use?  Setting down ulimit -m and ulimit -l to 1 
> > > > > kbytes
> > > > > did not prevent this process from getting 15 MByte of "RES" memory...
> > > > > 
> > > > > Is there some way to limit the actual amount of memory for a process 
> > > > > when I
> > > > > need to set up the datasize to allow for large unused virtual memory
> > > > > blocks?
> > > > 
> > > > I have found clues in getrlimit,setrlimit(2):
> > > > 
> > > >  RLIMIT_DATA The maximum size (in bytes) of the data segment 
> > > > for a
> > > >  process; this includes memory allocated via 
> > > > malloc(3)
> > > >  and all other anonymous memory mapped via mmap(2).
> > > > :
> > > >  RLIMIT_RSS  The maximum size (in bytes) to which a process's
> > > >  resident set size may grow.  This imposes a limit
> > > >  on the amount of physical memory to be given to a
> > > >  process; if memory is tight, the system will prefer
> > > >  to take memory from processes that are exceeding
> > > >  their declared resident set size.
> > > > 
> > > > Now I try to figure out the implications of this...  If I set the data 
> > > > size
> > > > so the sum of the data sizes for all processes in the system is larger 
> > > > than
> > > > physical memory + swap, then any process may allocate the last block of
> > > > memory in the system so a more important process later will fail to
> > > > allocate?
> > > 
> > > yes.
> > > 
> > > > 
> > > > And the memoryuse limit is rather 

Re: Multiple web servers behind NAT

[Radek]

Yes, my servers share the same ext IP. 
It is 5.9. I am trying to configure relayd. I commented out previous "rdr-to" 
rules from /etc/pf.conf and added as below.
10.0.30.101, 10.0.30.201 - it is not a mistake - ( 10.0.8.11, 10.0.8.22 was 
just an exemplary IP)
All websites are unreachable now.

#grep relayd /etc/pf.conf
anchor "relayd/*"

#relayd -n
configuration OK

#cat /etc/relayd.conf
ext_addr="msk0" 
host1="10.0.30.101" 
host2="10.0.30.201" 

table  { $host1 } 
table  { $host2 } 

http protocol "web_one" { 
   return error
   pass
   match request header "Host" value "1.domain.com" forward to  
} 

http protocol "web_two" { 
   return error
   pass
   match request header "Host" value "4.domain.com" forward to  
} 

relay relay_one { 
   listen on $ext_addr port 80 
   protocol "web_one" 
   forward to  check tcp port 80 
} 

relay relay_two { 
   listen on $ext_addr port 80 
   protocol "web_two" 
   forward to  check tcp port 80
}

#/etc/rc.d/relayd -df restart
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing rc_check
relayd
doing rc_stop
doing _rc_wait stop
doing rc_check
doing rc_check
doing _rc_rm_runfile
(ok)
doing _rc_parse_conf
doing _rc_quirks
relayd_flags empty, using default ><
doing _rc_read_runfile
doing rc_check
relayd
doing rc_pre
configuration OK
doing rc_start
doing _rc_wait start
doing rc_check
doing _rc_write_runfile
(ok)


On Fri, 30 Sep 2016 07:26:22 -0400
Josh Grosse  wrote:

> On Fri, Sep 30, 2016 at 11:42:11AM +0200, Radek wrote:
> > Hi, 
> > I have one web_serwer_1 behind OpenBSD 5.9 router/NAT with single IP. 
> > 
> > web_serwer_1 -apache,virtualhosts- (10.0.8.11):
> > 1.domain.com
> > 2.domain.com
> > 3.domain.com
> > 
> > pf.conf:
> > pass in log quick on $ext_if inet proto tcp from any to $ext_if port 80 
> > rdr-to $web_serwer_1 port 80 set prio (1, 6) keep state
> > pass in log quick on $ext_if inet proto tcp from any to $ext_if port 443 
> > rdr-to $web_serwer_1 port 443 set prio (1, 6) keep state
> > 
> > Everything works fine.
> > 
> > Now, I need to add another web_serwer_2. It would be the "main" web server.
> >  
> > web_serwer_2 - native httpd,virtualhosts- (10.0.8.22):
> > 4.domain.com
> > 5.domain.com
> > 6.domain.com
> > 
> > How can I make it work?
> > Any help appreciated.
> 
> If the two web servers share the same external IP address, use relayd(8),
> as it is designed to inspect HTTP URLs.
> 


-- 
radek

Re: umb is cool

[Jan Klemkow]

+1 from me, too.  The umb(4) drive was great work from gerhard@!

On Tue, Oct 04, 2016 at 09:36:26PM -0700, Scott Bonds wrote:
> I've got a WWAN card that required a bunch of fiddling with pppd under
> 5.9 to get online. I upgraded to 6.0 and my pppd dialup script stopped
> working. I soon discovered I had a new NIC: umb0. A man page read and
> an ifconfig command later, I've got a working WWAN-based connection to
> the internet. Piece of cake.
> 
> I really like being able to connect with a simple ifconfig command
> instead of all that pppd config I had before. If you're in the market
> for a WWAN card for your OpenBSD box and you have a BIOS that doesn't
> get in the way of using the hardware of your choice, you might want to
> consider the list of devices on the umb man page.

Re: Fwd: Booting BSD on a Libreboot system - documentation needed

[Leah Rowe]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Thanks.

On 05/10/16 05:39, Scott Bonds wrote:
> I've started a stab at it. My x200 is in pieces at the moment and I
> want to retest my instructions before I submit a PR, so it may be a
> couple weeks.
> 
> On 10/05, Leah Rowe wrote:
>> 

- -- 
Leah Rowe

Libreboot developer

Use free software. Free as in freedom.
https://en.wikipedia.org/wiki/Free_software

Use a free operating system, GNU/Linux.

Use a free BIOS.
https://libreboot.org/

Support computer user freedom.
https://peers.community/

Minifree Ltd, trading as Ministry of Freedom | Registered in England,
No. 9361826 | VAT No. GB202190462
Registered Office: 19 Hilton Road, Canvey Island, Essex SS8 9QA, UK |
Web: http://minifree.org/
iQEcBAEBCAAGBQJX9LprAAoJEP9Ft0z50c+UUe4IAJmZ9w5nAQAuQRBF07+vSftI
9aW/gz/u0VRbbTbgJJAQYABtgmIOsGUWmV3eg2DvLYz/M5k7uGow4jNkr94e4gtc
xxrjbsD1Vp5WlDF9MEBI9BFsWmY4ZGGT3wdG7/ioDzek1wKqVfRDgZsjnx5XBJz9
IhG+BPme1+Z2CFTlml633HVEpT8dPqa2M8TqNozbZpCPLHnaaGAAoN41zgrlRTNN
coeQyQwvVZvvvz1M6a3A1EGRtwwcaWW5bRQRR9HaESq9YkVkSMrPZh6LgElpp8ei
l7FWuT6UG1Uq3N1zXfRCusSginE8UazKr0Q+V0XduNquVs3hqvA4Qv+7YfDVk18=
=9M7A
-END PGP SIGNATURE-

Re: umb is cool

[Aaron Mason]

Oh yes.  Direct IP is awesome.

On Wed, Oct 5, 2016 at 3:36 PM, Scott Bonds  wrote:
> I've got a WWAN card that required a bunch of fiddling with pppd under 5.9
> to get online. I upgraded to 6.0 and my pppd dialup script stopped working.
> I soon discovered I had a new NIC: umb0. A man page read and an ifconfig
> command later, I've got a working WWAN-based connection to the internet.
> Piece of cake.
>
> I really like being able to connect with a simple ifconfig command instead
> of all that pppd config I had before. If you're in the market for a WWAN
> card for your OpenBSD box and you have a BIOS that doesn't get in the way of
> using the hardware of your choice, you might want to consider the list of
> devices on the umb man page.
>



-- 
Aaron Mason - Programmer, open source addict
I've taken my software vows - for beta or for worse