Re: amd5.9 mp softdep bufcachepercent

[Tinker]

Can you please test the kern.bufcachepercent values 10, 20 and 90 
separately and have each of those individual tests based on 5 tests each 
for the respective setting?


You would need to ensure that the buf cache had equal contents at each 
test iteration, so each test needs to be preceded by a reboot + perhaps 
some "load into bufcache" niceness e.g. find + cat > /dev/null, before 
your actual rm and sync. Also please clarify if the files that you rm 
were created before or after that reboot.


Looking forward to read your results.

On 2016-04-17 11:43, Predrag Punosevac wrote:
I rm -rf /usr/ports on the newly upgraded laptop running generic 5.9 
amd

MP kernel. /usr is mounted with the option softdep. It took ages for rm
to finish. Laptop has 4GB of RAM. The HDD is SSD with the capacity 931
GB and fully encrypted.

Playing little bit with

sysctl bufcacheparcent

and  BUFCACHEPERCENT in

/sys/conf/param.c

I noticed that it is at least somewhat faster using value 10 instead of
default value 20. Has anybody noticed anything strange with rm and du 
-h

-s on the directories with lots of small files. Any suggestions how to
speed things up?

Predrag

amd5.9 mp softdep bufcachepercent

[Predrag Punosevac]

I rm -rf /usr/ports on the newly upgraded laptop running generic 5.9 amd
MP kernel. /usr is mounted with the option softdep. It took ages for rm
to finish. Laptop has 4GB of RAM. The HDD is SSD with the capacity 931
GB and fully encrypted.

Playing little bit with 

sysctl bufcacheparcent

and  BUFCACHEPERCENT in 

/sys/conf/param.c

I noticed that it is at least somewhat faster using value 10 instead of
default value 20. Has anybody noticed anything strange with rm and du -h
-s on the directories with lots of small files. Any suggestions how to
speed things up?

Predrag

Re: amd5.9 mp softdep bufcachepercent

[Tinker]

..And if you buy more RAM someday remember that the buffer cache is 
limited to 32 bits currently i.e. <4GB in total size. I trust that limit 
will be lifted someday.


On 2016-04-17 11:50, Tinker wrote:

Just set it to 90??

(And ensure you set the right thing i.e. kern.bufcachepercent rightly 
spelled.)


On 2016-04-17 11:43, Predrag Punosevac wrote:
I rm -rf /usr/ports on the newly upgraded laptop running generic 5.9 
amd
MP kernel. /usr is mounted with the option softdep. It took ages for 
rm

to finish. Laptop has 4GB of RAM. The HDD is SSD with the capacity 931
GB and fully encrypted.

Playing little bit with

sysctl bufcacheparcent

and  BUFCACHEPERCENT in

/sys/conf/param.c

I noticed that it is at least somewhat faster using value 10 instead 
of
default value 20. Has anybody noticed anything strange with rm and du 
-h

-s on the directories with lots of small files. Any suggestions how to
speed things up?

Predrag

Re: amd5.9 mp softdep bufcachepercent

[Tinker]

Just set it to 90??

(And ensure you set the right thing i.e. kern.bufcachepercent rightly 
spelled.)


On 2016-04-17 11:43, Predrag Punosevac wrote:
I rm -rf /usr/ports on the newly upgraded laptop running generic 5.9 
amd

MP kernel. /usr is mounted with the option softdep. It took ages for rm
to finish. Laptop has 4GB of RAM. The HDD is SSD with the capacity 931
GB and fully encrypted.

Playing little bit with

sysctl bufcacheparcent

and  BUFCACHEPERCENT in

/sys/conf/param.c

I noticed that it is at least somewhat faster using value 10 instead of
default value 20. Has anybody noticed anything strange with rm and du 
-h

-s on the directories with lots of small files. Any suggestions how to
speed things up?

Predrag

Re: Upgrade to 5.9 full disk encryption

[Predrag Punosevac]

Bryan Everly wrote:
> 
> Boot the installer. Exit to the shell. Then do:
> 
> bioctl -c C -l /dev/sd0a softraid0
>

Unless I did something really stupid I would swear that I upgraded fully
encrypted laptop running 5.8 to 5.9 easier. 

I downloaded bsd.rd for 5.9 and put into /. Then I rebooted the laptop.
When prompted for boot password and entered it. Then I booted from
bsd.rd and chose the upgrade option. When upgrade manager asked me what
is the installation disk I pointed it to the crypto disk. In my case
physical device is

/dev/sd0

and crypto device is /dev/sd1

No softraid passwords were needed. After upgrade was finished I booted
into 5.9 and did usual sysmerge, cleaning files and upgrading packages.

Best,
Predrag

Re: Odd (incorrect?) zsh output

[Christian Weisgerber]

On 2016-04-16, Geoff Wozniak  wrote:

> How can I go about determining why this is happening?  That is, 
> what are some good techniques on OpenBSD that let me debug this? 
> And are there any clues that anyone can recommend I keep an eye 
> out for?

My first guess would be different shell initialization files.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

Re: Odd (incorrect?) zsh output

[Ted Unangst]

Geoff Wozniak wrote:
> 
>   # zsh -c 'x=$(false); echo $?'
>   0
> 
> This seemed odd to me; I expect the result to be '1'.  In fact, 
> '1' is what both ksh and bash produce on the same system.  On 
> other systems I have access to (Linux variants, FreeBSD, other 
> OpenBSD installs), ksh, bash, and zsh also produce '1'.

You can try just running

$ false
$ echo $?

Maybe ktrace of false. Maybe another command that fails, like ls xx.

Odd (incorrect?) zsh output

[Geoff Wozniak]

Immediately after a fresh install of 5.9 on my Thinkpad X200, I 
installed zsh5.2p2 using "pkg_add zsh" (via 
openbsd.cs.toronto.edu).  I then ran a test.


 # zsh -c 'x=$(false); echo $?'
 0

This seemed odd to me; I expect the result to be '1'.  In fact, 
'1' is what both ksh and bash produce on the same system.  On 
other systems I have access to (Linux variants, FreeBSD, other 
OpenBSD installs), ksh, bash, and zsh also produce '1'.


As another test, I installed 5.9 (fresh) on another machine and 
ran the test:


 # zsh -c 'x=$(false); echo $?'
 1

I've done this a few times and every time it's the same: on my 
Thinkpad X200 the test produces '0' whereas any other machine I 
install 5.9 on it produces '1'.


How can I go about determining why this is happening?  That is, 
what are some good techniques on OpenBSD that let me debug this? 
And are there any clues that anyone can recommend I keep an eye 
out for?


It seems very strange that it is machine specific, but all 
evidence points to that.  I've scoured the manuals and POSIX specs 
looking for enlightenment to no avail.  (As far as I can tell, 
assignment is not a command, so the exit status should be the 
result of "false", which is 1.)


Note: Everything else seems to work fine in the 5.9 install, 
including all aspects of zsh, save for command substitution exit 
status.  My X200 is now a usable machine thanks to OpenBSD.  So a 
sincere thanks for that.


-- Geoff

Re: openbsd vs freebsd NAT performance

[Mike Burns]

On 2016-04-16 12.57.56 -0700, bluesun08 wrote:
> So, please can you say me how to activate SMP pf support in OpenBSD ?

Be sure to run a MP kernel. You can verify with: dmesg | grep MP

To track progress so you know where to help, search tech@ for "unlock":
http://marc.info/?l=openbsd-tech=2=1=unlock=b

-Mike

Re: openbsd vs freebsd NAT performance

[bluesun08]

So, please can you say me how to activate SMP pf support in OpenBSD ?



--
View this message in context: 
http://openbsd-archive.7691.n7.nabble.com/openbsd-vs-freebsd-NAT-performance-tp294095p294121.html
Sent from the openbsd user - misc mailing list archive at Nabble.com.

Re: Help with IPsec multiple transform policy

[Vijay Sankar]

  Quoting Sly Midnight :

> I got it to work exactly as you suggested using isakmpd.conf.
>
> It took me quite a bit of searching to find the correct sort of syntax
> for that file to achieve what I wanted but it now allows me to connect.
>
> But I've run into another issue that I cannot resolve myself.
>
> Once I connect from ANY client, I can only move data on the VPN for a
> few seconds then it goes dead.
>
> I thought it might be an MTU issue, but I tried setting the MRU setting
> fairly low (such as 1200) in npppd and that didn't solve it.
>
> I tried setting skip on enc0 as well as pppx0 in pf rules and that
> didn't work either.
>
> What else could I be missing? Why would it work, but only briefly?
> Sly

Hi,

If appropriate/practical, it may be useful to provide some details about
your configuration to the list (ipsec.conf, isakmpd.conf, npppd.conf,
pf.conf, sysctl.conf, etc)..

 From your description I am assuming that phase 2 dies on you but of course
it is just a guess.

I am in the process of migrating my 5.7 infrastructure to 5.9 and do not
see any issues with npppd -- android, blackberry, and ios clients are able
to use IPSec, and access dovecot, opensmtpd, and apache-httpd-openbsd
without any problems. I tested native mobile device clients that use
ActiveSync as well as imap and smtp -- no issues to report so far. I am
also testing Windows clients from my home to my lab environment to test
Samba 4.3.8 this weekend and so far so good with the VPN.

I don't have "enterprise" type connections but have three ISP links at my
office (ADSL that uses PPPoE, vDSL, and Cable) and two links at the lab
(vDSL, Cable). Only place where I had to change MTU etc., was with ADSL and
I had to do a "match on pppoe0 scrub (no-df max-mss 1340)" in my pf.conf
re. ADSL for VPN to work properly.

Looking through my logs I see long-lived connections such as (changed IP
addresses but the rest are from the log):

Apr 14 04:51:29 mx2 npppd[19526]: ppp id=175 layer=base logtype=TUNNELUSAGE
user="xx" duration=58390sec layer2=L2TP_ipv4 layer2from=a.b.c.d:1701
auth=MS-CHAP-V2 data_in=277392bytes,3364packets
data_out=235270bytes,2576packets error_in=1 error_out=0 mppe=yes
mppe_in=128bits,stateless mppe_out=128bits,stateless iface=tun0

Vijay
-- 
Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
vsan...@foretell.ca

Re: Standard way to create a generic queue in ksh

[Raul Miller]

The way I have created a queue in shell is:

(1) create a directory to manage queue entries.
(2) file names in that directory are high precision timer values
(3) file contents in that directory are command lines.

Then you need something to service the queue.

In my case, I also decided that I could tolerate one retry of a queued
command, but that in general queued commands needed to be designed to
"catch up" when they were falling behind on work (by taking bigger
bites out of the workload - more efficient, but higher latency).

Also, in my case, I occasionally was getting code running on machines
which did not have that command queue which was written for the
machine which had that command queue. This was pure sloppiness, but I
decided that I wanted those queue attempts to fail. If you decide that
that fits how you are working, the details of how you detect whether
the command queue service routine is available should relate to
whatever you have servicing the queue... So, if your queue directory
is $HOME/q/ that gives you a shell script something like this:

#!/bin/sh
set -e

if [ -r FIXME ]; then
if [ 2 -gt $(fgrep -l "$*" $HOME/q/* 2>/dev/null | wc -l) ]; then
echo "$*" >$HOME/q/$(perl -MTime::HiRes -e 'print
Time::HiRes::time')
else
echo "$* is backed up, not adding another retry"
fi
else
echo FIXME
exit 1
fi

Replace the FIXME bits with something appropriate and/or redesign it
to your own specifications. Just remember that when you push the
limits of whatever resources you have available, things can break and
you will need to do something to isolate and address those problems.

I hope this helps,

-- 
Raul


On Sat, Apr 16, 2016 at 8:59 AM, andrew fabbro  wrote:
> On Sat, Apr 16, 2016 at 4:32 AM, Craig Skinner 
> wrote:
>
>> A bloated way to do that is with an SQLite database, with a table's
>> unique primary key being some (job number) attribute. Another column
>> could auto timestamp on row insertion, so you could query on job number
>> or time added. Unless you've other data to retain, it is rather bloated.
>>
>
> Not sure I agree - sqlite is pretty lightweight.  I have a job system that
> runs hundreds of jobs on many systems, each dumping results into local
> daily sqlite files which are then scp'd back and consolidated for
> reporting.  This gives us the ease of standardized job results and
> reporting without the need to have an HA DB every system can report to,
> load DB clients all over the place, DB security with remote access, etc.
>  (We need to gather results somehow, so rather than write some custom
> format or something like XML, sqlite is an easy format to use).  You can
> access sqlite on the command line in shell scripts if need be.  DB sizes
> are in MB.
>
> You might be saying bloated because it's writing SQL, etc. and for a
> sysadmin who's focused on systems and is not a code-writer, that's totally
> fair - SQLite is much more pleasant when you have perl or python and can
> properly bind variables, etc.
>
> I'd say the OP is crossing into programming rather than scripting.  I'm
> making an artificial distinction (since shell scripts are certainly
> programs) but in my experience, once you start needing more complex data
> structures, you've outgrown the shell and should look at something like
> perl, python, etc.  Not saying there aren't ways to do queues in
> bash/ksh/etc., just...why would you?
>
> --
> andrew fabbro
> and...@fabbro.org

Re: openbsd vs freebsd NAT performance

[Mihai Popescu]

> Hi,

> beside OpenBSD 5.8 i installed FreeBSD 10.3 on my router-pc. For routing i
> use pf.

Pf has not very much to do with routing, as I remembeer. Can you give
more details?

> I noticed that the routing/NAT-performance is in FreeBSD noticeable higher
> than in OpenBSD.

How did you test? The results would be interesting. Please add some
hardware description of your machine.

> I think that is due to the SMP-support of pf in FreeBSD.

Should be. As far as I know, pf is a real hungry beasts when it comes
to multicore CPUs.
But I think you didn't activate SMP pf support in OpenBSD, did you?

> Is there also a SMP support (intended) in OpenBSD?

Yes there is, still some things are in work.

dhcp-class-identifier in dhclient

[mxb]

Hey,
is there any reason to no setting dhcp-class-indentifier by default in
dhclient?
My guess is that this is probably not mandatory?

//mxb

Re: diff for help.1

[Theo de Raadt]

>Why not ed?

And why not list all commands??

Because it is unneccessary.  I believe you are not trying to
contribute anything valuable with your comment.

new

[Tito Mari Francis Escano]

0
C Philippines
P National Capital Region
T Taguig City
Z 1633
O EDGEKIT Computer Systems
I Tito Mari Francis H. Escano
A Block 1 Lot 24 Zone 1 Central Bicutan
M titomarifran...@edgekit.com
U http://www.edgekit.com/
B +63 916 640 2020
X N/A
N EDGEKIT Computer Systems is proudly the only company in the Philippines
that provides professional support, consultancy and training services on
 OpenBSD and PostgreSQL, the best in their class free and open source
enterprise platforms.

EDGEKIT Computer Systems provides small and medium scale businesses
with simple, secure and functional solutions built on our Weapon of
Choice selection of secure platforms, tools, methodologies and
resources to give them the competitive edge.

openbsd vs freebsd NAT performance

[bluesun08]

Hi,

beside OpenBSD 5.8 i installed FreeBSD 10.3 on my router-pc. For routing i
use pf.
I noticed that the routing/NAT-performance is in FreeBSD noticeable higher
than in OpenBSD. I think that is due to the SMP-support of pf in FreeBSD.

Is there also a SMP support (intended) in OpenBSD?

Regards

Alex



--
View this message in context: 
http://openbsd-archive.7691.n7.nabble.com/openbsd-vs-freebsd-NAT-performance-tp294095.html
Sent from the openbsd user - misc mailing list archive at Nabble.com.

Re: Help with IPsec multiple transform policy

[Sly Midnight]

I got it to work exactly as you suggested using isakmpd.conf.

It took me quite a bit of searching to find the correct sort of syntax
for that file to achieve what I wanted but it now allows me to connect.

But I've run into another issue that I cannot resolve myself.

Once I connect from ANY client, I can only move data on the VPN for a
few seconds then it goes dead.

I thought it might be an MTU issue, but I tried setting the MRU setting
fairly low (such as 1200) in npppd and that didn't solve it.

I tried setting skip on enc0 as well as pppx0 in pf rules and that
didn't work either.

What else could I be missing? Why would it work, but only briefly?

Sly

Re: Standard way to create a generic queue in ksh

[andrew fabbro]

On Sat, Apr 16, 2016 at 4:32 AM, Craig Skinner 
wrote:

> A bloated way to do that is with an SQLite database, with a table's
> unique primary key being some (job number) attribute. Another column
> could auto timestamp on row insertion, so you could query on job number
> or time added. Unless you've other data to retain, it is rather bloated.
>

Not sure I agree - sqlite is pretty lightweight.  I have a job system that
runs hundreds of jobs on many systems, each dumping results into local
daily sqlite files which are then scp'd back and consolidated for
reporting.  This gives us the ease of standardized job results and
reporting without the need to have an HA DB every system can report to,
load DB clients all over the place, DB security with remote access, etc.
 (We need to gather results somehow, so rather than write some custom
format or something like XML, sqlite is an easy format to use).  You can
access sqlite on the command line in shell scripts if need be.  DB sizes
are in MB.

You might be saying bloated because it's writing SQL, etc. and for a
sysadmin who's focused on systems and is not a code-writer, that's totally
fair - SQLite is much more pleasant when you have perl or python and can
properly bind variables, etc.

I'd say the OP is crossing into programming rather than scripting.  I'm
making an artificial distinction (since shell scripts are certainly
programs) but in my experience, once you start needing more complex data
structures, you've outgrown the shell and should look at something like
perl, python, etc.  Not saying there aren't ways to do queues in
bash/ksh/etc., just...why would you?

-- 
andrew fabbro
and...@fabbro.org

Re: diff for help.1

[Rob Pierce]

> From: "Pavan Maddamsetti"  
> To: "misc"  
> Sent: Friday, April 15, 2016 10:55:28 PM 
> Subject: Re: diff for help.1 
> 
> Why not ed? 

vi(1) mentions ex(1) which is maybe good enough for a new user. 

Both the "Welcome to OpenBSD" email message as well as help.1 reference man
and highlight the -k option, so maybe that is enough. Running man -k editor
lists them all. 

When I reviewed help.1, given the other commands presented (e.g. cd, ls,
cat) I would have expected some mention of a text editor to help a new
user/admin (though afterboot.1 does reference vi(1) under SEE ALSO).

Rob

Re: Standard way to create a generic queue in ksh

[Craig Skinner]

Hi Jon,

On Fri, Apr 15, 2016 at 10:29:24PM +0200, Jon S wrote:
> 
> As a port of a backup solution, i needa a queue. Having looked around the
> net, i haven found any standard way (besides named pipes) that I can use.
> 

Here's some shell scripting suggestions (which can be BASH specfic...)
http://mywiki.wooledge.org/NamedPipes
http://mywiki.wooledge.org/BashFAQ

> Requirements are:
> * There will be few elements in the queue (<20)
> * No critical section  is needed (this is already adressed using mkdir)
> * The queue must be possible to examine without altering it
> * It would be nice if the queue only can contain unique elements (no point
> in queing the same job twice)
> 
> My own two ideas are:
> * Files in a dir. Gives uniqueness if names are well selected, but I dont
> know how to handle element order neatly

These might give you some ideas:
http://mywiki.wooledge.org/BashFAQ/099
http://mywiki.wooledge.org/BashFAQ/003
With find(1) -newer [a touched timestamp file] and/or sort(1).

> * Rows in a file: No guarantee for unique jobs. Easy to define order and to
> add a new job at end (or beginning) of a file or pop a job
> 

A bloated way to do that is with an SQLite database, with a table's
unique primary key being some (job number) attribute. Another column
could auto timestamp on row insertion, so you could query on job number
or time added. Unless you've other data to retain, it is rather bloated.

> Any ideas or proposal of standard way to do this?
> 

Could the base batch(1) or at(1) tools do it well enough for you?

On OpenBSD, you'll be using ksh, but there's a lot of ideas on the web
with the search "bash queue" which might be useful starting ideas.

Cheers!
-- 
On-line, adj.:
  The idea that a human being should always be accessible to a computer.

library-specs(7) erratum

[Bob NW8L]

Hello,

Now that architectures without shared libraries aren't supported [1], 
shouldn't the reference to them should be removed from library-specs(7)?


Index: library-specs.7
===
RCS file: /cvs/src/share/man/man7/library-specs.7,v
retrieving revision 1.11
diff -u -p -r1.11 library-specs.7
--- library-specs.7 27 Sep 2015 14:27:52 -  1.11
+++ library-specs.7 16 Apr 2016 07:18:00 -
@@ -92,16 +92,6 @@ If a specific major number is needed, us
 If the minor component is left empty, any minor will do.
 If both components are left empty, any version will do.
 .Pp
-If a given architecture does not support shared libraries, all
-.Ev LIB_DEPENDS
-will be turned into simple
-.Ev BUILD_DEPENDS
-checks, and so,
-failure to mention
-.Ev RUN_DEPENDS
-if the port needs anything beyond libraries from the dependent port will
-lead to strange errors on such architectures.
-.Pp
 Most specifications won't mention a
 .Pa path :
 .Xr resolve-lib 1

[1] http://article.gmane.org/gmane.os.openbsd.cvs/154333