Re: allocating contiguous memory in user space

2021-05-16 Thread gwes

On 5/15/21 9:17 PM, Alessandro Pistocchi wrote:

Hi all,
is there any way in openbsd to allocate contiguous memory pages in user
space?

Thanks,
A

mmap(2) will give you a block of contiguous virtual pages
mlock(2) will lock them down

The OS tries very hard to conceal physical page addresses
from a user program. There are a number of security attacks
via the virtual memory system which the OS is designed to mitigate.

The physical address of allocated memory is entirely
controlled by the VM system.

Some I/O devices require contiguous physical pages or memory
in a particular region.

In all other cases the physical address of a page doesn't matter,
can't be determined and can be reallocated.

Without more information about your application it's impossible
to give a more complete answer.

Geoff Steckel



Re: httpd - conditional redirects

2021-04-28 Thread gwes




On 4/28/21 8:45 PM, theni...@gmail.com wrote:

Hope I'm putting this in the appropriate mailing list.

A minor (I hope) potential feature request for httpd:

I wish to redirect clients not from a certain IP (e.g. my public IP at
home) to a different location, temporarily. The purpose of this is to
allow setting up a "maintenance" page so that I can properly test my
site with a piece of mind before actually making it available to
other visitors. I personally consider this a crucial ability.


pf (4) is happy to redirect anywhere, anywhen.

  2nd httpd on another machine listening on standard ports
or
  2nd httpd on same machine listening on chosen nonstandard ports

redirect in gateway

or
  1 httpd listening on outside address
  1 httpd listening on internal address
pf on server redirects as necessary

or

use testing.yourdomain.whatever if you can

or

play fast and loose with a schizo name server:
   server.yourdomain is served by an outside nameserver
   server.yourdomain is intercepted by your local nameserver

or...

some of these won't work for you
somebody else can think up more

Geoff Steckel



Re: 6.9 Current amd64 xfce seems to freeze and not respond to mouse clicks or keystrokes

2021-04-10 Thread gwes




On 4/10/21 5:22 PM, Tom Smyth wrote:

Hello,

1) issue does not occur with fvwm or with chrome running in fvwm

so the issue seems to be confined to xfce, and I was running  just 1
xfce terminal session
2) (so the issue is not related to chromium)

Thanks


O
--
Kindest regards,
Tom Smyth.


Hi Tom,
Some application that you can't see is grabbing focus and not letting go.

On another OS using xfce (XUbuntu) the screensaver sometimes causes 
something

extremely similar. Mouse cursor moves but nothing else responds.
The workaround is to use control-alt-F1 to get a
plain console and ps -ax | grep screen then doas kill .

If it's not a screensaver it's almost always a second browser copy.
I just find likely greedy candidates in the ps and kill until the 
problem goes away.


If the X server won't let you use control-alt-Fx to change screens you'll
have to ssh in.

Geoff Steckel



Re: home printer

2020-09-18 Thread gwes

On 9/17/20 3:15 PM, Greg Thomas wrote:

I've always been happy with the cheap Brother laser printers with ethernet,

On Thu, Sep 17, 2020 at 10:07 AM Ingo Schwarze  wrote:

Jan Stary  writes:

Can people please recommend a home laser printer
that is known to work well with OpenBSD?

I would like to avoid cups, and possibly a2ps
and foo* and if= and all that dance
- a printer that speaks postscript and is as easy as
lp:lp=/dev/lp:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:HP at least used to 
(and I assume still do) make several decent
printers that spoke Postscript.

That answer used to be spot on until about the year 2000.  After
that, quality of HP laser printers went down the drain very rapidly.


In particular, I've used the
CP1525nw in the past with OpenBSD.  Haven't tried it in a couple
years, though; none of my OpenBSD machines need to print, these
days.

Same here.  Currently, a Kyocera P2135dn is sitting on the desk here,
but i can't say whether it is good because i'm printing so little.


Brother - 7 years internal plastic piece broke,
 could have fixed it but complete disassembly didn't seem worth it
Canon - 2 years (replaced, didn't fail)
HP M402n - 2 1/2 years 0 problems expensive cartridges
  Still running. Relatively low usage - +-4000 pages.
  HP printers seem to come in at least 2 grades.

All worked with lpr & postscript without problems.

Whatever you get be -sure- to configure pf
so it can't call home! Turn off wireless as
well if you don't need it.
Big security holes.

Geoff Steckel



Re: exFAT support

2020-08-06 Thread gwes

On 8/6/20 5:47 PM, Bryan Steele wrote:

On Thu, Aug 06, 2020 at 02:16:11PM -0700, jo...@armadilloaerospace.com wrote:

With Microsoft's release of the [exFAT] spec last year, is the path open for
kernel support now, when someone gets around to it?

I don't know the details, but I believe one issue with exFAT has been
Microsoft and software patents, not just available documentation. Linux
may eventually get a kernel implementation but I'm not sure that helps
us.

https://www.zdnet.com/article/microsoft-readies-exfat-patents-for-linux-and-open-source/
The license as published tries very hard to restrict use only to "Linux 
systems."

There are two sentences that might allow additional licenses.
Lawyer would be involved.

geoff steckel



Re: strlcpy version speed tests?

2020-07-01 Thread gwes

On 7/1/20 8:05 AM, Luke Small wrote:

I spoke to my favorite university computer science professor who said
++n is faster than n++ because the function needs to store the initial
value, increment, then return the stored value in the former case,
while the later merely increments, and returns the value. Apparently,
he is still correct on modern hardware.

For decades the ++ and *p could be out of order, in different
execution units, writes speculatively queued, assigned to aliased registers,
etc, etc, etc.

Geoff Steckel



Re: OpenBSD Readonly File System

2020-06-27 Thread gwes




On 6/27/20 10:57 AM, Stuart Henderson wrote:

On 2020-06-26, Marko Cupać  wrote:

On 2020-06-24, Aaron Mason  wrote:
Auto filesystem repair is bad juju.

On 2020-06-25 11:17, Stuart Henderson wrote:
Nonsense. For many, the possible downsides of automatically running
fsck -y are much less a problem than the downsides of *not* running it.

Some time ago I wrote here on misc@ about read-only setup, where I
intended to modify rc(8) in order to be able to relink kernel before
mounting filesystems read-only, and - if I remember correctly - I was
warned never to modify rc(8) directly as it's considered as part of base
system, and I should only affect it with rc.local, which I did.

Is there a way to run fsck -y automatically without modifying rc(8)? Is
modifying rc(8) now supported?

No, you still need to modify rc to do that, so you need to remember to
reinstate it after updating. It would be nice if that wasn't needed but
diffs to make it configurable have never been approved.



20 years or so I worked on a network appliance based on FreeBSD.
It was required to come up after power failures no matter what.
We thought that this was the simplest way to harden the
system enough for our requirements:

We separated the boot environment from the runtime environment.

The initial root filesystem was never writable except during updates.
It was populated with the absolute minimumnecessary for
the system to come up capable of calling home.

During the transition to normal operation filesystems
containing the usual files were mounted over it
making it invisible and inaccessible.

The runtime filesystems could fail fsck during boot and the system could
be remotely repaired.
They could be refreshed via newfs and tarballs during boot if desired.

best,
Geoff Steckel





axen - need working USB NIC using axen to test driver change

2020-05-03 Thread gwes

Currently axen.c has its PHY address hardwired to 3.
I have a StarTech which has the PHY at 0.
The driver currently searches for all PHYs connected to the MII
and then ignores the result.
I want to test my fix on devices which work now.

Can anyone point me to a USB NIC which works with axen?
thanks
Geoff Steckel



how to find lock contention

2020-02-04 Thread gwes

I'm copying a directory tree from one ssd to another.
Top reports +- 33% system time and +- 28% spin time.

Is there any easy way to determine which lock is hot?
Mostly from morbid interest & curiosity about gruesome
detail of the current filesystem implementation.
Geoff Steckel



Re: How do I change the birth time of a file?

2020-01-17 Thread gwes

On 1/17/20 4:20 AM, Otto Moerbeek wrote:

On Thu, Jan 16, 2020 at 09:20:58PM -0800, William Ahern wrote:


On Thu, Jan 16, 2020 at 01:16:47PM +0100, Otto Moerbeek wrote:

On Thu, Jan 16, 2020 at 11:20:10AM +, gritzmann wrote:


Hi,

How do I change the birth time of a file? `touch -acm -d "1980-01-01 00:00:00" 
myfile` changes only the access, modify and change times.

`stat myfile` returns `10 215746 -rw-r--r-- 1 me me 0 0 "Jan  1 00:00:00 1980" "Jan  1 
00:00:00 1980" "Jan 16 13:00:33 2020" 16384 0 0 myfile`

Thanks!

Sent with ProtonMail Secure Email.


Change time != creation time. There is no such thing as creation time
in unix.

Not in Unix, but with UFS2 FreeBSD added birth time, which is documented
everywhere--including in source code--as being synonymous with "creation
time". OpenBSD added the st_birthtime field to struct stat in 2004,

   
https://cvsweb.openbsd.org/src/sys/sys/stat.h?rev=1.14=text/x-cvsweb-markup
   
https://github.com/openbsd/src/commit/cc2fc615c6e2dee87e5a3cd5a655a2ee5ef778c8

but as far as I can tell it's not set anywhere in the kernel.

FWIW, birth time has been adopted by ext4, ZFS, AFS, HAMMER2, and possibly
other file systems, but only the *BSDs seem to have added st_birthtime in
struct stat. On Linux it's stx_btime in struct statx, and as best I can tell
Solaris requires querying the A_CRTIME (creation time?) attribute using
getattrat. None of this is particularly relevant to OpenBSD, and I don't
mean to advocate, but after having done the leg work I feel like I should
commit this to an archive for posterity...


The change time (c_time in struct stat) cannot be explicitly set by
any API and is maintained by the kernel.

As far as I can tell from the FreeBSD man page for utimes and friends, this
is likewise true for birth time.


Thanks for this detailed extra info. I was vaguely aware that some
filesystem implementations have a creation time, but

- it is not in Posix and, even if *some* filesystems have it,
- there is no API to set it and no generally accepted API to get it.

-Otto


If these times were intended to help backup programs determine whether
a particular inode's content and attributes have changed since the last
backup (as I added and used them for backup purposes under TOPS-10)
then there must not be any way for any program to modify them. The kernel
must only change them if it changes the file in such a way that a backup
is out of date.

I suspect this is the reason for these fields. If not, they should be
used for that purpose.



Re: How do I change the birth time of a file?

2020-01-16 Thread gwes

On 1/17/20 12:20 AM, William Ahern wrote:

On Thu, Jan 16, 2020 at 01:16:47PM +0100, Otto Moerbeek wrote:

On Thu, Jan 16, 2020 at 11:20:10AM +, gritzmann wrote:


Hi,

How do I change the birth time of a file? `touch -acm -d "1980-01-01 00:00:00" 
myfile` changes only the access, modify and change times.

`stat myfile` returns `10 215746 -rw-r--r-- 1 me me 0 0 "Jan  1 00:00:00 1980" "Jan  1 
00:00:00 1980" "Jan 16 13:00:33 2020" 16384 0 0 myfile`

Thanks!

Sent with ProtonMail Secure Email.


Change time != creation time. There is no such thing as creation time
in unix.

Not in Unix, but with UFS2 FreeBSD added birth time, which is documented
everywhere--including in source code--as being synonymous with "creation
time". OpenBSD added the st_birthtime field to struct stat in 2004,

   
https://cvsweb.openbsd.org/src/sys/sys/stat.h?rev=1.14=text/x-cvsweb-markup
   
https://github.com/openbsd/src/commit/cc2fc615c6e2dee87e5a3cd5a655a2ee5ef778c8

but as far as I can tell it's not set anywhere in the kernel.

FWIW, birth time has been adopted by ext4, ZFS, AFS, HAMMER2, and possibly
other file systems, but only the *BSDs seem to have added st_birthtime in
struct stat. On Linux it's stx_btime in struct statx, and as best I can tell
Solaris requires querying the A_CRTIME (creation time?) attribute using
getattrat. None of this is particularly relevant to OpenBSD, and I don't
mean to advocate, but after having done the leg work I feel like I should
commit this to an archive for posterity...


The change time (c_time in struct stat) cannot be explicitly set by
any API and is maintained by the kernel.

As far as I can tell from the FreeBSD man page for utimes and friends, this
is likewise true for birth time.


In the extremely dim past I added this field to a TOPS-10 filesystem
to help backup programs select candidates for archiving

geoff steckel



Re: Userland PCI drivers possible in OpenBSD?

2020-01-10 Thread gwes

On 1/9/20 10:58 PM, Joseph Mayer wrote:

Maybe this topic is better suited for tech@, you tell:

Is there some way I can implement PCI drivers in userland in OpenBSD?

Is there any reason not to write a conventional device driver and
build an OS including that driver?

While the kernel environment for a device driver is admittedly
complex, it's likely that there are enough examples and historical
information in published papers and mailing list history to help.
There are a lot of drivers to look at for clues.

There may be a driver which you could extend or adapt to your needs.
Adding an IOCTL, for instance, might suffice.

Long established policy is that your driver is not supported in any way.
Questions which show full research beforehand and good comprehension
of the kernel environment are sometimes answered. Ones showing little
diligence beforehand are ignored or laughed at.

Geoff Steckel



Re: Awaiting a diff [was: Re: File systems...]

2020-01-08 Thread gwes

Suggestion: to improve file system performance,
first document the bad behavior in detail.

Begin with examples of traces/logs of disk accesses associated
with file system operations.

Include scenarios (one hopes reproducible ones) to provoke
bad behavior.

Are reads worse than writes? Sequential vs. random?
Interleaved r/w on one file? On more than one file simultaneously?

Examples from other O/S which are better or worse?

Without this very detailed data it's all noise.

Being able to get good traces & correlate them with OS activity
shows at least some competence dealing with OS internals.

geoff steckel



Re: Turn off Swap on boot disk

2019-11-24 Thread gwes

On 11/24/19 9:35 AM, Stuart Henderson wrote:

On 2019-11-22, gwes  wrote:

First, why is your workload causing swapping? That hasn't been
a good idea since the beginning of computing.

Even if the main workload is OK, relinking the kernel (reorder_kernel)
causes swapping on smaller-memory systems.

Been there in 1980 on a KA-10. We fixed the linker to
do multiple passes so it never had to have all the inputs
in core at the same time. Not gonna happen to gnu ld.

Just for giggles I split the link into 4 partials with -r
then linked the partials with the low core.

Reduced RSS from over 200M to about 80M.
The output text size is identical.
Unfortunately some bss allocation changed so I can't
say the output is identical.

That could be tracked down if there were any interest.

Observation: ulimit -d 9 didn't change behavior even
when dsize was well over 100. Or am I assuming wrong things?

I've never seen an Alix so this may be impossible but
why don't you install a larger boot drive?

With how they're often used, it's usually easier to replace the whole
machine. Maybe also cheaper, if they can be replaced by mailing a new
machine rather than having to visit a remote site (swapping the CF card
requires removing the system board from the case, not just opening the
case).

Adding swap on USB is one way to eke out another release or two's use
from the machine that can be done fairly easily without a visit..

All very reasonable when the machine is in someone else's place.
I'm thinking ahead about my little Edgerouter... will I have to
replace that for 6.8? Will i386 die [well, it should have decades ago]

Geoff Steckel



Re: Turn off Swap on boot disk

2019-11-22 Thread gwes

On 11/21/19 2:47 AM, Sean Kamath wrote:

Hello.

Can someone provide me a pointer to how to do this?

I have a bunch of Alix 2d13 boxes.  With 6.6, I’ve found I need more swap than 
the default layout on a 2G compact flash drive has.  So, I got some 1G USB 
thumb drives, and want to use JUST those for swap.  Despite different attempts 
(setting the mount_opts to xx, setting mount_opts to “priority=1”), I can’t 
seem to prevent the swap on the boot disk being added with priority = 0.

Can I do anything to turn it off or change the priority, short of changing the 
filesystem type?

Thanks,
Sean


I think you're trying to solve the wrong problem(s).

First, why is your workload causing swapping? That hasn't been
a good idea since the beginning of computing.

Second, USB sticks are not designed to do frequent writes.
If you need more swap space and have a USB port open, get a cheap 100G
flash drive with a USB interface like a portable drive.

I've never seen an Alix so this may be impossible but
why don't you install a larger boot drive?

Geoff Steckel

































i



Re: teco, and Re: vi in ramdisk?

2019-11-15 Thread gwes

On 11/15/19 1:59 PM, gwes wrote:

TECOC from github...
For general amusement:

without video (curses)
  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT TT TIME COMMAND
 1000 29775 86827   0  28   0   540  1296 -   T p2 0:00.00 ./tecoc
$ size tecoc
text    data    bss dec hex
102449  13096   13424   128969  1f7c9

with video (curses)
$ size tecoc
text    data    bss dec hex
114772  13456   12432   140660  22574
  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT TT TIME COMMAND
 1000 82440 86827   0  28   0   808  2296 -   T p2 0:00.01 ./tecoc

for comparison:

$ size /bin/ed
text    data    bss dec hex
207704  10800   24264   242768  3b450

  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT TT TIME COMMAND
 1000 75971 86827   0   3   0   256   196 -   Tp p2 0:00.00 ed

Interesting to note that the text size of ed(1) is almost twice that 
of vi.

RSS is larger for teco. 1.3MB isn't too bad, though.

On disk:

12412$ ls -l tecoc
-rwxr-xr-x  1   xxx  256920 Nov 15 13:48 tecoc*

12494$ ls -l /bin/ed
-r-xr-xr-x  1 root  bin  229928 Apr 13  2019 /bin/ed*


As Mr. Davis kindly points out, everything in /bin is statically linked.

With -Bstatic
$ ls -l tecoc
-rwxr-xr-x  1     1472504 Nov 15 15:47 tecoc*

Still not huge. I don't know what the current upper limit for
programs in the install medium is. As this is a totally irrelevant
thread, I suspect that squashing teco into the single install
executable would only raise it 250K because it uses only very
vanilla libraries.

Geoff Steckel



Re: Home NAS

2019-11-15 Thread gwes

[misc intermediate comments removed]
On 11/15/19 3:54 AM, Andrew Luke Nesbit wrote:

In particular I'm trying to figure out a generally applicable way of 
taking a 
_consistent_ backup of a disk without resorting to single user mode.


I think COW file systems might help in this regard but I don't think
anything like this exists in OpenBSD.


COW in the filesystem, no. However...
a backup is a precautionary copy-before-write.
The only difference is the time granularity.

Consistency? An arbitrary file system snapshot doesn't guarantee that
you won't see -application level- inconsistent data, just that the files
didn't change during backup. Even a COW system that doesn't reveal
a new version of a file until it's been closed won't protect you
from an inconsistent group of files.

What groups of files --must-- be perfectly archived?

If you (a) can pause user activity
(b) can tolerate some inconsistency in captured system log files,
then just run a backup.
Partial DB transactions had better be OK or get a new DBM.
You might have to pause cron(8).
I don't remember any other daemon that would care.

Some archive programs will complain if the size or modification time
of a file changes during copy or if the directory contents change.
Something could be done to automatically go back for those.

Depending very much on your mix of uses, don't even stop anything.

Breaking up the backup into sections - per project, per file system, etc.
can make the pauses less objectionable. It can make recovery easier as well.
Assuming you have control over the system files those only need a couple of
copies when they change, for instance.

Brute force:
  ls -Rl /users /proj1 /proj2 > before0
  $BACKUP -o /$BACKUPFS/backup-$(date)
  ls -Rl /users /proj1 /proj2 > after0

# remove known don't-cares
  sed -f ignores before0 > before
  sed -f ignores after0 > after

# check to see if any action needed
  diff before after > changed

  grep -f vitalfiles changed > urgent
  cat urgent changed | mail -s "changes during backup $(date)" you

# calculate list of files needing recopy
  $SCRIPT changed > newbackup

# copy files missed - should run quickly
  $BACKUP -o /$BACKUPFS/bdelta-$(date) -f newbackup

This worked pretty well for me.
The truly paranoid would put a while loop around the diff & recopy...

Binary files can be regenerated if the source *and* environment
are backed up.


Storing the environment is a tricky problem that I haven't found an 
entirely satisfactory solution for, yet.

The key is for the project never to use an unqualified program -
 always "our current version".

One solution is to copy or link a consistent set of utilities
(compiler, linker, libraries) into the project and always use
those in production. Then a backup will capture everything.
This won't necessarily work if the OS changes its ABI but it
can be pretty effective.
I've been in a project that used this approach and it did work.

Keeping an automatic record of utility and library versions used works as
long as the system itself is backed up well.

The discipline to keep everything tidy, ... well.
Without regard to backups, the precaution to take periodic
snapshots of a project, transplant it into an empty system
and make sure the snapshot actually works
has been erm, revealing.

# mv /usr/bin/cc /usr/bin/saved-cc
# rm /usr/bin/cc
$ make
.not found 

Andrew

It can be a pain to design a procedure that fits your needs
and doesn't need a staff of operators (:-(

Good luck!

Geoff Steckel



teco, and Re: vi in ramdisk?

2019-11-15 Thread gwes

TECOC from github...
For general amusement:

without video (curses)
  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT  TT TIME COMMAND
 1000 29775 86827   0  28   0   540  1296 -   T p2 0:00.00 ./tecoc
$ size tecoc
text    data    bss dec hex
102449  13096   13424   128969  1f7c9

with video (curses)
$ size tecoc
text    data    bss dec hex
114772  13456   12432   140660  22574
  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT  TT TIME COMMAND
 1000 82440 86827   0  28   0   808  2296 -   T p2 0:00.01 ./tecoc

for comparison:

$ size /bin/ed
text    data    bss dec hex
207704  10800   24264   242768  3b450

  UID   PID  PPID CPU PRI  NI   VSZ   RSS WCHAN   STAT  TT TIME COMMAND
 1000 75971 86827   0   3   0   256   196 -   Tp    p2 0:00.00 ed

Interesting to note that the text size of ed(1) is almost twice that of vi.
RSS is larger for teco. 1.3MB isn't too bad, though.

On disk:

12412$ ls -l tecoc
-rwxr-xr-x  1   xxx  256920 Nov 15 13:48 tecoc*

12494$ ls -l /bin/ed
-r-xr-xr-x  1 root  bin  229928 Apr 13  2019 /bin/ed*



Re: Home NAS

2019-11-14 Thread gwes

On 11/14/19 3:52 PM, Andrew Luke Nesbit wrote:

Hi Dave,

On 15/11/2019 07:44, Raymond, David wrote:

I hadn't heard about file corruption on OpenBSD.  It would be good to
get to the bottom of this if it occurred.


I was surprised when I read mention of it too, without any real claim 
or detailed analysis to back it up.  This is why I added my disclaimer 
about "correcting me if I'm wrong because I don't want to spread 
incorrect information".


The reason why I brought it up on a public mailing list was to find 
out if anybody else has heard any inkling _at all_ regarding this, 
even a skerrick of a mention.


I have a feeling I may have even heard about it on this list but I'm 
not sure.  If somebody out there genuinely suspects that this happened 
then it would be good to know so we can clear it up.


Kind regards,

Andrew


There was a thread a couple of months ago started by someone either pretty
ignorant or a troll.
The consensus answer: no more than any other OS, less than many.

On 11/14/19, U'll Be King of the Stars  
wrote:


A couple of months ago I read a couple of reports of filesystem
corruption on OpenBSD.  I didn't have time to investigate deeply and I
don't know if these issues were even real.  Even if they were real I
don't know if the problem was due to user error or a defect in the OS.

This is an excellent reason for implementing a system that includes not
only backups, but long term storage /archives/ too.

Andrew

One size definitely doesn't fit all.
Backup strategies depend on user's criteria, cost of design and
cost of doing the backups - administration & storage, etc.

In an ideal world every version of every file lasts forever.
Given real limitations, versioning filesystems can't and don't.

If your data are critical, invest in a dozen or more portable
USB drives. Cycle them off-site. Reread them (not too often)
to check for decay. If you have much  available, get a
modern tape system.

The backup system used over 50 years ago still suitable for many
circumstances looks something like this:
  daily backups held for 1 month
  weekly backups held for 6-12 months
  monthly backups held indefinitely offsite.
Hold times vary according to circumstances.

The backup(8) program can assist this by storing deltas so that
more frequent backups only contain deltas from the previous
less frequent backup.

The compromise between backup storage requirements and granularity
of recovery points can be mitigated. The way to do it depends on
the type and structure of the data:

Some data are really transient and can be left out.

Source code control systems (or whatever the name is this week)
are a good way for intermittent backups to capture a good history
of whatever data is around if it's text.

DBs often have their own built-in backup mechanisms.

Binary files can be regenerated if the source *and* environment
are backed up.

Etc.

YMMV and MEGO
geoff steckel
been there, mounted the wrong tape... what write protect ring?



Re: OpenBSD and solid state disks

2019-11-03 Thread gwes

On 11/2/19 4:10 PM, Raymond, David wrote:

I recently installed OpenBSD on a Lenovo X1 Carbon with a solid state
drive and it works great.

My question is whether OpenBSD addresses the special characteristics
of solid state drives, especially those having to do with longevity
and reliability.  I can't find anything written on this.  Linux has
certain means for addressing this issue, such as fstrim as well as
various kernel options.  Is there anything I have missed with OpenBSD
on this subject?

Dave Raymond


Any modern drive will have write levelling. Check the rated number
of writes for the drive. Run iostat for a week or two to determine
average writes/time interval. Compare that against 10% of rated
writes. When you get there, replace the drive.

500 TB is a good number for write endurance.
Completely writing a 1TB drive every day gives you 50 days.
Writing 100GB a day gives you 500 days...
Do you write 20 DVDs a day? That's your answer.

Geoff Steckel



Re: surprisingly good net speed with 2 REs

2019-10-22 Thread gwes




On 10/22/19 11:06 PM, Chris Cappuccio wrote:

g...@oat.com [g...@oat.com] wrote:

Peaks at about 500mb/sec

tcpbench is a better test because it won't measure your disk i/o at the same
time

also, the realtek chip you mention has a hard limitation of around 500Mbps
on either transmit or receive, i'm not sure. this is according to luigi
rizzo's netmap testing on freebsd. try pairing it with a better chip
for testing. i'm able to get over 1Gbps single-TCP stream file xfer with
modern openbsd and fast machines and SSDs, but that's nothing notable
these days.


My interest was that (a) this was a real use case (b) the host
machines are old and slow (c) those machines in that use case
could max out the admittedly wimpy Realtek with considerable
CPU resources left over.
From the first versions introduced Realtek's advantage has been
very low price. They've never transferred at full line speed.
em is faster and I've seen 850-900 mb with it.
My point is that OpenBSD performs well with meager resources.



surprisingly good net speed with 2 REs

2019-10-22 Thread gwes
People occasionally comment about OpenBSD network performance.
A data point:
 Two REs connected via a switch.
 Looks like they are running as fast as they can.
 REs are notoriously slow. Cheap though so they're everywhere.

Peaks at about 500mb/sec

Mostly filesystem limited on the OpenBSD end. We know
that hasn't had a lot of work to speed it up for a long time.
Other vital things have had higher priority.

One vital detail: both ends used 256K buffers and might
have used larger ones effectively. Faster interfaces definitely
need very large buffers which can hold a large fraction of
a second of full rate traffic.

CPU usage in the 25% range so faster interfaces probably
would perform well.

Both CPUs are at least 4 years old and pretty slow even then.
EMs are faster but I don't have two of them running right now.

>From linux: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz 
   r8169 :01:00.0 eth0: RTL8168g/8111g 
minion:/home$ sudo tar cf - gwes | nc -O 262144 -N 192.168.2.9 47567

To OpenBSD 6.5: Intel(R) Celeron(R) CPU 1037U @ 1.80GHz
   re0 at pci3 dev 0 function 0 "Realtek 8168" 
store:$ nc -l -I 262144 0.0.0.0 47567 > /dome/save.tar

re0 in re0 out  total in   total out  
 bytes   bytes   bytes   bytes  
.
  34904768  86536834904768  865368  
  55758366 136828855758366 1368288  
  41976672 102508841976672 1025088  
  61146922 150108061146922 1501080  
  43696142 108448843696142 1084488  
  32553870  80451632553870  804516  
   5621180  150588 5621180  150588  
  13818462  34436413818462  344364  
.


geoff steckel



Re: Why regex doesn't work in while loop's condition?

2019-09-08 Thread gwes




On 9/6/19 5:27 PM, Theo de Raadt wrote:

Andreas Kusalananda Kähäri  wrote:


On Fri, Sep 06, 2019 at 02:38:18PM -0600, Theo de Raadt wrote:

Christian Weisgerber  wrote:


On 2019-09-06, Andreas Kusalananda Kähäri  wrote:


read x; while [ "$x" != [abc] ]; do echo "Not a, b or c"; break; done

The shells in the OpenBSD base system do not support matching regular
expressions with that syntax.  You may have been thinking of bash,

Just to head off crazy rumors: bash doesn't either.

Doesn't bash perform this task by invisibly calling out to perl?

You're thinking of zsh.

No I'm thinking it is all crazy.


CRAZY=true
[ is not [[
man ksh
[[ expression ]]
 Similar to the test and [ ... ] commands (described later), with
 the following exceptions:
...
   The second operand of the ‘!=’ and ‘=’ expressions are
 patterns (e.g. the comparison [[ foobar = f*r ]]
 succeeds).


X=a
if [[ "$X" == [abc] ]] ; then echo yes ; fi
prints yes
ksh is in base and is often a user's login shell



Re: IPv6 problems

2019-08-14 Thread gwes

On 8/14/19 4:45 PM, freda_bundc...@nym.hush.com wrote:

Hi, I just thought since the interface was vio that you're running in a virtual
environment. Providers like Vultr say "Important Note: If you add an IPv6
subnet to an existing machine, you must restart the server via the Vultr
control panel before IPv6 will work. Restarting via SSH or similar is not
sufficient. IPv6 would not work at all until the server has been restarted."


If the provider says anything like this and the VM hasn't been hard reset
via the VM host all bets are off



Re: IPv6 problems

2019-08-14 Thread gwes




On 8/14/19 2:36 PM, list wrote:

My hostname.vio0 now looks like this:

         inet6 alias /64
         !route add -inet6 default fe80::2de:361a:24aa:d7a6%vio
When doing a "ifconfig vio0" I get:

     vio0: flags=8843 mtu 1500

[...]
     inet6 fe80::2de:361a:24aa:d7a6%vio0 prefixlen 64 scopeid 0x1
     inet6  prefixlen 64

Take the "alias" out of your inet6 line in your hostname.vio0

Since your interface is vio0 I am assuming you are running a
guest VM on a server. I am also assuming that ip4 traffic is passing.

Your VM server should be sending you Route Advertisement messages.
You shouldn't have to set any route yourself. Doing so will confuse
things mightily.

Can you ping your own ipv6 address? If not something is really strange.

If you say
# tcpdump -ni -s 1500 icmp6

You should eventually see (lines wrapped)

13:17:46.508540 fe80::669e:f3ff:feec:fc7f > ff02::1:
  icmp6: router advertisement [class 0xe0]
Along with

13:17:19.309191 your_gateway_ip6 > 2xxx0::1:
  icmp6: neighbor sol: who has 2xxx0::1
13:17:19.311828 2xxx0::1 > 2xxx0::2:
      icmp6: neighbor adv: tgt is 2xxx0::1 [class 0xe0]

 It may take up to 20 minutes to see these messages.

If you never see any route advertisements your server isn't configured
to give you inet6 service.

Who are you trying to ping? Someone on your /64 or someone outside?
You must see neighbor solicitation msgs if you try to ping someone
on your /64. You must see them for your gateway if you try to ping
someone outside. Keep the tcpdump running and do the pings from
another virtual terminal.

If you say
# ndp -a

You should see

Neighbor Linklayer Address   Netif Expire    
S Flags

your_gateway             64:9e:f3:ec:fc:7f    vio0 4s    D R
your_hostname    52:54:00:27:22:43    vio0 permanent R l
fe80::669e:f3ff:feec:fc7f%vio0   64:9e:f3:ec:fc:7f    vio0 23h58m18s S R
fe80::bd8b:afb3:be72:bd06%vio0   52:54:00:27:22:43    vio0 permanent R l

If you say
# netstat -s
Among a ***lot*** of other statistics you should see something like
ip6:
    1312572 total packets received <<<
    907754 packets for this host <<<
    1107139 packets sent from this host <<<
.
icmp6:
    640 calls to icmp6_error
    Output packet histogram:
    unreach: 640
    echo reply: 1328
    multicast listener report: 6
    neighbor solicitation: 137965
    neighbor advertisement: 137761

    Input packet histogram:
    echo: 1328
    router advertisement: 56998 
    neighbor solicitation: 137770 
    neighbor advertisement: 137956 

.

The netstat -s output should show nonzero in the marked lines.

If you CAN ping hosts on your /64 and you CAN'T ping anyone else
if you CAN ping your gateway as a last resort set your default
ipv6 route via that host.

If things still don't work, excerpts of netstat -s
and the output from ndp -an and tcpdump -ni icmp6 should be informative.

geoff steckel




Re: SCM

2019-07-26 Thread gwes




On 7/26/19 8:29 PM, Австин Ким wrote:

Hi, all,
Sorry, been hella busy rushing to finish final graduation projects for school
and had no idea so many people weighed in with so much awesome feedback!


That said, OpenBSD has a cultural restriction of requiring people to
inspect the patches before incorporating them. Adopting git would be a
step away from that practice.

I was suggesting Mercurial (hg), not Git; I know Git would be problematic for
the OpenBSD Project in many ways.  Plus I find it unnecessarily complex.  And
also, regardless of which SCM was used, responsible area owners would obviously
be required to inspect patches before merging into the main branch, so I don't
see why adopting hg would necessarily be a step away from that practice.


Some of us (including myself) actually prefer CVS over git for tasks
where it is suffiecient because KISS.

I definitely appreciate the KISS argument, but I still feel that for new
developers Mercurial would present a lower learning curve than CVS; isn't that
also a fair measure of simplicity (conceptual simplicity)?


it is hard, almost impossible, to avoid destroying part of the
history during the conversion of the repository.

That argument makes sense; but on the flip side that argument also seems a
little fatalistic, basically resigning oneself to being stuck with CVS forever
because no one wants to invest the energy of activation required for migration.
I think back to how the FreeBSD Project moved heaven and earth to migrate from
CVS to SVN, a bold and technically challenging undertaking that impressed the
hell out of me personally; that was also not trivial, and I understand that the
FreeBSD Project has greater staffing and resources, but I honestly believe
OpenBSD developers could be up to the task of even a greater migration, e.g.,
from CVS to Mercurial (but only if >= 99% of the OpenBSD developer community
were all on board, i.e., a consensus of buy-in emerged from the community so
everyone would be all in so as not to engender hurt feelings or animosities).


Almost all developers prefer working on actual quality and
functionality of the system over spending time and effort on
infrastructure around it, unless the latter is really
important to make progress with the former.

I can't argue with that, and obviously code quality is infinitely more
important than what SCM you use, but I feel you run the risk of turning off
potential new developers coming out of colleges and universities who cut their
teeth on distributed SCM systems like hg and Git who might be taken aback at
why the Project is still stuck with CVS (and again, I am not advocating for
Git; though if it isn't obvious by now I really believe OpenBSD developers
would honestly like Mercurial; to me it just seems consistent with OpenBSD's
culture of cleanliness and simplicity).  I understand the flip-side argument,
that I'm sure some developers might be personally proud of having arcane CVS
knowledge borne out of slogging through for years with CVS, but to me that
seems more like an issue of personal pride than an indication that CVS is
objectively better than hg.


However, it requires rewriting git from scratch because the reference
implementation of git is not free software.  It comes infected with
a viral license.

Isn't CVS also GPL-licensed?  Or did OpenBSD completely rewrite CVS from
scratch under a BSD license?  Mercurial is still GPL v2, which is at least
better than GPL v3?

Finally my biggest argument (besides making contributing to the Project more
inviting to new developers, especially recent CS/CE/EE graduates) would be that
the bold challenge of migrating the entire codebase from CVS to Mercurial would
present a once-in-a-lifetime opportunity to start over with a fresh, clean
slate, once and for all tackle any issues that plagued working with CVS, and
have the rare opportunity to reset and redefine new processes that capitalize
on a quarter century of OpenBSD developers' working on maintaining a codebase
that is second to none.  It would be a monumental, fresh, clean start, albeit
an immensely technically challenging one; but one I have no doubt OpenBSD
developers could surmount.


Mercurial would require python in base and maybe someday it will require
also Rust.

Wow, I have no counter-argument for that :/
Of all the arguments made for CVS over hg, for me this is the one sole argument
I don't have an adequate response to.

Thanks to everyone who shed light on this potentially fraught issue.  I really
appreciate eveyone who took the time to write thoughtful, insightful responses,
based on technical considerations as opposed to dogma.  I only wish the most
salient points could be distilled and presented on an About page for the
Project for future newbies like myself who are newly coming to OpenBSD without
the quarter century of past context and its concomitant biases and are just
initially struck by seeing a major contemporary project still using CVS.

Thanks so much 

Re: Write to DVD-RAM

2019-07-26 Thread gwes




On 7/25/19 7:14 PM, Zhi-Qiang Lei wrote:

On Jul 25, 2019, at 10:24 PM, gwes  wrote:


On 7/24/19 10:19 PM, Zhi-Qiang Lei wrote:

Hi, I’m trying to encrypt a DVD-RAM before putting some files onto it on my 
OpenBSD 6.5 desktop. But neither dd nor disklabel seems able to work on the 
drive. Did I miss something?

$ dmesg | grep cd
cd0 at scsibus3 targ 1 lun 0:  ATAPI 5/cdrom 
removable serial.13fd3940302020202020
cd0 at scsibus3 targ 1 lun 0:  ATAPI 5/cdrom 
removable serial.13fd3940302020202020

$ doas dd if=/dev/urandom of=/dev/rcd0c bs=1k
dd: /dev/rcd0c: Invalid argument
1+0 records in
0+0 records out
0 bytes transferred in 0.000 secs (0 bytes/sec)

$ doas disklabel -E cd0
cd0> a
partition: [a]
offset: [0]
size: [2236704]
FS type: [4.2BSD]
cd0> w
cd0> p
OpenBSD area: 0-2236704; size: 2236704; free: 0
#size   offset  fstype [fsize bsize   cpg]
   a:  22367040  4.2BSD   2048 16384 1
   c:  22367040  unused
cd0> q
No label changes.

The same drive can be formatted and used on Mac OS X.

Thanks and best regards,
Siegfried


Did you try 2K blocks? The low level of CDROM only works that way.



Blocks larger than or equal to 2k get a "dd: /dev/rcd0c: short write on 
character device”. Regarding to cd(4) I thought the device is readonly, so dd(1) and 
disklabel(8) cannot write on it, but fdisk(8)  works fine.

$ doas dd if=/dev/urandom of=/dev/rcd0c bs=2k
dd: /dev/rcd0c: short write on character device
dd: /dev/rcd0c: Invalid argument
1+0 records in
0+1 records out
512 bytes transferred in 0.008 secs (57960 bytes/sec)

$ doas dd if=/dev/urandom of=/dev/rcd0c bs=512
dd: /dev/rcd0c: Invalid argument
1+0 records in
0+0 records out
0 bytes transferred in 0.000 secs (0 bytes/sec)


/dev/cd0 is likely a symbolic link to something else in /dev.
It's not clear what's going on unless we know exactly what's being used.
"cd0" is not a usual OpenBSD device access even though one sees
that in dmesg.

OpenBSD disk-like devices are usually referenced in the very
old style which distinguishes "raw" [unbuffered direct to device]
from "cooked" [system buffered]. This differs from at least Linux practice.
Dunno about other BSDs or Macs.
Buffered devices are essentially only used to mount as filesystems.

A raw device is /dev/r
A buffered device is 
/dev/

Note that there is always a partition letter.
The kernel will always emulate a 'c' partition = whole device if necessary.

So the most specific way to refer to your cd device is /dev/rcd0c.

As a convenience and to reduce operator errors, many system maintenance
programs will deduce /dev/rc from a bare device
like sd0. This can be confusing to people new to OpenBSD.



Re: Write to DVD-RAM

2019-07-25 Thread gwes



On 7/24/19 10:19 PM, Zhi-Qiang Lei wrote:

Hi, I’m trying to encrypt a DVD-RAM before putting some files onto it on my 
OpenBSD 6.5 desktop. But neither dd nor disklabel seems able to work on the 
drive. Did I miss something?

$ dmesg | grep cd
cd0 at scsibus3 targ 1 lun 0:  ATAPI 5/cdrom 
removable serial.13fd3940302020202020
cd0 at scsibus3 targ 1 lun 0:  ATAPI 5/cdrom 
removable serial.13fd3940302020202020

$ doas dd if=/dev/urandom of=/dev/rcd0c bs=1k
dd: /dev/rcd0c: Invalid argument
1+0 records in
0+0 records out
0 bytes transferred in 0.000 secs (0 bytes/sec)

$ doas disklabel -E cd0
cd0> a
partition: [a]
offset: [0]
size: [2236704]
FS type: [4.2BSD]
cd0> w
cd0> p
OpenBSD area: 0-2236704; size: 2236704; free: 0
#size   offset  fstype [fsize bsize   cpg]
   a:  22367040  4.2BSD   2048 16384 1
   c:  22367040  unused
cd0> q
No label changes.

The same drive can be formatted and used on Mac OS X.

Thanks and best regards,
Siegfried


Did you try 2K blocks? The low level of CDROM only works that way.



Re: Future of X.org?

2019-06-28 Thread gwes




On 6/28/19 1:56 PM, Christopher Turkel wrote:

Probably someday. X won’t be going away anytime soon.

On Friday, June 28, 2019, Nathan Hartman  wrote:


Came across this:

https://www.phoronix.com/scan.php?page=news_item=X.Org-
Maintenance-Mode-Quickly

Long story short, Red Hat hopes to switch from X.Org to Wayland and
expects X.Org to go into "hard maintenance mode" after that.

Relevant to OpenBSD?


I regularly run programs on one machine connected to a display
on another machine. AFAIK, the current state of Wayland makes
that difficult. I confess to not following it closely.

Implementing something as huge as Wayland in the kernel
mega-bloat. As a tightly coupled server process, maybe.
Sorta like X with a very different interface.

It also seems to assume a heavyweight desktop suite
to implement common X features Mega-bloat.

If I'm wrong, please point out sources.
Otherwise for my usage it's not nearly ready and
requires some complex porting/additional programs.

geoff steckel



Re: Filesystem corruption on OpenBSD routers after power outage?

2019-06-04 Thread gwes




On 6/4/19 3:30 PM, Mogens Jensen wrote:

I'm going to build a router for use in a remote location, and I have
chosen OpenBSD 6.5 for the task. Unfortunately, it's not possible to
protect the router with an UPS, so it will have to be resilient enough
to survive sudden power outages and still boot without manual
intervention.

In the past I have built a few Linux based routers and they were
configured to run from RAM. I have made some research to see if this is
also possible on OpenBSD and found that, while there are solutions to
have / read-only, none of this is officially supported.

Can anyone with experience running OpenBSD routers without UPS, tell if
filesystem corruption is going to be a problem after power outages, or
if there are any officially supported ways to make the system resilient
enough to not break after a power outage?

I'm using an mSATA disk with MLC flash in the router.

Thanks in advance.

Mogens Jensen

As Mr. Holland points out, a UPS doesn't really help overall reliability.

In practice, /, /bin, and /usr are effectively read-only except for
kernel and shared library randomization at boot time.
/var gets written infrequently for logs, etc.
/tmp, of course, is frequently written but its contents are irrelevant
after a reboot.

An important way to reduce disk activity is to mount all
filesystems "noatime". This suppresses effectively all writes
to /, /bin, and /usr after boot. Changes to /var get pushed to
disk fairly quickly.
The likelihood of significant corruption is very small.

In practice, I knock my router off-line once or twice a month by
messing with power cables nearby. The only way I find out is by
looking at the logs. I've never had to manually fsck any of my
routers except after electrical storms - and only then after moving
the disk to a non-smoking chassis.

Physical access to a console by a trusted person or remote console
access is required. Not for any failings of OpenBSD in particular but for
the guaranteed perversity of electronic devices and unforseeable
acts of nature and man messing up the local environment.

You will [should] access the system twice a year to install the latest
release.

[ insert standard disclaimers here ]

Geoff Steckel



Re: octeon: make syspatch copy /bsd to real boot directory

2019-05-20 Thread gwes
A few corrections to the previous diff, sorry
Index: Makefile.octeon
===
RCS file: /cvs/src/sys/arch/octeon/conf/Makefile.octeon,v
retrieving revision 1.49
diff -u -p -r1.49 Makefile.octeon
--- Makefile.octeon 9 Feb 2018 03:59:15 -   1.49
+++ Makefile.octeon 20 May 2019 21:50:55 -
@@ -28,6 +28,14 @@ S!=  cd ../../../..; pwd
 _machdir?= $S/arch/${_mach}
 _archdir?= $S/arch/${_arch}
 
+# as long as there is no boot program which loads from ufs,
+# we must copy any kernels to the FAT filesystem where the
+# manufacturer provided finds it
+# this script assumes a valid partition and filesystem exist
+
+BOOTROOTDIR?=  /mnt/
+BOOTROOTDEV?=  /dev/sd0i
+
 INCLUDES=  -nostdinc -I$S -I${.OBJDIR} -I$S/arch
 CPPFLAGS=  ${INCLUDES} ${IDENT} ${PARAM} -D_KERNEL -D__${_mach}__ -MD -MP
 CWARNFLAGS=-Werror -Wall -Wimplicit-function-declaration \
@@ -139,6 +147,7 @@ vers.o: ${SYSTEM_DEP:Ngap.o}
 clean:
rm -f *bsd *bsd.gdb *.[dio] [a-z]*.s assym.* \
gap.link ld.script lorder makegap.sh param.c
+   [[ -z ${BOOTROOTDIR} ]] || umount -f ${BOOTROOtDEV}
 
 cleandir: clean
rm -f Makefile *.h ioconf.c options machine ${_mach} vers.c
@@ -154,14 +163,28 @@ context.o cp0access.o exception.o locore
 lcore_access.o lcore_ddb.o lcore_float.o tlbhandler.o: assym.h
 mips64r2.o: assym.h
 
-hardlink-obsd:
+hardlink-obsd: mount_bootroot
[[ ! -f /bsd ]] || cmp -s bsd /bsd || ln -f /bsd /obsd
+   [[ -z ${BOOTROOTDIR} && ! -f ${BOOTROOTDIR}/bsd ]] || \
+   cmp -s bsd ${BOOTROOTDIR}/bsd || \
+   ln -f ${BOOTROOTDIR}/bsd ${BOOTROOTDIR}/obsd
 
-newinstall:
+newinstall:mount_bootroot
umask 077 && cp bsd /nbsd && mv /nbsd /bsd && \
sha256 -h /var/db/kernel.SHA256 /bsd
+   [[ -z ${BOOTROOTDIR} ]] || \
+   cp /bsd ${BOOTROOTDIR}/nbsd && \
+   mv ${BOOTROOTDIR}/nbsd ${BOOTROOTDIR}/bsd
+
+mount_bootroot:
+   [[ -z ${BOOTROOTDIR} ]] || \
+   mount ${BOOTROOTDEV} /mnt
+
+umount_bootroot:
+   [[ -z ${BOOTROOTDIR} ]] || \
+   umount -f ${BOOTROOTDEV}
 
-install: update-link hardlink-obsd newinstall
+install: update-link hardlink-obsd newinstall umount_bootroot
 
 # pull in the dependency information
 .ifnmake clean



octeon: make syspatch copy /bsd to real boot directory

2019-05-20 Thread gwes
Install on octeon correctly copies /bsd to the MSDOS filesystem
where the manufacturer's boot program finds it

It appears that syspatch doesn't. If so, fixing this is important
because as it stands security patches won't actually be installed
in running systems.

This is an * untested * * conceptual * patch. My only octeon
system is my firewall. Another can be purchased if consensus is
that this should be tested here.

I haven't looked for update-link yet

geoff steckel

Index: Makefile.octeon
===
RCS file: /cvs/src/sys/arch/octeon/conf/Makefile.octeon,v
retrieving revision 1.49
diff -u -p -r1.49 Makefile.octeon
--- Makefile.octeon 9 Feb 2018 03:59:15 -   1.49
+++ Makefile.octeon 20 May 2019 17:40:17 -
@@ -28,6 +28,14 @@ S!=  cd ../../../..; pwd
 _machdir?= $S/arch/${_mach}
 _archdir?= $S/arch/${_arch}
 
+# as long as there is no boot program which loads from ufs,
+# we must copy any kernels to the FAT filesystem where the
+# manufacturer provided finds it
+# this script assumes a valid partition and filesystem exist
+
+BOOTROOTDIR?=  /mnt/
+BOOTROOTDEV?=  /dev/sd0i
+
 INCLUDES=  -nostdinc -I$S -I${.OBJDIR} -I$S/arch
 CPPFLAGS=  ${INCLUDES} ${IDENT} ${PARAM} -D_KERNEL -D__${_mach}__ -MD -MP
 CWARNFLAGS=-Werror -Wall -Wimplicit-function-declaration \
@@ -154,12 +162,22 @@ context.o cp0access.o exception.o locore
 lcore_access.o lcore_ddb.o lcore_float.o tlbhandler.o: assym.h
 mips64r2.o: assym.h
 
-hardlink-obsd:
+hardlink-obsd: mount_bootroot
[[ ! -f /bsd ]] || cmp -s bsd /bsd || ln -f /bsd /obsd
+   [[ -z $BOOTROOTDIR && ! -f $BOOTROOTDIR/bsd ]] || \
+   cmp -s bsd $BOOTROOTDIR/bsd || \
+   ln -f $BOOTROOTDIR/bsd $BOOTROOTDIR/obsd
 
-newinstall:
+newinstall:mount_bootroot
umask 077 && cp bsd /nbsd && mv /nbsd /bsd && \
sha256 -h /var/db/kernel.SHA256 /bsd
+   [[ -z $BOOTROOTDIR ]] || \
+   cp /bsd $BOOTROOTDIR/nbsd && \
+   mv $BOOTROOTDIR/nbsd $BOOTROOTDIR/bsd
+
+mount_bootroot:
+   [[ -z $BOOTROOTDIR ]] || \
+   mount -t $BOOTROOTDEV /mnt
 
 install: update-link hardlink-obsd newinstall
 



ffs undelete was: Re: single user question

2019-05-17 Thread gwes




On 5/17/19 2:34 PM, Nathan Hartman wrote:

On Fri, May 17, 2019 at 12:28 PM ropers  wrote:


In the history of the (Berkeley) Fast File System, has there ever been
an attempt to implement DOS-like undelete for FFS/UFS?

Maybe that could work for "normal delete" while making available a separate
"secure delete" that cannot be un-deleted and furthermore overwrites the
deleted data with random garbage. Administrators could optionally force the
secure overwrite delete.


I haven't looked at e.g. zfs in a long time.

A journal-like system which held the deleted/overwritten files
or a system of renaming wouldn't be *that* hard to instantiate
There are some problems:
(a) denial of service by writing and deleting huge [numbers, size] files.
(b) retention policy - under what conditions does the system
  guarantee existence of backup files?
(c) versioning - If I create & delete 'a' six times, how many copies are 
held.

(d) cost of undelete operation - it's not clear how to make
 that efficient.

I'm sure people can find more.

A test version substituting a new open(2) and unlink(2) in libc would be 
easy to make.


geoff steckel



Re: single user question

2019-05-16 Thread gwes




On 5/16/19 9:05 PM, James Huddle wrote:

First of all, I must say that it is with genuine gratitude that I read your
responses!

Mov
Probably the same reason that you would say "...I might trigger other
people to say some rude things..."  Often I feel that by merely stating
my opinion, here, I have opened the door to the proverbial darkroom.
Sorry!  That, and a multi-user system has been the heart and cornerstone
of Unix & co. for MILLENNIA.  That's fine.  But my laptop is not a 1985 VAX.
I just think that pushing the idea forward of using the most popular
multiuser OS in history - in single-user mode - might meet with a little
friction.


I think this is where you are fatally confused.

2) Also, what is a "user"?

Good question.  I am a user.  Someone who has hacked into my multi-user
system as a different user is a user.  And apparently, so is the cups
daemon?

You are correct on the surface and very misled as to the underlying concept.

In Unixish parlance,

"single user" = a system running with no resource restrictions
   and all but the absolutely essential services and processes stopped

"multi user" = a system operating with normal division of privilege and
  resources and all normal services available.

A system in "single user" state is normally only accessed by one
person, for a short time, to perform vital maintenance.
In that state a mistake can destroy the system - even to make
the system unrecoverable, a "brick"

A "user" in the context of [multiprocess] computing is a label for
a set of privileges [access, execute, etc.] & resources [storage, etc.]
It can be assigned to a person, a functionality, a condition, or many
other concepts. This restriction is vital for normal operation.

Why?
No program can be guaranteed to be perfect, and no person can be guaranteed
to never make a mistake. By restricting what can be done by a process or
a person in a given situation, the consequences of an error, a bug,
or a deliberate intrusion can be minimized.

In order to be useful, your laptop must perform many tasks invisibly and
concurrently. To promote reliable operation, each task [process, thread, 
etc]

is assigned resources and privileges. We hope that the set assigned to
each is sufficient but does not allow destruction [overwriting, renaming,
etc.] of resources necessary to other tasks or exposure of secrets.

The CUPS daemon can delete files. Do you want it to be able to delete
ANY file? It is given an identity [set of resources and privileges] to
print and otherwise manage ONLY the files YOU give it.

You can delete files. Do you want to be able to accidentally delete ANY
file? Or do you want to be able to write-protect some of them?

A prime example of a "single user" system according to your definition
is MSDOS. No restrictions on anything. How reliable is/was it?

A server may ordinarily have no people sitting at a console connected
to the machine. It may have hundreds or thousands of different identities
requesting service, none of which should be able to affect any other.
So it, by custom parlance, has hundreds of users.

You probably don't want to run your laptop in Unixish "single user"
since most of the services (graphics, networking, Bluetooth, etc.)
are not available and a simple typing error can erase every file on
the system.

I hope this brings you to an understanding of what the convention
of "single user" and "multi user" mean and why running, for instance,
your laptop in "single user" mode would make it useless for you.

geoff steckel



Re: Good options for SAS HBA or SATA expansion cards?

2019-04-14 Thread gwes

On 04/14/19 15:25, John Long wrote:

On Sun, 14 Apr 2019 14:53:34 -0400
gwes  wrote:

  

On 2019-04-11, John Long  wrote:

I have a Dell server that was advertised to support 4x3.5 +
2x2.5 drives but when I popped it open I found there are only
4 SATA ports on the motherboard total. So of the 6 claimed
drives, I can actually only install 3 drives because the stock
DVD drive consumes a mobo port.

Yeah T30 PowerEdge. The local shop has the card Dell
recommended, but I'm not sure I trust them since it's unlikely
Dell tests anything but a thousand variants of Windows and
*maybe* RedHat.

/jl
  

[ lots of good stuff snipped ]
I'll second the LSI Logic/Avago/Broadcom? SAS/SATA controllers.
They run as many disks as I want at full speed. As previously
mentioned they can be quite inexpensive if you buy one relabelled
as (for instance) an IBM card.

They do need to be flashed to a recent firmware version.
Older firmware versions limit themselves to 32-bit block numbers.

Can I flash one of these cards without installing it in a Windows box?
Because I don't have one ;)

They can be flashed from Linux. I think a USB drive formatted right with the
appropriate files & running Linux from a cd-rom would suffice.
I did something like that.

Geoff Steckel



Re: Good options for SAS HBA or SATA expansion cards?

2019-04-14 Thread gwes



On 2019-04-11, John Long  wrote:

I have a Dell server that was advertised to support 4x3.5 + 2x2.5
drives but when I popped it open I found there are only 4 SATA
ports on the motherboard total. So of the 6 claimed drives, I can
actually only install 3 drives because the stock DVD drive
consumes a mobo port.

Yeah T30 PowerEdge. The local shop has the card Dell recommended,
but I'm not sure I trust them since it's unlikely Dell tests
anything but a thousand variants of Windows and *maybe* RedHat.

/jl


[ lots of good stuff snipped ]
I'll second the LSI Logic/Avago/Broadcom? SAS/SATA controllers.
They run as many disks as I want at full speed. As previously
mentioned they can be quite inexpensive if you buy one relabelled
as (for instance) an IBM card.

They do need to be flashed to a recent firmware version.
Older firmware versions limit themselves to 32-bit block numbers.

A probably obvious note:
PC type boxes have unfortunate limitations unless one
is prepared to spend $$$ for high end or Xeon/Opteron.
PCI lanes and memory subsystems can saturate :-(

Geoff Steckel



Re: compared filesystem performance, was Re: 10GBit network performance on OpenBSD 6.4

2019-04-08 Thread gwes




On 04/08/19 19:29, Chris Cappuccio wrote:

gwes [g...@oat.com] wrote:

What is the rated transfer rate of the SSD you're using to test?
SATA 3 wire speed is 6G/sec and realistically 500MB/sec raw rate
is near the top.

Anything over that is an artefact probably from a cache somewhere.


He's using NVMe with its own DRAM cache, which should perform higly. There
is a limiter somewhere, it seems.


That doesn't answer the question: if you say
dd if=/dev/zero of=/dev/sda (linux) /dev/rsd0c (bsd) bs=64k count=100
what transfer rate is reported

That number represents the maximum possible long-term filesystem
performance on that drive.

There are other non-filesystem overheads which have to be excluded
before you can be sure that the differences are truly the filesystem
code and algorithms without cache differences.



compared filesystem performance, was Re: 10GBit network performance on OpenBSD 6.4

2019-04-08 Thread gwes




On 04/08/19 17:46, Anatoli wrote:
That was with Samsung 960 EVO U.2 (PCIe) on i7-8550u with 32GB RAM. 
OpenBSD read/write was around 220-240MB/s (with FS encryption), Linux 
without FS cache about 2.6-2.8GB/s and with cache over 3.5GB/s.


I don't have a dmesg right now as I installed Gentoo on top and just 
saved a printscreen of the tests (below), but I can reinstall OpenBSD 
and make more specific tests if anybody is interested (I do am 
interested in a reasonable OpenBSD performance, but I thought 12x 
slower and no cache to improve things when I/O lags wasn't that strange).


If you can suggest some specific tests to analyze the cause (i.e. 
filesystem, hardware issues, scheduling, etc.), please let me know.




*From:* Chris Cappuccio 
*Sent:* Monday, April 08, 2019 16:28
*To:* Anatoli 
*Cc:* Misc 
*Subject:* Re: 10GBit network performance on OpenBSD 6.4

Anatoli [m...@anatoli.ws] wrote:

I've seen extremely slow HDD performance in OpenBSD, like 12x slower 
than on
Linux, also no filesystem cache, so depending on your HDD with scp 
you may

be hitting the max throughput for the FS, not the network.

12x slower? That's insane. What are you talking about? USB HDD? USB 
Flash?

SATA? Driver? You should submit a bug report with lots of details.

Chris




A quick test on a slow laptop running linux shows
  dd if=/dev/zero of=a bs=64k count=2
runs 1.3 GB/sec. The physical disk transfer rate is 80 MB/sec max.
Linux caches very aggressively.

What is the rated transfer rate of the SSD you're using to test?
SATA 3 wire speed is 6G/sec and realistically 500MB/sec raw rate
is near the top.

Anything over that is an artefact probably from a cache somewhere.

I suspect that if you tried to write more data than physical memory
can hold the transfer rate would slow to something under the
disk or channel rate.

OpenBSD saves a great deal less in its cache. This slows repetitive
accesses to large data sets a painful amount. That's a separate problem
which I'd like to look at but don't have the time to write the tools
to do it.



shrinkfs(8) and movefs(8)?

2019-01-04 Thread gwes

The approved method to reararrange filesystems
is to dump(8) and restore(8) or equivalent.
Sometimes this is impossible or extremely difficult - think
hosted or other systems without any accessible additional mass storage.

If a shrinkfs(8) and movefs(8) existed, would anybody use them?

shrinkfs would be implemented such that if it were interrupted
at any point fsck would result in an intact filesystem.
The tradeoff of security vs. speed requires that compacting
a large almost-full fs could take a long time.

movefs of a filesystem over itself, if interrupted, could result
in corruption which would require another program to recover
It still would be safer than attempting do this manually.

geoff steckel



Re: USB stick recovery after dd with miniroot64.fs

2019-01-04 Thread gwes

On 01/04/19 10:04, Mihai Popescu wrote

sysutils/testdisk is very good.
No success with that. It looks like all partitioning information has 
vanished. I don't know partitioning at bit level so I cannot try more. 
If anyone succeded with this kind of overwrite, or if there is any 
chance to recover something, please write it here.


Wiping the first 1mb of almost any file system can completely destroy it.
FAT and (to a great extent) NTFS keep all the vital information
at the beginning.
Unix-derived UFS, FFS, ext2,3,4 file systems spread most of the information
enough to allow recovery of most of the files after such a disaster.

Some triage - sorry but I've lost the beginning of this thread.

Since most sticks are preformatted as FAT32 that's what I'm assuming

The root directory and the information linking the
blocks of each file together are in the first part of the disk.
In the best case a sufficiently smart
program could find the -first- data block of files which were -not- in
the root directory. I don't know of any such program but there may be one.

If the file(s) are not editable text, assume it's impossible in most cases.

If any file on the stick has been modified or deleted since it was new
and any new data written recovery is much harder since data are now
scattered and interposed.
Assume it's impossible unless you have very great need and a lot of time.

If there was only one file on the stick, it was a text file and it was a
new stick, everything but the first 100-500k or so might be recovered by
   dd if=stick of=recoveredjunk \
count=
and then editing recoveredjunk

If there were multiple text files, add all their estimated sizes together
If it's all text you *might* be able to reassemble them by editing the file.
 
Good luck!

geoff steckel



Re: recommended h/w for fanless audio-out?

2018-11-12 Thread gwes




On 11/12/18 07:30, Colin Bortner wrote:

Hello,

I’d like to use OpenBSD to build a MIDI synthesizer using SoundFonts, as the 
OpenBSD MIDI and audio subsystems are remarkably understandable and sane, 
compared to everything else out there today.


I’ve heard a fair bit here about USB audio not working very well, or at all, in 
-stable right now. I’m unsure if this only applies to XHCI ports or not?

As far as I know isochronous transfer mode required by USB audio class devices 
is not supported by the xhci driver. I believe a few people are poking at this, 
but that doesn't mean there will be support anytime soon. However - not all is 
lost...


For the NUC, USB 3.0 can be disabled in the bios, forcing the ehci driver, and 
allowing me to use the USB DAC without issues.

Alas, the UCA-202 hangs...

uaudio0 at uhub0 port 3 configuration 1 interface 0 "Burr-Brown from TI USB Audio 
CODEC" rev 1.10/1.00 addr 7
uaudio0: audio rev 1.00, 2 mixer controls
audio1 at uaudio0
uhidev3 at uhub0 port 3 configuration 1 interface 3 "Burr-Brown from TI USB Audio 
CODEC" rev 1.10/1.00 addr 7
uhidev3: iclass 3/0
uhid0 at uhidev3: input=1, output=0, feature=0

uaudio_chan_open: error creating pipe: err=INVAL endpt=0x02
audio1: failed to start playback

Disabling xhci allows it to run.

I'd love to fix isochronous mode in my Infinite Spare Time...
once I learn the USB stack & hang my machine 100+ times,
submit the code and get very embarrassed, try again...
Very likely many of the developers who are much more familiar with
the USB stack would like to fix it in their Infinite Spare Time.

Geoff Steckel



Re: make(1) and multiple outputs

2018-09-02 Thread gwes




On 08/31/18 03:23, Kristaps Dzonsons wrote:

Short: is there a way to manage multiple outputs from a single command
with OpenBSD's make(1)?

Longer story.  I have a site that generates a few hundred articles using
sblg(1).  Each output article is indexNNN.html, which depends upon every
input indexNNN.xml.  So a change to any indexNNN.xml must result in
rebuilding all indexNNN.html using a single command.

In GNU make, I can use the pattern substring match to effect this:

all: index001.html index002.html

index001%html index002%html: index001.xml index002.xml
sblg -L index001.xml index002.xml

But obviously that's GNU-only.  It is, as a fallback, possible to have
sblg(1) create one output per input and play nice with make(1):

index001.html: index001.xml index002.xml
sblg -C index001.xml index001.xml index002.xml

But with hundreds of articles (each of which depends upon parsing
hundreds of articles), those are a lot of wasted cycles.

I currently just use the GNU make, but I'd rather use only stock
components on the server.  Any thoughts?


Your example and request aren't clear to me.

Do you mean you have index000.xml, index001.xml, ... index999.xml
  and you want index000.html, index001.html, ... index999.html
such that if you touch *any* .xml you want to compile *all* the xml
to produce *all* the html?

You can introduce a set of proxy files to represent the dates
of the xml files.

.suffixes .xml .dummy

.xml.dummy:
         *.xml
        touch $@

Once you have the dummy files to represent individual dates,
you can use various make variables ($<, etc), pattern matching
and substitution functions to produce filenames.

If this isn't what you want I can't understand your question.
The gnu-style makefile doesn't make sense to me and I've used
the % feature in gmake.

If I were doing your project I would work *very* hard
to work around the many-to-many dependency.

Geoff Steckel



ipv6 ndp proxy ip6addr/128?

2018-03-07 Thread gwes

I noticed in ndp.c code to add a netmask to
an ipv6 address proxy was #if 0

Is this a philosophical "proxying more than 1
IPv6 address is wrong", lack of time, lack of interest?

My application is bridging a single IPv6 subnet
over openvpn such that    xx::23:34:56
        | <-> [client1]
gateway <-> extif[firewall] <-vpn-> | xx::3a:bc:de
xx::1       xx::2       | <-> [client2]

where xx is the same in all places. I have a /64 from
my virtual host provider.

As ndp proxying works now,

if client's address was fixed, ndp proxy at firewall
would work

if client's address isn't fixed (e.g. privacy) then
a new ndp proxy would have to be put in place
potentially overflowing tables in firewall.

I haven't looked at ip6 routing in the kernel...

If some knowledgeable person sees this, is there
a simple answer?

thanks,
Geoff Steckel



Re: Hellos from the Lands of Norway.

2017-12-08 Thread gwes

On 12/07/17 07:31, Ywe Cærlyn wrote:
I saw AMDs "semi-custom" CPU email form and told them that I wanted a 
CPU, that is clockspeed oriented, not cores (might aswell be singlecore 
with high HZ), that could be using several instruction macros (combining 
two or three), for max virtual clockspeed, and an optimizing compiler 
for this. And wondered if an additional poweroff mode could be added to 
the binary stream of 1 0, so that bitwise i/o and cpu scheduling could 
be done.


If one could get the virtual clockspeed up to 12ghz, I think no regular 
user would ever use more than a single core. And it´d be a megahit.


Fixing all inefficiency hardware wise. Philosophically aswell.

Peaceful Salutations.


CPU clock speed != performance.

Factor in:
main memory: latency, bus width, and access/cycle time.
caches: levels, speeds, sizes, widths
CPU access patterns interacting with the above
clocks per instruction: average, best case, worst case
cost or even feasibility of super high CPU clocks
propagation time of signals across chips

A very fast CPU clock on a CPU with very low clocks-per-instruction
a small die and a huge memory matching speed == the RISC ideal

Even RISC with floating point hardware, for instance, often takes
many cycles.

Adding cores is often seen as the best way of increasing
>system< performance significantly at the lowest cost.

geoff steckel





Re: CUPS and AVAHI (bloatware)

2017-10-29 Thread gwes

The last time AVAHI got installed on one of my systems
the installer started it immediately.
Avahi then proceeded to scribble on that system's
network configuration and confuse other systems on
that subnet.

I would assert that Avahi should be either (a)
not automatically started when installed or (b)
split.

I am not asking for a general split. This one
package causes a lot of confusion if the daemons
are started. A simple "do you want to enable the
daemons?" would be good enough.

Is this worth considering?

thanks
geoff steckel



Re: CUPS and AVAHI (bloatware)

2017-10-29 Thread gwes

On 10/26/17 07:24, Rupert Gallagher wrote:

It is well known that cups does not need avahi.

Avahi is an option, it requires dbus, which requires X11. If you have a server 
with limited resources and without X11,  you cannot install the present cups 
package.

Please remove cups's dependency on avahi.



Check the FAQs for how to build ports.
It's possible to build a version of cups without avahi.
You would need to do it on a moderately capable system:
any recent laptop or desktop system would suffice.
It must have the same type of CPU as your target system.

I'm not sure I have all the details correct, but this is
what should work.

After setting up your system to build ports from
the instructions in the FAQ:

go to /usr/ports/print/cups and edit Makefile to
remove all mentions of avahi and mdns

make print-build-depends > list_of_dependencies

go through that list and install all of them using
pkg_add. This saves considerable time since the
make will build and install all missing dependencies.

This is the crucial step:
make CONFIGURE_ARGS='--disable-avahi --disable-mdns'

you may have to use 'doas' for this step:
make package
this will create a cups package which can be installed
with pkg_add on the system of your choice.
It -will- install dbus. Removing that is harder.

When this works you should probably work with the ports
group to make this version available. They may not accept
it because compiling another version of cups on their
build systems would take too long. In any case posting
a succinct list of the changes you had to make might
be interesting to some people.


geoff steckel



Re: 5-button wheeled mouse and X

2017-10-29 Thread gwes

On 10/25/17 07:20, Cág wrote:

Natasha Kerensikova wrote:


it started as a bug report: it have a 5-button mouse with a wheel, even
though I don't use much the buttons 4 and 5 (I think only for previous
and next in firefox history). I recently switched to OpenBSD, and I was
surprised to find these buttons cause scrolling, like the wheel. If this
behavior is intended, the rest of this e-mail is moot.


This isn't a bug apparently. Are the extra buttons recognised by xev? If
they are, remap them with xmodmap(1). If they aren't, well, then it should
be done by xf86-input-evdev or libinput. There's been some work in FreeBSD,
and the drivers are in ports, but OpenBSD doesn't have them.


I use a 4-button Logitech trackball mouse. It required remapping in
xmodmap to make it work.
Yes, xev showed enough information to see "button up" and "button down"
events to use as input to xmodmap.

geoff steckel



smtpd vs. uw-imap locking

2017-07-30 Thread gwes

smtpd locks user mailboxes in /var/mail using lockspool(1)
uw-imapd locks using its own dotfile locker mlock(not the syscall)

Before I go into uw-imapd and do some nasty additions and
if()s:
does anyone else care?
does anyone else use uw-imap?
has anyone attacked this problem?
is there anywhere else to ask these questions?

switch(answers) {
*, *, *, yes: go there
no, *, *, no: go do ugly coding in the dark
*, *, yes, *: may I please see what you did?
yes, yes, no, *: please get in touch
default: go do ugly coding
}

I'm not interested in any other server unless it is
at least as lightweight as uw-imap. Courier, Cyrus,
etc. are obese by comparison.

thanks
Geoff Steckel



usb RTC, was Re: octeon port, ubiquity edgerouter

2017-07-22 Thread gwes

On 07/22/17 12:10, Theo de Raadt wrote:


I'd really like if someone could find a USB RTC clock, which is a viable
affordable product which we can then create good support for.  I've searched
and found a few prototypes and 'licence key' products, but nothing readily
available which we could support & encourage as a solution for the RTC
problem.


What kind of packaging, fit & finish, and price would be acceptable?

For example:
A commodity microcontroller on a tiny PCB, a Dallas Semi RTC on
another tiny commodity PCB and a lithium coin cell in a
4cm x 6cm x 2cm commodity box could be professionally assembled
and sold in unit quantities in a month for $50.

Accuracy limit is the 32Khz crystal. Temperature compensation
to match the standard quartz curves in the ucode. Burn-in
and trimming to a few PPM would add maybe $5.

The same parts on a single PCB in a slightly smaller box might take
three months, an expected sale quantity of 50, and sell for $30.

USB stick size, etc. - I'm sure you can extrapolate from there.

Geoff Steckel



Re: Can I bind USB/other interface/device number (e.g. cdceX) to particular MAC, USB serial number or the like?

2017-06-05 Thread gwes

On 06/04/17 19:09, Kevin Chadwick wrote:


fxp0,1,2 are in order of pci slot. I assume usbs are the same after
boot so anyone who unplugs and plugs devices and doesn't check the
outcome on critical hardware deserves what they get. Also having
critical hardware that can be physically damaged is also asking for
trouble, so I cannot see an issue at all??


I've seen more than one instance where a critical system has
to add an interface on the fly to either (a) replace a failing interface
while maintaining traffic on all the other interfaces or (b)
add an interface when all installed ports are busy.

Saying "you shouldn't do that" or "why would you want to
do that" shows lack of real-world experience.

That said, I agree that the Linux solution is bad.
OpenBSD reports hardware during boot and new hardware
with kernel messages. Using that information to configure
the appropriate daemons and utilities is the right solution.

Geoff Steckel



mousedrv(4x) man vs environment under X

2017-04-17 Thread gwes

The mousedrv(4x) man page says

  Option "ButtonMapping" "N1 N2 [...]"
  Specifies how physical mouse buttons are mapped to logical
  buttons.  Physical button 1 is mapped to logical button N1,
  physical button 2 to N2, and so forth.  This enables the use of
  physical buttons that are obscured by ZAxisMapping.
 Default: "1 2 3 8 9 10 ...".

The code in mouse.c for that initialization is identical in
FreeBSD and OpenBSD. Under FreeBSD "xinput get-button-map" shows
1 2 3 8 9 10

Under OpenBSD somewhere between the mousedrv code and the environment 
when X is started leaves the mapping "1 2 3 4 5 6..."

This is on a 6.0 freshly upgraded to 6.1 with an empty
xorg.conf.d, no .xinitrc

This behavior, of course, perplexes the user.
Trying to xmodmap physical button 4 (supposedly now button 8)
to a middle button fails. xmodmap physical 4 to middle button
does work.

I -think- the FreeBSD behavior is correct, since the
usage of button events 4-7 is very murky given the universe
of wildly different pointing devices.

Is there a configuration file somewhere I overlooked?

thanks
Geoff Steckel



Re: DDB "boot sync" or "boot dump" hangs

2017-04-16 Thread gwes

On 04/16/17 02:37, Philip Guenther wrote:

On Sun, 12 Mar 2017, gwes wrote:

I'm trying to debug the following panic.
I can't get a crash dump.
At the DDB prompt, either "boot sync" or "boot dump"
the system prints "Syncing disks: 2" and nothing more.


So you asked it to sync the disks and that was unable complete: if it did
it would have said "done".  Why couldn't it complete?  Hard to say;
perhaps one of the other CPUs was holding a lock when ddb stopped it, or
maybe you have a filesystem mounted from the involved USB device.

(What's the '2'?  It's a count of buffers that couldn't be written out;
the kernel normally pauses a moment to give disks time to do their stuff
and thus tries flushing everything again, leading to a string of numbers
with pauses between them, ala "12 12 12 8 7 6 3 1 done".  That behavior
goes back a *long* ways; I recall watching that count down on SunOS 4.1
systems back in the early '90s.)

Instead of doing "boot dump", you can try "boot crash" which skips the
file system sync before dumping the kernel crash.  If you can reliably
reproduce the problem then I would suggest remounting as many filesytem
read-only as possible before triggering the panic to minimize the possible
filesystem damage.


Philip Guenther



Thanks very much for looking at this.

1) fresh 6.1 install on two machines

2) machine "B" still hangs after panic doing "boot crash".
   It does have a LSI Logic mpii installed which I thought
   might cause problems while sync()ing.

3) machine "B" still panics accessing a USB device programmer.
   This machine has an Intel EHCI in a 1037U cpu.
   I posted the stack trace earlier.

4) machine "A" running the same program with the same usb device
doesn't crash. Instead, it seems to be in an interruptible retry
loop.  This machine has the Braswell xHCI in a N3050 cpu.

For now, I think I'll chase the USB bug because it prevents
me from using the device programmer. The bug appears to be
in an error path peculiar to the device and the avrdude program
trying to access it. kgdb willing I'll have more information soonish.

I don't expect anyone to debug either problem from this information.
If anyone has seen something like it and will relay any clues I'd 
appreciate it.

If someone has patches which might fix either case I'll gladly try them.

Geoff Steckel



Machine "A":
OpenBSD 6.1 (GENERIC.MP) #20: Sat Apr  1 13:45:56 MDT 2017
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8023588864 (7651MB)
avail mem = 7775739904 (7415MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xebef0 (17 entries)
bios0: vendor American Megatrends Inc. version "P1.20" date 06/02/2015
bios0: ASRock N3050B-ITX
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT AAFT MCFG SSDT SSDT SSDT UEFI 
LPIT CSRT
acpi0: wakeup devices UAR1(S4) UAR2(S4) XHC1(S4) HDEF(S4) PXSX(S4) 
RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PWRB(S4)

acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) CPU N3050 @ 1.60GHz, 1600.38 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT

cpu0: 1MB 64b/line 16-way L2 cache
cpu0: TSC frequency 1600385000 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 79MHz
cpu0: mwait min=64, max=64, C-substates=0.2.0.0.0.0.3.3, IBE
cpu1 at mainbus0: apid 4 (application processor)
cpu1: Intel(R) Celeron(R) CPU N3050 @ 1.60GHz, 1599.94 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT

cpu1: 1MB 64b/line 16-way L2 cache
cpu1: smt 0, core 2, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 115 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (RP01)
acpiprt2 at acpi0: bus 2 (RP02)
acpiprt3 at acpi0: bus -1 (RP03)
acpiprt4 at acpi0: bus -1 (RP04)
acpiec0 at acpi0: not present
acpicpu0 at acpi0
C2: state 6: substate 8 >= num 3
C3: state 7: substate 4 >= num 3: C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0
C2: state 6: substate 8 >= num 3
C3: state 7: substate 4 >= num 3: C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: CLK0, resource for CAMD
acpipwrres1 at acpi0

DDB "boot sync" or "boot dump" hangs

2017-03-12 Thread gwes

I'm trying to debug the following panic.
I can't get a crash dump.
At the DDB prompt, either "boot sync" or "boot dump"
the system prints "Syncing disks: 2" and nothing more.

I've tried:
  removing all disks and/or controllers other than
  the disk holding the root

  removing physical memory so it is 4G < swap space (5G)

  unplugging the offending USB device after the panic

  running 6.0 release, 6.0 stable, and the snapshot below

The panic and hang are identical for all cases.

Is there something obvious about my configuration which is
causing this?
Alternatively, is the panic known? It's 100% reproduceable.
I'll submit bug reports for both if that's the best approach.
I wanted to try to debug this myself to possibly get a fix
quickly.

thanks
Geoff Steckel

panic: ehci_device_clear_toggle: queue active
Stopped at  Debugger+0x9:   leave   
TIDPIDUID PRFLAGS PFLAGS  CPU COMMAND 

 * 2171   2171  0 0x3  00 avrdude 


Debugger() at Debugger+0x9
panic() at panic+0xfe
ehci_device_clear_toggle() at ehci_device_clear_toggle+0x2b
usbd_clear_endpoint_stall() at usbd_clear_endpoint_stall+0x24
ugen_do_read() at ugen_do_read+0x4e6
ugenread() at ugenread+0x48
spec_read() at spec_read+0x2c5
VOP_READ() at VOP_READ+0x3f
vn_read() at vn_read+0xa1
dofilereadv() at dofilereadv+0x204
sys_read() at sys_read+0x89
syscall() at syscall+0x27b
--- syscall (number 3) ---
end trace frame: 0x0, count: 3
0x108592d58a1a:


Mar 10 20:26:39 store /bsd: OpenBSD 6.1-beta (GENERIC.MP) #224: Thu Mar 
9 18:50:15 MST 2017
Mar 10 20:26:39 store /bsd: 
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

Mar 10 20:26:39 store /bsd: real mem = 8466022400 (8073MB)
Mar 10 20:26:39 store /bsd: avail mem = 8204775424 (7824MB)
Mar 10 20:26:39 store /bsd: mpath0 at root
Mar 10 20:26:39 store /bsd: scsibus0 at mpath0: 256 targets
Mar 10 20:26:39 store /bsd: mainbus0 at root
Mar 10 20:26:39 store /bsd: bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb450 
(20 entries)
Mar 10 20:26:39 store /bsd: bios0: vendor American Megatrends Inc. 
version "4.6.5" date 06/05/2013

Mar 10 20:26:39 store /bsd: bios0: BIOSTAR Group NM70I-1037U
Mar 10 20:26:39 store /bsd: acpi0 at bios0: rev 2
Mar 10 20:26:39 store /bsd: acpi0: sleep states S0 S1 S4 S5
Mar 10 20:26:39 store /bsd: acpi0: tables DSDT FACP APIC FPDT MCFG HPET 
SSDT SSDT SSDT
Mar 10 20:26:39 store /bsd: acpi0: wakeup devices PS2K(S4) UAR1(S4) 
P0P1(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) 
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) [...]

Mar 10 20:26:39 store /bsd: acpitimer0 at acpi0: 3579545 Hz, 24 bits
Mar 10 20:26:39 store /bsd: acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
Mar 10 20:26:39 store /bsd: cpu0 at mainbus0: apid 0 (boot processor)
Mar 10 20:26:39 store /bsd: cpu0: Intel(R) Celeron(R) CPU 1037U @ 
1.80GHz, 1796.21 MHz
Mar 10 20:26:39 store /bsd: cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT

Mar 10 20:26:39 store /bsd: cpu0: 256KB 64b/line 8-way L2 cache
Mar 10 20:26:39 store /bsd: cpu0: TSC frequency 1796209080 Hz
Mar 10 20:26:39 store /bsd: cpu0: smt 0, core 0, package 0
Mar 10 20:26:39 store /bsd: mtrr: Pentium Pro MTRR support, 10 var 
ranges, 88 fixed ranges

Mar 10 20:26:39 store /bsd: cpu0: apic clock running at 99MHz
Mar 10 20:26:39 store /bsd: cpu0: mwait min=64, max=64, 
C-substates=0.2.1.1.2, IBE

Mar 10 20:26:39 store /bsd: cpu1 at mainbus0: apid 2 (application processor)
Mar 10 20:26:39 store /bsd: cpu1: Intel(R) Celeron(R) CPU 1037U @ 
1.80GHz, 1795.92 MHz
Mar 10 20:26:39 store /bsd: cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT

Mar 10 20:26:39 store /bsd: cpu1: 256KB 64b/line 8-way L2 cache
Mar 10 20:26:39 store /bsd: cpu1: smt 0, core 1, package 0
Mar 10 20:26:39 store /bsd: ioapic0 at mainbus0: apid 2 pa 0xfec0, 
version 20, 24 pins

Mar 10 20:26:39 store /bsd: acpimcfg0 at acpi0 addr 0xf800, bus 0-63
Mar 10 20:26:39 store /bsd: acpihpet0 at acpi0: 14318179 Hz
Mar 10 20:26:39 store /bsd: acpiprt0 at acpi0: bus 0 (PCI0)
Mar 10 20:26:39 store /bsd: acpiprt1 at acpi0: bus -1 (P0P1)
Mar 10 20:26:39 store /bsd: acpiprt2 at acpi0: bus 2 (RP01)
Mar 10 20:26:39 store /bsd: acpiprt3 at acpi0: bus 3 (RP02)
Mar 10 20:26:39 store /bsd: acpiprt4 at acpi0: bus -1 (RP03)
Mar 10 20:26:39 store /bsd: acpiprt5 at acpi0: bus -1 (RP04)
Mar 10 20:26:39 store /bsd: acpiprt6 at acpi0: bus -1 (RP05)
Mar 10 20:26:39 store /bsd: acpiprt7 at acpi0: bus -1 (RP06)
Mar 10 20:26:39 store /bsd: acpiprt8 at 

Re: lpr duplex printing

2017-01-10 Thread gwes
>I wonder if somebody could educate me on duplex printing with lpr
>command from the base

> What does actually happen with the document when I use switch -s2 in
> a2ps?

> Thanks
> Predrag

lpr is very minimal. It could have a filter added
to send a file through a2ps on its way to the physical printer.
You could wrap a one or two line shell script
around lpr to do the same thing.

a2ps generates postscript for the text input
It adds a flag to print duplex in that postscript.
The printer interprets that and does what you asked.

You can send a postscript command string to set the
printer into duplex mode and another to reset it.
I don't remember it offhand. It's one or two lines
to set a value into devicedict or systemdict.
If you look at the a2ps sources it's in there.

Geoff Steckel



Re: Hardware recommendations for compact 1U firewall

2016-12-16 Thread gwes

On 12/15/16 12:07, Ryan Freeman wrote:

On Thu, Dec 15, 2016 at 11:30:31AM +, Stuart Henderson wrote:

On 2016-12-15, Aaron Mason  wrote:

All

I'm looking for a 1U appliance that I can re-purpose into a firewall
using OpenBSD.  I've tried the near-free method by using an old Lacie
Ethernet Disk appliance I had lying around, but it turns out the
onboard SATA chipset is toast on this particular unit (it freezes at
CDBOOT when it detects hard drives and the BIOS freezes when I set it
to IDE mode with drives attached, plus it only has one onboard NIC and
one PCI slot, so I can't install another SATA card without removing
the other NIC I installed), so I'm looking for other options that fit
a limited budget.

The most important criteria are that it must be 1U and it must fit
within a 420mm (~16.5") space (for reasons I will explain below).  I
have a couple of Sun Netra X1s that meet the need, but I can't push
more than ~60mbps over the onboard FE ports and they run quite hot to
the point of causing kernel panics.


Can you get anything in your price range with a single NIC and USB?
The axe driver seems to work pretty well. I bought a USB GE nic
for under $30 US. It seems to work well on a USB extension cord.
That's what I use for my firewall machine. I haven't tried very hard
but I know it can transfer over 100mb/sec.

Geoff Steckel



gethost.. replacement?

2016-12-07 Thread gwes

The gethost* DNS query functions don't have a man page in 6.0.
What is the approved replacement for a user-written program?

I saw discussions about reworking the resolver but can't find
any definition of what a user should code.

Thanks
Geoff Steckel



long fat pipe problem

2016-11-28 Thread gwes

I've got 2 6.0 systems: one in BOS one in LA. Transit
time is about 100ms. There's adequate bandwidth end-to-end.

I can only get 1.5 - 2.5 MB/sec due to the long fat
pipe problem: the receiving system won't open the window
large enough to let the sender put enough bytes in the pipe.

It's not congestion or starvation on either end or in
the middle. I can open multiple streams and get at least
1 MB/sec more for each one opened.

Summarized and trimmed this is what tcpdump shows:

0.129225 BOS -> LA 1448 timestamp 1082831729 1595875102
0.129471 BOS -> LA 1448 timestamp 1082831729 1595875102
0.129721 BOS -> LA 1448 timestamp 1082831729 1595875102

0.133218 LA -> BOS win 17014 ts 1595875102 1082731729
0.133218 LA -> BOS win 16652 ts 1595875102 1082731729
0.133218 LA -> BOS win 16290 ts 1595875102 1082731729
0.133218 LA -> BOS win 15928 ts 1595875102 1082731729
0.133218 LA -> BOS win 17376 ts 1595875102 1082731729

0.133558 ... 0.135478 9 packets of 1448

0.136721 ... 0.136791 3 acks 18825, 21721, 21721

0.137081 ... 0.137220 4 packets of 1448

etc.

In the bad old days I'd crank tcp.recvspace up to 20
or so and get at least 5x bandwidth.

What can I tune? I'll gladly patch & recompile.
snd_sshthresh?

thanks
Geoff Steckel



Re: Why on earth would online voting be insecure?

2016-11-15 Thread gwes

On 11/15/2016 00:55, Joel Wirāmu Pauling wrote:

So yes, back to my original point. A Civic's blockchain, one that does not
rely on the integrity (or rather is resilient to) the system it runs on, or
the security of the transmission media ; as a platform for use in civic's -
needs to exist first.



Combining two systems entirely separate in concept, implementation,
and space increases the probability of a correct answer. Three
would be better. Using the electronic system as a supplement to
the traditional one could be good as long as it does not compromise
the virtues of the old system.

The blockchain starts after the votes are entered. Two physically
separate systems composed of entirely different CPUs and peripherals
at the voting place would be good.

You still haven't addressed the problems of privacy while casting
the vote.

I think that your concepts for the technical parts of the
system are good. You haven't addressed some serious problems
where your system can be subverted.

Suggesting weekly votes is a very bad idea. Search science
fiction, for instance, to see very plausible predictions
of voter burnout.

I think this is no longer a computer systems discussion.

Geoff Steckel



Re: Why on earth would online voting be insecure?

2016-11-14 Thread gwes

On 11/14/2016 22:19, Alan Corey wrote:

OK, it's relevant to OpenBSD because I wouldn't consider anything else
safe enough to run on the servers.  Not that I'm in a position to do
any of it.  The servers could even be run from custom official live
CDs so they were harder to tamper with, with maybe a RAM drive for
speed.

There seems to be a conflict between having anonymous votes and having
something similar to paper ballots that can be recounted.  So let
authentication, identification, etc. be handled by one machine and
stored in one database then the transaction is handed over to another
machine which stores the votes.  That could be something simple like a
tab-delimited file which could be counted by hand, one line per voter.
The file could be only writeable by the owner. The same person can't
vote twice because the first machine wouldn't allow them in a second
time.



How do you know if the voter is under duress or being watched?

Paper can last two thousand years. It's pretty easy to make
paper that can't be duplicated in any useful quantity.
Functionally indelible ink, too.

Using machines to assist voting is a good thing.
Physical objects are much more convincing and easier to secure.

Oh yes -- the magic ghost Intel has put in every processor
for years. With a secret key -- security by obscurity.
Disk drives can be secretly reprogrammed. Network interfaces
have microcode, too. The memory system is also vulnerable
to secret tampering. All of these are back doors which are
or could be in place.

Securing the system is far harder than securing a program
or group of programs.

Geoff Steckel



Re: How assign some logic to handle system-gone-totally-unresponsive events (if not else then to enable admin with differentiated failure tracking between userland and hardware failures)

2016-10-18 Thread gwes

On 10/17/2016 22:47, Tinker wrote:

[...]

If you have any thought about how make that happen feel free to share.

Anyhow in the absence of any such logic, just doing a
hardware reset is fine, it's just a bit constrained as
it comes without automated reporting that 
could be used to distinguish hardware/kernel issues 
from userland issues, which encourages hardware
replacement and userland software debugging beyond what's really 
necessary.


Tinker


The first option you should consider (if not already mentioned)
is a serial link from the console port to another system.
One could then construct some sort of periodic probe & response.
On failure, break into DDB and dump state, sync disks, and reboot.

One possibility would be to start a very small user process at
the highest prioritycommunicating with a process at normal
or slightly below priority.
The lower priority process would send a message every N seconds.
At T + (say) 10N the high priority process would kick off whatever
action you want.

Saving state to disk in a hung system is problematic.
Network communication is chancy.
Serial communication is ancient but effective.

Geoff Steckel



Re: axen(4) usb ethernet problems

2016-10-14 Thread gwes

On 10/14/2016 03:35, Mark Carroll wrote:

On 13 Oct 2016, Ilya Kaliman wrote:


I have a "Plugable USB 3.0 ethernet adapter" with ASIX AX88179
chipset. The device is successfully recognized by axen(4) driver but
behaves strangely. When I plug in the ethernet cable the ifconfig
axen0 status says active and the leds start blinking. But after a
second or two both leds turn off and status says: no carrier.
Re-plugging the cable have no effect. Re-plugging the adapter itself
brings it up again for a second or two.

The device itself seems to be fine as it works in other OSes without
problems. I suspect it has to do with OpenBSD driver.


I'm afraid that I can't offer any useful help but I can at least confirm
the problem: I also have one of these devices (actually, maybe a couple)
and see exactly the same issue with OpenBSD, at least with 5.9, I didn't
try since with 6.0. At the time I chatted to a competent-seeming vendor
guy and apparently they've seen the same problem at their end with
OpenBSD and have no idea what the issue is. So, at least I can say:
you're correct, it's probably indeed not just that you have a bad
adapter. I don't know if any developers might like to have one of these
mailed to them.

-- Mark


I have this:
Oct 15 00:01:57 river /bsd: axe0 at uhub1
Oct 15 00:01:57 river /bsd:  port 3 configuration 1 interface 0 "Belkin 
Components F5D5055" rev 2.00/0.01 addr 6

Oct 15 00:01:57 river /bsd: axe0: AX88178, address 00:22:75:d7:1c:6d
Oct 15 00:01:57 river /bsd: ukphy0 at axe0 phy 1: Generic IEEE 802.3u 
media interface, rev. 4: OUI 0x00a0bc, model 0x0001


which had similar symptoms under 5.8. It seems to be usable under 5.9

the FreeBSD driver definitely works. Porting it or adapting it without 
the data sheet looks difficult and comments in the FreeBSD driver say 
that there are undocumented problems. Haven't looked at Linux drivers. 
They have worked for several years.


One clue from looking at the drivers is that the physical interface 
number in the OpenBSD driver was fixed at 0 and it appears that other 
drivers searched for the active one.


geoff steckel



use of divert-packet and table insertion

2016-10-06 Thread gwes

I need to use dns blacklisting on incoming email. Spamd caused
a user revolt because of its unpredictable delay.

smtpd maintainers have more urgent projects than working
on filter-dnsbl.

What I'd like to do is:

in pf.conf

pass in on ingress from  to any port smtp
pass in on ingress from  to any port smtp \
divert-to [spamd-port]
pass in on ingress from  to any port smtp \
#insert proper action here = pass on or send to spamd
pass in from any to any port smtp divert-packet  \
no-state

in dnsbld:
bind to divert socket 
   parallel loop:
receive syn packet for smtp connection
initiate dnsbl lookup
good reply:  insert address into 
bad reply:   insert address into 
timeout: insert address into 
reinject syn packet

in dnsbld-cleaner:
maintain lists of expiry times and remove entries
from the pf tables appropriately

What I'd like to happen is that the first syn packet will go
to dnsbld. By inserting entries into pf tables, when the syn
is finally reinjected or retry syn packets arrive, they will
match a table thus creating a state
.
Subsequent packets of that connection wouldn't go to the divert
socket. dnsbld should only see syn packets, usually only one
if the dns lookup is quick.

What I don't want to do is interpose dnsbld for the entire
smtp connection.

Is this likely to work? My reading of the code suggests it
should but pf is pretty intricate. I don't know if the pf
rule optimizer would rearrange things detrimentally.

thanks
Geoff Steckel



Re: Industrial use of line printers, does/would your company/organization use them with our lpd?

2016-02-18 Thread gwes

On 02/18/2016 16:33, Chris Bennett wrote:

On Thu, Feb 18, 2016 at 04:10:06PM -0500, gwes wrote:
.
They don't do dynamic autoconfiguration.
In an industrial environment autoconfiguration can be very bad.
(examples like directing confidential output somewhere unexpected)

I haven't looked at the code from LPRng, but it provides options to use
a pool of printers for certain jobs to be sent to.


I think that case is rare but should be considered.


The only function I can think of that lpr doesn't have is
the capability to request a forms change and wait until
it has been done. That could be an entirely separate subsystem
invoked by lpr.

When you say forms change, are you talking about paper size/type
changing or something else?

Forms change can mean size, material, preprinted forms, ribbon,
type chain, etc.. pretty much anything beyond "change input tray".

One more function that I can think of is scheduled access
dept. A from 4:00am to 11:00am, dept. B from 11:01 to 14:00, etc.

I've been in many places where many wifi printers were wide open and in
several adjacent businesses.


Ouch!

The case for retiring lpr et al really depends on your use model.
One size fits all could be difficult.
 How much access and use control?
 How much initial setup?
 How much per-user setup?
 How many printers and per-printer setup?

As you mention, wifi printers are common. Without access control,
they short-circuit any administration. Then anyone with the
password can do anything. A piece of javascript and a browser
would give users as much control as is possible.

So... wired printers - again, if they are open on the net,
access control is difficult or impossible. A very few
I've heard of have per-IP access control.

Either of these two cases really are "submit" and "monitor
for done". The only central administration that's possible
is to recommend which printer someone is to use.

IMnotsoHO, lpr works fine for these cases. The user
selects which printer to use and then queues the job.

Automatic printer discovery could be a boon or a
disaster. Better reporting of printer errors
and daemons not running and easy restart would be good.

Wired printers on a host have two cases.

The simple subcase: printer is a general resource
that happens to be connected to a machine that's
really someone's workstation, etc. Here, the biggest
problem is setting up a server daemon.

I think the previous cover 90% of use.
What improvements do these categories need?

Once you have multiple printers, things get complex.
It's probably one for businesses. Is there
any IT staff? The VAX/VMS print spooler probably has a lot of
the controls for this case. It assumes an operator.

My take: the interface to lpr, lpd, etc could be
cleaned up. For most uses, the functionality is adequate.
Adding a "find a printer" program would help.

The complex case, well, how much work do you
want to put in defining requirements?

None of this, of course, covers running out of paper
when all the stores are closed.

Geoff Steckel



Re: Industrial use of line printers, does/would your company/organization use them with our lpd?

2016-02-18 Thread gwes

On 02/17/2016 12:49, Chris Bennett wrote:

After reading up on printers in use, I discovered that there is
significant use of line printers due to their very low cost of
consumables, production of a very long lasting output, unlike
laser/thermal/inkjet printers and high reliability.

Is anyone using these in a high volume output setting (not like a
restaurant or other low volume)?

If not using, but would like to, what is broken, missing or otherwise
wrong with our lpd/lpr system?

I do see that lpc, lpq, lprm are dinosaurs and have to be made extinct
and replaced with something more functional with more information output
and better capabilities.

Thanks,
Chris Bennett



I'm not sure what measure of "better" you're trying to apply.

lpr et al. don't have a GUI. One could be wrapped around them.

They don't do dynamic autoconfiguration.
In an industrial environment autoconfiguration can be very bad.
(examples like directing confidential output somewhere unexpected)

I worked for a company that ran as many IBM 1403 printers as
they could buy. Line printers are very simple to run.
They don't need elaborate output filters.

The only function I can think of that lpr doesn't have is
the capability to request a forms change and wait until
it has been done. That could be an entirely separate subsystem
invoked by lpr.

A laptop floating in many places could use something
complex like autoconfigure. Again, that could be wrapped
around lpr et al.

Geoff Steckel



Re: Industrial use of line printers, does/would your company/organization use them with our lpd?

2016-02-18 Thread gwes

On 02/17/2016 12:49, Chris Bennett wrote:

After reading up on printers in use, I discovered that there is
significant use of line printers due to their very low cost of
consumables, production of a very long lasting output, unlike
laser/thermal/inkjet printers and high reliability.

Is anyone using these in a high volume output setting (not like a
restaurant or other low volume)?

If not using, but would like to, what is broken, missing or otherwise
wrong with our lpd/lpr system?

I do see that lpc, lpq, lprm are dinosaurs and have to be made extinct
and replaced with something more functional with more information output
and better capabilities.

Thanks,
Chris Bennett


CUPS installs AVAHI. That is a security risk - it attempts
to change DNS lookups, etc.

Any package which pulls in something as disastrous as avahi
isn't welcome here.

lpr et al are primitive. They work fine for me and have
worked fine at all the places I worked except one
which was Linux-centric.

I just created and will submit to ports a version
of ghostscript which doesn't pull in cups - it
turns out the configuration has a switch for that case.

Geoff Steckel



usb audio stutters during inteldrm scrolling

2015-11-08 Thread gwes

inteldrm seems to block USB output during scrolling.

sox and other programs sending directly to the audio
device work perfectly unless the inteldrm console
changes. Then multiple short dropouts occur sounding
like scratches.

I don't see any errors logged anywhere.

Has anyone seen this?

Geoff Steckel

OpenBSD 5.8-current (GENERIC.MP) #1537: Tue Oct 20 09:44:09 MDT 2015
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8466022400 (8073MB)
avail mem = 8205316096 (7825MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb450 (20 entries)
bios0: vendor American Megatrends Inc. version "4.6.5" date 06/05/2013
bios0: BIOSTAR Group NM70I-1037U
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT SSDT SSDT
acpi0: wakeup devices PS2K(S4) PS2M(S4) UAR1(S4) P0P1(S4) USB1(S4) 
USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4) 
PXSX(S4) RP02(S4) PXSX(S4) [...]

acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) CPU 1037U @ 1.80GHz, 1796.21 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT

cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Celeron(R) CPU 1037U @ 1.80GHz, 1795.92 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,XSAVE,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT

cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xf800, bus 0-63
acpihpet0 at acpi0: 14318179 Hzacpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (P0P1)
acpiprt2 at acpi0: bus 2 (RP01)
acpiprt3 at acpi0: bus 3 (RP02)
acpiprt4 at acpi0: bus -1 (RP03)
acpiprt5 at acpi0: bus -1 (RP04)
acpiprt6 at acpi0: bus -1 (RP05)
acpiprt7 at acpi0: bus -1 (RP06)
acpiprt8 at acpi0: bus -1 (RP07)
acpiprt9 at acpi0: bus -1 (RP08)
acpiprt10 at acpi0: bus 1 (PEG0)
acpiprt11 at acpi0: bus -1 (PEG1)
acpiprt12 at acpi0: bus -1 (PEG2)
acpiprt13 at acpi0: bus -1 (PEG3)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C2(350@80 mwait.1@0x20), C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: FN00, resource for FAN0
acpipwrres1 at acpi0: FN01, resource for FAN1
acpipwrres2 at acpi0: FN02, resource for FAN2
acpipwrres3 at acpi0: FN03, resource for FAN3
acpipwrres4 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpitz2 at acpi0: critical temperature is 85 degC
acpibat0 at acpi0: BAT0 not present
acpibat1 at acpi0: BAT1 not present
acpibat2 at acpi0: BAT2 not presentehci0 at pci0 dev 26 function 0 
"Intel 7 Series USB" rev 0x04: apic 2 int 16

usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
azalia0 at pci0 dev 27 function 0 "Intel 7 Series HD Audio" rev 0x04: msi
azalia0: codecs: Realtek ALC662, Intel/0x2806, using Realtek ALC662
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 7 Series PCIE" rev 0xc4: msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 28 function 1 "Intel 7 Series PCIE" rev 0xc4: msi
pci3 at ppb2 bus 3
re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x07: RTL8168E/8111E-VL 
(0x2c80), msi, address b8:97:5a:5c:28:df

rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 5
ehci1 at pci0 dev 29 function 0 "Intel 7 Series USB" rev 0x04: apic 2 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
pcib0 at pci0 dev 31 function 0 "Intel NM70 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 7 Series AHCI" rev 0x04: msi, 
AHCI 1.3

ahci0: port 0: 6.0Gb/s
ahci0: port 2: 3.0Gb/s
ahci0: port 4: 3.0Gb/s
ahci0: port 5: 3.0Gb/s
scsibus2 at ahci0: 32 targets
sd0 at scsibus2 targ 0 lun 0:  SCSI3 
0/direct fixed naa.5001b4498e7f1be5

sd0: 61057MB, 512 bytes/sector, 125045424 sectors, thin
sd1 at scsibus2 targ 2 lun 0:  SCSI3 
0/direct fixed naa.5000c50050f08f56

sd1: 1907729MB, 512 bytes/sector, 3907029168 sectors
sd2 at scsibus2 targ 4 lun 0:  SCSI3 
0/direct fixed naa.50014ee0590e804b

sd2: 2861588MB, 512 

proposed speedup for diff -q

2014-05-15 Thread gwes
Proposed enhancement to diff:

diff of two very different files can take a very long time
and a lot of memory.
diff -q uses the same algorithm even though the status is
known at the first difference.

I propose ending the comparison at the first difference if
  diff is invoked with -q
  diff is not invoked with -w, -i, or -b

The changes pass the regression tests and all the tests I've tried.
I believe the changes are not machine dependent.
I invite criticism and counterexamples.

Example:

$ ls -l trash.120403 trash.120711
-rw---  1 gwes  users  249686538 Apr  3  2012 trash.120403
-rw-r--r--  1 gwes  users  142356923 Jul 11  2012 trash.120711

$ time diff -q trash.120403 trash.120711
diff: 
1m51.52s real 1m47.66s user 0m2.46s system

top output:

load averages:  1.02,  0.91,  0.58.oat.com 15:41:54
49 processes: 47 idle, 2 on processor
CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU1 states: 98.4% user,  0.0% nice,  1.6% system,  0.0% interrupt,  0.0% idle
Memory: Real: 403M/785M act/tot Free: 796M Cache: 312M Swap: 0K/1248M

  PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
18740 gwes  570  362M  333M onproc/1  biowait   1:05 95.61% diff


$ time work/newdiff/diff -q trash.120403 trash.120711
Files trash.120403 and trash.120711 differ
0m0.00s real 0m0.00s user 0m0.00s system

The code changes

$ diff -u diff.h work/newdiff/diff.h
--- diff.h  Thu May 15 16:29:15 2014
+++ work/newdiff/diff.h Thu May 15 15:57:30 2014
@@ -64,6 +64,10 @@
 #define D_PROTOTYPE0x080   /* Display C function prototype */
 #define D_EXPANDTABS   0x100   /* Expand tabs to spaces */
 #define D_IGNOREBLANKS 0x200   /* Ignore white space changes */
+   /* test for possible return at first difference 
*/
+#define CANBRIEFRETURN(flags) (((flags)  (D_FOLDBLANKS | D_IGNORECASE \
+   | D_IGNOREBLANKS \
+   )) == 0)
 
 /*
  * Status values for print_status() and diffreg() return values

$ diff -u diffreg.c work/newdiff/diffreg.c 
--- diffreg.c   Thu May 15 16:29:15 2014
+++ work/newdiff/diffreg.c  Thu May 15 16:31:19 2014
@@ -366,6 +366,15 @@
status |= 1;
goto closem;
}
+   if ((diff_format == D_BRIEF)  CANBRIEFRETURN(flags)) {
+   anychange = 1;
+   if (flags  D_HEADER) {
+   diff_output(%s %s %s\n, \
+   diffargs, file1, file2);
+   flags = ~D_HEADER;
+   }
+   goto closem;
+   }
if (lflag) {
/* redirect stdout to pr */
int pfd[2];



copyout from physio, why?

2011-12-28 Thread gwes
I hope someone can shed some light on this.

I'm running 5.0-current on an AMD64 with 4GB of physical memory.

Reading large chunks (64K or multiples) from /dev/rsd0c using
the AMD chipset SATA controller and a modern 1G drive:

time dd if=/dev/rsd3c of=/dev/null bs=128k count=1
1+0 records in
1+0 records out
131072 bytes transferred in 11.348 secs (115498831 bytes/sec)
0.0u 2.0s 0:11.34 17.9% 0+0k 0+1io 0pf+0w

Profiling the kernel shows that copyout() is being called from
physio() via uvm_vsunlock_device() for every MAXPHYS byte chunk.

On first inspection, physio calls uvm_vslock_device(..., map)
which checks to see if all pages in the request satisfy
PADDR_IS_DMA_REACHABLE(). If so, it returns NULL in map.
After strategy() returns, map is sent to uvm_vsunlock_device,
which calls copyout() if map != NULL.

There's a comment on uvm_vslock_device saying it always returns
something in *retp, but the code seems to indicate otherwise.
PADDR_IS_DMA_REACHABLE checks against dma_constraint, which
is 0..0x which should allow all memory  4G to be used
for DMA.

What have I missed?
I believe that the copyout() shouldn't happen.
I'm trying to run multiple 140MB/sec drives simultaneously and
the copyout() is a killer - it's eating more of the
system memory and CPU bandwidth than I'd like.

thanks
Geoff Steckel



pf question: multiple multihomed machines

2011-01-05 Thread gwes
What is the recommended pf.conf to get symmetrical routing
for incoming and outgoing connections using a dual-homed
gateway and internal hosts with static IPs on both WANs?

I'm assuming route-to and reply-to are the correct
tools to use.

I've looked at the FAQ, googled for dual  multihomed machines,
and haven't found a clear answer yet.

I know there's a multihome section in the FAQ, but
it only handles pools of nat-ed machines, and the last couple
of lines are not obvious.

I've got 2 WAN connections going to a gateway machine
with 3 physical interfaces and one virtual interface:

  vether0
 |
wan1 --- bridge0 --- wan2
 |
lan--|
 |nat-host-1
multihomed-host-1|
 |nat-host-2
multihomed-host-2|
 |nat-host-3
multihomed-host-3|
 |nat-host-4

For one wan, the PF can be reasonably simple, with most
of the rules on the WAN interfaces. Even now, it's quite long:

block in on $wan all
block in quick on $wans from evil-hosts to any
block out on wan proto udp from any to any port $bad_port_list
block out on wan proto tcp from any to internals
block out on wan proto udp from any to internals
etc

pass in on wan proto tcp from any to www-hosts port www
pass in on wan proto tcp from ssh-hosts \
to ssh-servers port ssh
pass in on wan proto tcp from mail-clients \
to mail-server port $mail-ports
pass in on wan proto tcp from any to mail-servers port smtp
 many pass in 
pass in on wan proto icmp $icmp_types to ping_hosts

pass out on wan from static_ip_range to ! static_ip_range

...and more things to handle nat-host-x on vether0
. voip port range rules are lengthy

I could generate 2 copies of the ruleset matching
each IP range and route-to/reply-to everywhere, but
that is lengthy, error prone, and otherwise painful.

Given the current pf.conf, presumably a

pass out on $wan2 from wan2_ip_range to \
 !any_internal_ip route-to ($wan2 $wan2_gateway)

and no state on any outgoing rules would work for
outbound traffic.

What about inbound traffic?
  no state on all incoming rules, and a

pass in on $wan2 from any to wan2_ip_range reply-to \
 ($wan2 $wan2_gateway)
rule could work.

Is this the best solution, given pf internals?

geoff steckel



Re: Resilient RAID

2010-05-21 Thread gwes
I ran a firewall/server for a year on a flash stick with full logging.
No problems.

As an ex-chip-verification-engineer, the BIG caveat is temperature.
Failures will at least double for every 10C above 20C or so.
Heat is electronics most vicious enemy.

geoff steckel
curmudgeon for hire, rent, or loan



Re: multi port ethernet card

2008-02-01 Thread gwes

I need 100mb interfaces.  I will probable go in a low end server class
machine.



 On 2008/02/01 12:21, Chris Bullock wrote:

  I need a recommendation for a quad port nic to put in my

 router/firewall.


www.bgmicro.com has a quad AIC-6915 card for $38 their part COM1204

I use it in my firewall. You need my 1-line patch to aic6915.c which
adds

in sf_stop:
cvs diff -c aic6915.c
Index: aic6915.c
===
RCS file: /cvs/src/sys/dev/ic/aic6915.c,v
retrieving revision 1.3
diff -c -r1.3 aic6915.c
*** aic6915.c   15 Dec 2006 15:28:27 -  1.3
--- aic6915.c   16 Jan 2008 14:35:07 -
***
*** 1245,1250 
--- 1245,1251 
ds-ds_mbuf = NULL;
}
}
+   sc-sc_txpending = 0;

if (disable)
sf_rxdrain(sc);

if you use them.

   geoff steckel



Re: pf.conf grammar botch

2006-11-05 Thread gwes

Peter N. M. Hansteen wrote:


The normal route for patch submissions is as far as I can tell via
posting the patch to tech@ and participating in any discussion that
ensues.  The developers very much want to be able to take a good look
at any code before it enters the tree.


This seems to be the case; I'd posted the suggestion first to see if
such a change might be acceptable even if the code were perfect.
There seemed to be an opinion that the parser was touchy.
I wanted any debate over functionality to be separate from that over
implementation details.
   thanks
  geoff



NICs on C7 cards, was Re: Via C7 fully supported?

2006-11-03 Thread gwes

Edward A. Gardner wrote:


Admittedly the price of gigabit NICs has come down to where I'd rather 
see them than 100mbit NICs.  They have advantages even if unable to run 
at full speed.  But running multiple gigabit links full speed, these 
boards won't.  Sigh.




Unless the 10/100 chips have improved markedly in the last three years,
none of them could handle a full load of minimum size packets.
The 1000 chips could saturate a 100 link with small packets.
This was a very strong reason to use the gbit NICs on 100M links.

   geoff steckel



Re: How much traffic can it route?

2006-11-03 Thread gwes

Der Engel wrote:


I have a doubt about if OpenBSD/PF can NAT 40Mbits with a simple rule
set and like 60 redirects.
The box has a xeon proc and two integrated NICs, one fxp and a bge,
can it handle it?


How many packets per second?  Or, how big are the packets?  The fxp 
would be the bottleneck - it cannot do 144000 PPS.  Two bge or even re 
would easily do 50,000 packets per second.  The biggest part of the 
overhead is per-packet not per-bit.

   geoff