Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Karel Gardas]

Tinker, what you basically try to describe as Fletcher is kind of how
ZFS is working. The Fletcher on the other hand is simple checksumming
algorithm. Please read something about ZFS design to know more about
it.
Now, what I did for RAID1 to become RAID1C is just to divide data area
of RAID1 to data area and chksum area. So layout is: . Also algorithm of placing chksums of
blocks is simply linear so far. That means: 1st block of data area is
CRC32ed into first 8 bytes of chksum area. 2nd block of data area is
CRC32ed into 2nd 8 bytes of chksum area. etc. For simplicity every 32k
of data in data area maps into 512 bytes (1 sector) of chksum area. As
you can see this is really as simple as possible and if you create ffs
in data area then if you force attach the drive as plain RAID1 you
still get the same data drive minus chksum area data amount (ffs
wise!) which means compatibility is preserved -- this is for case you
really like to get data out of RAID1C for whatever reason. This design
also supports detecting of your silently remapped block issue: Let's
have data block X and Y, both chksummed in CHX and CHY blocks in
chksum area. Now if you silently remap X -> Y, then X (on place of Y)
will not match with CHY. That's the case where both X and Y are in
data area. When not, then I assume your X is in data area and Y may be
either in  metadata area or in chksum area. in former case, meta-data
consistency is protected by MD5 sum (note: I have not tested
self-healing of this in this case). In the later case, by remapping X
to Y in chksum area you will basically corrupt chksum for a lot of
blocks in data area which will get detected and healed from the good
block(s) from good drive.
You also ask about I/O overhead. For read, you need to do: read data +
read chksum -- so 1 IO -> 2 IOs. For write it's more difficult:
generally you need to read chksum, write data, write new chksum. So 1
IO -> 3 IOs. This situation may be optimized to just 2 IOs in case of
32k aligned data write where the result is exactly alligned chksum
block(s) and so you don't need to read chksum, but just write
straight. That's also the reason why it's so important perfromance
wise to use 32k blocks fs on RAID1C. As I wrote I also tried to get
rid of read chksum (for general write) by using chksum blocks cache
but so far w/o success, read: it's buggy and corrupts data so far,
well I'm still just softraid beginner anyway and the problem is in not
knowing what upper layer (fs) and perhaps also on lower layer (scsi)
do which I don't know at all, I just try to fill the middle (sr) with
my code. Bad well man needs to learn, right. :-)
Last note: you talk about one RAID partition. Well, then no, neither
RAID1 nor RAID1C is for you since you need at least 2 RAID partitions
for this case, please read bioctl(8).



On Tue, Dec 1, 2015 at 9:03 PM, Tinker  wrote:
> Sorry for the spam - this is my last post before your next response.
>
> My best understanding is that within your RAID1C, Fletcher could work as a
> "CRC32 on steroids", because it would not only detect error when reading
> sectors/blocks that are broken because they contain inadvertently moved
> data, but also it would detect error when reading sectors/blocks where the
> write *did not go through*.
>
> In such a case, perhaps a disk mirror, or your self-healing area, could help
> figure out what should actually be on that provenly incorrect sector.
>
> This is awesome as it cements fread() integrity guarantees.
>
> The price it comes at, I guess, is a slight overhead (which is that the
> upper branches in the tree need to be updated), and also perhaps if there's
> a power failure that leaves the hash tree corrupt, correcting it would be
> pretty nasty - but that may be the whole point with it, that you're in a
> place where there always are backups and you just want to maximize the read
> correctness guarantees.
>
> For anything important I'd easily prefer to use that.
>
>
>
> On 2015-12-02 03:40, Tinker wrote:
>>
>> Just to illustrate the case. This is just how I got that it works,
>> please pardon the amateur level on algorithm details here.
>>
>> With the Fletcher checksumming, say that you have the Fletcher
>> checksum in a tree structure of two levels: One at the disk root, one
>> for every 100MB of data on the disk.
>>
>> When you read any given sector on the disk, it will be checked for
>> consistency with those two checksums, and if there's a failure,
>> fread() will fail.
>>
>>
>> Example: I write to sector/block X which is at offset 125MB.
>>
>> That means the root checksum and the 100MB-200MB branch checksums are
>> updated.
>>
>>
>> I now shut down and start my machine again, and now block/sector X
>> changed mapping with some random block/sector Y located at offset
>> 1234MB.
>>
>> Consequently, any fread() both of sector X and of sector Y will fail
>> deterministically, because both the root checksum and the 100-200MB
>> checksum and the 1200-1300MB 

Re: A branded USB stick as an alternative to the CD set?

[Marko Cupać]

Personally, I don't have the resources to contribute any amount
of money. Unix admin's job here in Serbia is paid in high 4 figures :)
Yearly, that's right.

But I work for a company whose networking relies heavily on
OpenBSD. My boss, although not from FOSS world, understands the value
of good software, so I managed to persuade him to approve purchase of
CD set every release. He also understand software is free to use, and
sees CD set as 'suggested semi-annual contribution'. Buying something
physical such as CD keeps our purchase department happy. It would be
much harder, or even impossible, to explain a donation, as would
purchase of multiple CD sets.

Now, if you take into account that the company I work for pays hundreds
of euros per year to a Cisco partner just to be able to get bug fixes
for single router, I don't think my boss would object to buying CD set
even if it was double or triple the price.

Just my point of view.
--
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: A branded USB stick as an alternative to the CD set?

[Adam Wolk]

On Tue, 1 Dec 2015 18:41:24 -0200
Michel Behr  wrote:

> Just one more thing: for non-developers, if you think there's any
> sense in this idea I just described, please, some "seconding" and/or
> additions would be welcomed. Also some e-mails directed to
> fundrais...@openbsdfoundation.org would be great in this regard too.

Oh God. I don't think Bob and others in the foundation will be happy
getting their mailboxes spammed. I'm sure they read this mailing list
- +1 emails don't sound like a good idea to me.

> (Again: OpenBSD developers should *NOT* need to get involved in this
> discussion, this is between non-developers and the OpenBSD
> foundation).
> 

OpenBSD foundation consists of at least some developers :)

Regards,
Adam

Re: A branded USB stick as an alternative to the CD set?

[Donald Allen]

"All I can do is buy the CD's and give some $ to the
foundation.  Any other suggestion is not productive."

I don't think that quite covers it. Those of us who have the choice
can send checks or Paypal money directly to Theo, as described on the
Donations page. I think checks are preferable, because they eliminate
Paypal skimming its credit-card-like fees, at the cost of a stamp. The
CDs also involve paying a middle-man. Checks to Theo get the maximum
amount of money to the place where it will do the project the most
good, which includes providing Theo with the money he needs to
continue doing what he's doing.

Zotac ZBOX-CI540

[bluesun08]

Hi,

i'm very frustrated and helpless.

I own a Zotac ZBOX-CI540. The installation of 5.8 works without any
problems.

But when i reboot the ZBOX the system won't start. The HDD light appear but
the system don't find the HDD and hangs.

What could be the problem here?

Regards

Alex 



--
View this message in context: 
http://openbsd-archive.7691.n7.nabble.com/Zotac-ZBOX-CI540-tp284264.html
Sent from the openbsd user - misc mailing list archive at Nabble.com.

Re: whats wrong with me?

[Krzysztof Strzeszewski]

Sorry, I'm beginner. I konow, my message was not logical.


uname -a:
#
OpenBSD hostname 5.8 GENERIC#0 i386
#


virtual server in httpd.conf:
#
server "hostname" {
listen on * port 80
listen on * tls port 443
log { access "access.log", error "error.log" }
tls { certificate "/etc/ssl/server.crt" key 
"/etc/ssl/private/server.key" }
root "/htdocs/hostname"
}
#


port 80 end 443 is open:
# netstat -a |grep http
#
tcp 0   0   localhost.https *.* LISTEN
tcp 0   0   *.https *.* LISTEN
#


in firefox:
#
Secure Connection Falied

An error occurred during a connection to my_domain. Cannot communicate
securely whih peer: no common encryption algorithm(s). (Error code:
ssl_error_no_cypher_overlap)
#


in log from httpd:
#
httpd: could not parse macro definition SSL
httpd[21336]: server_tls_init: failed to configure TLS - failed to read
private key: Operation not supported by device
#




Krzysztof Strzeszewski



On 30.11.2015 22:31, Alexander Salmin wrote:
> On 2015-11-30 20:52, Krzysztof Strzeszewski wrote:
>> Hi,
>> whats wrong?:
>>
>> httpd: could not parse macro definition SSL
>> httpd[21336]: server_tls_init: failed to configure TLS - failed to read
>> private key: Operation not supported by device
>>
>>
>> Krzysztof Strzeszewski
> Hey Krzysztof,
> 
> Two reasons why you did not receive much feedback on this.
> - You did not supply OpenBSD version (uname -a) so we can't replicate
> with same version.
> - You did not provide httpd.conf(8) so we can't replicate your exact setup.
> 
> A key to good free online OpenBSD support is to; "Always provide as much
> information as possible. Try to pin-point the exact problem. Give clear
> instructions on how to reproduce the problem. Try to describe the
> problem with as much accuracy and non-confusing terminology as possible,
> especially if it is not easy to reproduce." //
> http://www.openbsd.org/report.html
> 
> Continue to fail this and the world will just lead to sadness and despair.
> 
> Alexander

Re: OpenBSD 5.8 on VMware 5.5

[Rodney Hopkins]

>From: Felipe Gomes 
>To: misc@openbsd.org
>Sent: Tuesday, December 1, 2015 9:50 AM
>Subject: OpenBSD 5.8 on VMware 5.5
>
>Folks,
>I've been trying to search for more information on OpenBSD as a
>VMWare
>guest, but I wasn't able to find much... and the information is
>pretty much
>outdated.
>What are the recommendations for OpenBSD 5.8 (amd64) as a guest on
>VMware
>5.5?

Works fine.  No major caveats that I'm aware of.

>Guest Operating System: should I pick "Other (64bit)" or FreeBSD?

Currently "Other 64-bit" seems to be the way to go for me.  This has
varied a bit for me in the past.  "FreeBSD 64-bit" was my choice for
some time, then between about the 4.8 and 5.2 timeframe something
happened (not sure whether it was with VMWare or OpenBSD, but I began
having instability in my OpenBSD systems.  Random weird crashes.
Switched the OS selection to "Other 64-bit" and my problems went
away.  Rock solid stable now.  Conversely, I think the opposite was
also true at one point.  I used to run in "Other" mode and then a
upgraded  to  a  new  OpenBSD  version  and  began experiencing
instability.  Flipping to "FreeBSD" solved the issue for me then.  My
point in describing this is that so far, for me, changing the OS
setting between "FreeBSD" and "Other" has been the single most
important factor in determining the stability of OpenBSD on VMWare.

>How does OpenBSD work with "virtual sockets" and "cores per virtual
>socket"?

It just works.  Just like it does with physical cores.  Make sure
you select the .MP kernel if you want to take advantage of more than
one core/CPU in your OpenBSD VM. 

>What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?

I ran E1000 for years, no issue.  I converted to VMXNET3 around the
5.4 timeframe and haven't looked back.  Both are solid and both work
fine for my needs.

>What is the recommended SCSI Controller? LSI Logic Parallel, LSI
>Logic SAS
>or VMware Paravirtual?

I always use the suggested default for the OS type I select, which is
LSI Logic Parallel.  I've never seen the need, nor advantage to
changing it.

>I'd believe that all of these options work... I just don't know which
>is
>more stable or perform better.
>Any other tips on fine tunning or special setting?

As I said above, watch the "FreeBSD" "Other" setting.  If you upgrade
OpenBSD or patch VMWare and start getting crashes in OpenBSD try the
other OS setting for the guest VM.

>I'm planning on migrating a few Soekris boxes to virtual machines. Is
>this
>reliable? Is anyone running production OpenBSD servers on VMware?
>Thanks in advance!

I wouldn't call it "production" but I have OpenBSD guest VMs running
on VMWare 5.5 24x7 in my home lab.  They are solid and only go down,
when I tell them to.  Mind you, they don't get worked hard, it is only
a small home lab.  For my needs and in my setup, they are rock solid.

Hope that helps,

Rodney




_
Free E-mail by CamaroZ28.Com - FULL THROTTLE INTERNET

Re: whats wrong with me?

[Alexander Salmin]

On 2015-12-01 21:51, Krzysztof Strzeszewski wrote:

Sorry, I'm beginner. I konow, my message was not logical.


uname -a:
#
OpenBSD hostname 5.8 GENERIC#0 i386
#


virtual server in httpd.conf:
#
server "hostname" {
listen on * port 80
listen on * tls port 443
log { access "access.log", error "error.log" }
tls { certificate "/etc/ssl/server.crt" key 
"/etc/ssl/private/server.key" }
root "/htdocs/hostname"
}
#


port 80 end 443 is open:
# netstat -a |grep http
#
tcp 0   0   localhost.https *.* LISTEN
tcp 0   0   *.https *.* LISTEN
#


in firefox:
#
Secure Connection Falied

An error occurred during a connection to my_domain. Cannot communicate
securely whih peer: no common encryption algorithm(s). (Error code:
ssl_error_no_cypher_overlap)
#


in log from httpd:
#
httpd: could not parse macro definition SSL
httpd[21336]: server_tls_init: failed to configure TLS - failed to read
private key: Operation not supported by device
#


Check the following;

1) Does private key match certificate? Verify this like so
(should result in two exact same sha512 strings);
# openssl x509 -noout -modulus -in server.pem | openssl sha512
# openssl rsa -noout -modulus -in server.key | openssl sha512

2) Is httpd allowed to read key file?
# ls -lhart /etc/ssl/server.crt
# ls -lhart /etc/ssl/private/server.key

3) Check with browser random x on random other operating system y.

a little help with ipsec

[Marko Cupać]

Hi,

I am trying to setup IPsec VPN between fixed-ip central location and
dynamic-ip branch office. It works well once established, but when
public ip of branch office changes, it never re-establishes again. I
guess I misunderstood "dead peer detection" mechanism, which I hoped
will take care of realising the other side is dead, and try to
re-negotiate.

Is my ipsec.conf below optimal for such setup? Is it ok to use
"dynamic" on both sides or should i use "passive" in central office?
Should I go for "agressive" instead of "main" in branch office?

I can re-establish VPN by restarting no-ip2 on branch host, manually
restarting isakmpd, flushing SAs and reloading ipsec.conf with
ipsecctl after both hosts become aware that gate.noip.me points
to a new address. Should I script this with some pinger, or is there a
better way to accomplish my goal?

Thank you in advance.

ipsec.conf:
# central config
lan_central = "192.168.33.0/24"
lan_branche = "10.30.8.0/22"
gw_central =  "vpn.example.org" # <--- static
gw_branche =  "gate.noip.me" # <--- noip dynamic dns


ike dynamic esp from $gw_central to $gw_branche \
   main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   psk "hackme"

ike dynamic esp from $lan_central to $lan_branche peer $gw_branche \
   main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   psk "hackme"

# branch config
lan_central = "192.168.33.0/24"
lan_branche = "10.30.8.0/22"
gw_central =  "vpn.example.org" # <--- static
gw_branche =  "pppoe0"


ike dynamic esp from $gw_branche to $gw_central \
   main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   psk "hackme"

ike dynamic esp from $lan_branche to $lan_central peer $gw_central \
   main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
   psk "hackme"
--
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?

[Tinker]

(corrected the subject)

Karel,

So your current solution is *NOT* data-safe toward "mis-write":s and 
other write errors that go unnoticed at write time.


While I agree that the probability that the writes to both disks and to 
their checksum areas would fail are really low, the "hash tree"/"100% 
hash" way of ZFS must be said to be a big enabler because it's an 
integrity preservation/data safety scheme of a completely other, higher 
level:



The "checksum area" for the whole tree could be located right at the end 
of the disk too, meaning that the "backward compatibility" you describe 
would be preserved too.


You are right that Fletcher is just another hash function with the 
standard definition i.e. hash(data) => hashvalue -


ZFS' magic ingredient is a Merkle tree of hashes that's all.


The benefit I see with a hash tree is that you have in RAM always stored 
a hash of the whole disk (and the first level hashes in the hash tree).


This means that protection against serious transparent write 
errors/mis-write:s goes from none (although implausible) to really 
solid.



I see that the hash-tree could be implemented in a really simple, 
straightforward way:


What about you'd introduce an "über-hash", and then a fixed size of 
"first-level hashes".


The über-hash is a hash of all the first-level hashes, and the 
first-level hashes respectively are a hash of their corresponding set of 
bottom level checksums.


If for performance you need more levels then so be it, in all cases it 
can be contained right at the end of the disk.


The benefit here is that the über-hash and first level always will be 
kept in RAM. This means that as soon as any data or bottom-level 
checksums go out of the disk cache and later on are read from the 
physical disk, then the checking of all that data with the RAM-stored 
hashes, will give us the precious absolute fread() guarantee.


(Integrity between reboots will be a slightly more sensitive point. 
Maybe some sysctl could be used to extract the über-hash so you could 
doublecheck it after reboot.)


Thoughts?




Finally,

 * Really just a hashtree-based checksummed passthrough discipline would 
make all sense, e.g. JBOD .. or RAID 0.


   RAID 1 is nice but if you have many nodes and you just want Absolute 
fread() integrity on a single machine, hashtree-checksummed passthrough 
or JBOD or RAID 0 might be a preferable "lean and mean" solution.


   In an environment where you have perfect backups, RAID 1's benefit 
over passthrough is that disk degradation happens slightly more 
gracefully - instead of watching for broken file access and halting 
immediately then, then, as administrator you monitor those sysctl:s you 
introduce, that tell if either underlying disk is broken. I must admit 
that indeed that's pretty neat :)


   ..But still it could always happen that both disks break at the same 
time, so also still the passhtorugh usecase is really relevant also.


 * Do you do any load balancing of read operations to the underlying 
RAID:s, like, round robin?


 * About the checksum caching, I'm sure you can find some way to cache 
those so that you need to do less reads of that part of the disk, so the 
problem of lots of reads that you mention in your email will be 
completely resolved - if your code is correct, then the reading overhead 
from your RAID1C should be almost nonexistent.


Thanks,
Tinker

On 2015-12-02 05:15, Karel Gardas wrote:

Tinker, what you basically try to describe as Fletcher is kind of how
ZFS is working. The Fletcher on the other hand is simple checksumming
algorithm. Please read something about ZFS design to know more about
it.
Now, what I did for RAID1 to become RAID1C is just to divide data area
of RAID1 to data area and chksum area. So layout is: . Also algorithm of placing chksums of
blocks is simply linear so far. That means: 1st block of data area is
CRC32ed into first 8 bytes of chksum area. 2nd block of data area is
CRC32ed into 2nd 8 bytes of chksum area. etc. For simplicity every 32k
of data in data area maps into 512 bytes (1 sector) of chksum area. As
you can see this is really as simple as possible and if you create ffs
in data area then if you force attach the drive as plain RAID1 you
still get the same data drive minus chksum area data amount (ffs
wise!) which means compatibility is preserved -- this is for case you
really like to get data out of RAID1C for whatever reason. This design
also supports detecting of your silently remapped block issue: Let's
have data block X and Y, both chksummed in CHX and CHY blocks in
chksum area. Now if you silently remap X -> Y, then X (on place of Y)
will not match with CHY. That's the case where both X and Y are in
data area. When not, then I assume your X is in data area and Y may be
either in  metadata area or in chksum area. in former case, meta-data
consistency is protected by MD5 sum (note: I have not tested
self-healing of this in this case). In the later case, by remapping X
to 

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Tinker]

Wait, so you say the input for your CRC32 checksum is "metadata>".


So every sector/block in your model contains a CRC32 checksum of that, 
and on every fread() you check that.


Does the SR metadata contain the sector index number, so that if the 
sector index number would have changed inadvertently, your system would 
notice it (and if self-healing would fail,) fread() would fail?



On 2015-12-02 03:22, Karel Gardas wrote:
..

W.r.t. fletcher, I think we don't need it
and still will be able to detect moved block. That's due to layout
which is really simple: .

Re: A branded USB stick as an alternative to the CD set?

[Michel Behr]

As I understand, one of the reasons for the Foundation to avoid targeted
contributions is to preserve the independence of the project - in the
current model they are accountable for allocating the resources as they see
fit. So IMHO there is value in that model for that regard. On the other
hand, the fact that none of the donations is directed specifically to
compensate for the hard work of the developers (and more specifically
Theo's) gives the foundation the prerogative to, for example, have at the
donation page one donation account separate, specific for developers, with
a clear message that those resources would go directly to the developers
(or to one developer...), in contrast with the "standard" donation channel,
which funds only events, infrastructure, etc. It would a reasonable
exception. I think if this done with the same transparency things have
being managed so far, there's no problem.

And by the way, this suggestion is mine, not Theo's (and I'm far from being
a developer!), so I'm cc'ing the foundations' e-mail address - I see this
as matter of interest to the foundation because it touches directly their
purpose of providing the administrative support for the project to keep it
moving forward - e.g. providing a channel through the donations page for
developers to receive direct contributions would permit them the
flexibility to dedicate even more time to the project. It would also be one
more "communication channel" for recognition to the developers high-quality
code that's been produced over the years.

Anyway, just my $0.02... (I think this is a matter that's between the
non-developers community and the OpenBSD Foundation, Theo and the other
OpenBSD developers should not need to get involved on this discussion).

Kind regards,

On Tue, Dec 1, 2015 at 4:35 PM, Theo de Raadt 
wrote:

> > > > Now to be clear Theo, are donation via the paypal on the donations
> page
> > > > are directly to you and you can do as you see fit, and/or only checks
> > > > would be best?
> > >
> > > Correct, as I see fit.  I try to use it for the Project for things the
> > > Foundation doesn't fund.  I declared it that way on the web site.  I
> > > have not used it much for my own needs.
> >
> > I'd guess this has been thought of and just throwing in lame ideas
> > on the off chance it's of any use and maybe it's just extra site coding
> > work and there are legal complications, if not then are the people in
> > charge of the foundation website/operation privvy to this list? Is Bob
> > part of that?
> >
> > 
> > I wonder if it would gain any traction if there was a separate donation
> > box and cheque address with a statement along the lines of The OpenBSD
> > project leader works full time and receives no support from donations
> > to the foundation. If you would like to also support The project
> > leader directly then you can do so here or by sending a cheque to.
> >
> > ___Made up example, Don't send here
> >
> > Theo De Raadt
> > The OpenBSD project leader
> > 8101 160 Street
> > Edmonton, Alberta, Canada
> > T5R 2G9
> > ___
> >
> > Alternatively but perhaps more complex behind the scenes?.. a percentage
> > box so everytime someone makes a donation they can choose a percentage
> > of their donation to the foundation from 0%-?% that goes to support the
> > project leader.
> >
> > That way I guess the project leader could choose to waive it if the
> > foundation is ever in trouble financially should they wish so long as
> > the site foundation site discloses that possibility for legal reasons I
> > guess?
>
> We've heard numerous times that the OpenBSD Foundation avoids
> targetted contributions.  I don't think what you suggest is the
> right method of solving this (essentially, splitting a pie).

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Tinker]

Sorry for the spam - this is my last post before your next response.

My best understanding is that within your RAID1C, Fletcher could work as 
a "CRC32 on steroids", because it would not only detect error when 
reading sectors/blocks that are broken because they contain 
inadvertently moved data, but also it would detect error when reading 
sectors/blocks where the write *did not go through*.


In such a case, perhaps a disk mirror, or your self-healing area, could 
help figure out what should actually be on that provenly incorrect 
sector.


This is awesome as it cements fread() integrity guarantees.

The price it comes at, I guess, is a slight overhead (which is that the 
upper branches in the tree need to be updated), and also perhaps if 
there's a power failure that leaves the hash tree corrupt, correcting it 
would be pretty nasty - but that may be the whole point with it, that 
you're in a place where there always are backups and you just want to 
maximize the read correctness guarantees.


For anything important I'd easily prefer to use that.


On 2015-12-02 03:40, Tinker wrote:

Just to illustrate the case. This is just how I got that it works,
please pardon the amateur level on algorithm details here.

With the Fletcher checksumming, say that you have the Fletcher
checksum in a tree structure of two levels: One at the disk root, one
for every 100MB of data on the disk.

When you read any given sector on the disk, it will be checked for
consistency with those two checksums, and if there's a failure,
fread() will fail.


Example: I write to sector/block X which is at offset 125MB.

That means the root checksum and the 100MB-200MB branch checksums are 
updated.



I now shut down and start my machine again, and now block/sector X
changed mapping with some random block/sector Y located at offset
1234MB.

Consequently, any fread() both of sector X and of sector Y will fail
deterministically, because both the root checksum and the 100-200MB
checksum and the 1200-1300MB checksum checks would fail.


Reading other parts of the disk would work though.


On 2015-12-02 03:31, Tinker wrote:

Hi Karel,

Glad to talk to you.

Why the extra IO expense?


About the Fletcher vs not Fletcher thing, can you please explain to me
what happens in a setup where I have one single disk with one single
RAID partition on it using your disciple, and..

 1) I write a sector/block on some position X

 2) My disk's allocation table gets messed up so it's moved to another
random position Y

 3) I read sector/block on position Y

 4) Also I read sector/block on position X

Re: ansible openbsd_rcctl module

[Alexander Salmin]

On 2015-12-01 09:54, Sarevok Anchev wrote:

Hello,

Recently I submitted openbsd_rcctl to ansible. In order to speed up the
process of having it included by default, I'm asking the community to
review/test the module and drop a comment at
https://github.com/ansible/ansible-modules-extras/pull/1296

Let me know if there are other OpenBSD-specific modules you'd like to see
for ansible.

p.s: not subscribed to the list, cc me

Hey again,

Much appreciated as I said already. I left my computer and instantly
remembered a few more things. Hope it is OK.

My second wish; vlandev for vlan-interfaces and carpdev for 
carp-interfaces. See below.
Third wish, I'd like description from all interfaces visible. See below 
for vlan example but same for all.
Fourth wish; I'd like carp demote counters, advbase and advskew visible 
for carp.

Fifth wish; vhid for carp

I am very grateful for carp status however, it is already implemented.

Alexander




## TEST OPENBSD MACHINE

# uname -a
OpenBSD test46.local.lan 5.8 GENERIC#1534 amd64

# ifconfig vlan34 create vlandev bge0

# ifconfig vlan34
vlan34: flags=8843 mtu 1500
lladdr 00:24:81:eb:1f:14
priority: 0
vlan: 34 parent interface: bge0
groups: vlan
status: active

## LAPTOP WITH ANSIBLE (no vlandev is visible)

# ansible -m setup test46.local.lan -a 'filter=ansible_vlan34'
test46.local.lan | success >> {
"ansible_facts": {
"ansible_vlan34": {
"device": "vlan34",
"flags": [
"UP",
"BROADCAST",
"RUNNING",
"SIMPLEX",
"MULTICAST"
],
"ipv4": [],
"ipv6": [],
"macaddress": "00:24:81:eb:1f:14",
"mtu": "1500",
"status": "active",
"type": "unknown"
}
},
"changed": false
}

Re: A branded USB stick as an alternative to the CD set?

[Michel Behr]

Just one more thing: for non-developers, if you think there's any sense in
this idea I just described, please, some "seconding" and/or additions would
be welcomed. Also some e-mails directed to fundrais...@openbsdfoundation.org
would be great in this regard too. (Again: OpenBSD developers should *NOT*
need to get involved in this discussion, this is between non-developers and
the OpenBSD foundation).

On Tue, Dec 1, 2015 at 6:18 PM, Michel Behr  wrote:

> As I understand, one of the reasons for the Foundation to avoid targeted
> contributions is to preserve the independence of the project - in the
> current model they are accountable for allocating the resources as they see
> fit. So IMHO there is value in that model for that regard. On the other
> hand, the fact that none of the donations is directed specifically to
> compensate for the hard work of the developers (and more specifically
> Theo's) gives the foundation the prerogative to, for example, have at the
> donation page one donation account separate, specific for developers, with
> a clear message that those resources would go directly to the developers
> (or to one developer...), in contrast with the "standard" donation channel,
> which funds only events, infrastructure, etc. It would a reasonable
> exception. I think if this done with the same transparency things have
> being managed so far, there's no problem.
>
> And by the way, this suggestion is mine, not Theo's (and I'm far from
> being a developer!), so I'm cc'ing the foundations' e-mail address - I see
> this as matter of interest to the foundation because it touches directly
> their purpose of providing the administrative support for the project to
> keep it moving forward - e.g. providing a channel through the donations
> page for developers to receive direct contributions would permit them the
> flexibility to dedicate even more time to the project. It would also be one
> more "communication channel" for recognition to the developers high-quality
> code that's been produced over the years.
>
> Anyway, just my $0.02... (I think this is a matter that's between the
> non-developers community and the OpenBSD Foundation, Theo and the other
> OpenBSD developers should not need to get involved on this discussion).
>
> Kind regards,
>
> On Tue, Dec 1, 2015 at 4:35 PM, Theo de Raadt 
> wrote:
>
>> > > > Now to be clear Theo, are donation via the paypal on the donations
>> page
>> > > > are directly to you and you can do as you see fit, and/or only
>> checks
>> > > > would be best?
>> > >
>> > > Correct, as I see fit.  I try to use it for the Project for things the
>> > > Foundation doesn't fund.  I declared it that way on the web site.  I
>> > > have not used it much for my own needs.
>> >
>> > I'd guess this has been thought of and just throwing in lame ideas
>> > on the off chance it's of any use and maybe it's just extra site coding
>> > work and there are legal complications, if not then are the people in
>> > charge of the foundation website/operation privvy to this list? Is Bob
>> > part of that?
>> >
>> > 
>> > I wonder if it would gain any traction if there was a separate donation
>> > box and cheque address with a statement along the lines of The OpenBSD
>> > project leader works full time and receives no support from donations
>> > to the foundation. If you would like to also support The project
>> > leader directly then you can do so here or by sending a cheque to.
>> >
>> > ___Made up example, Don't send here
>> >
>> > Theo De Raadt
>> > The OpenBSD project leader
>> > 8101 160 Street
>> > Edmonton, Alberta, Canada
>> > T5R 2G9
>> > ___
>> >
>> > Alternatively but perhaps more complex behind the scenes?.. a percentage
>> > box so everytime someone makes a donation they can choose a percentage
>> > of their donation to the foundation from 0%-?% that goes to support the
>> > project leader.
>> >
>> > That way I guess the project leader could choose to waive it if the
>> > foundation is ever in trouble financially should they wish so long as
>> > the site foundation site discloses that possibility for legal reasons I
>> > guess?
>>
>> We've heard numerous times that the OpenBSD Foundation avoids
>> targetted contributions.  I don't think what you suggest is the
>> right method of solving this (essentially, splitting a pie).

Failure to boot install media using bootia32.efi

[Callum Davies]

I have two "devices" using IA32 UEFI firmware with 64-bit
hardware.  An Asus EeeBook X502TA and qemu-system-x86_64 with
an IA32 TianoCore firmware.  Neither of these will boot from
snapshots/amd64/install58.fs.

Attempting to run bootia32.efi from the UEFI shell of the qemu system
simply tells me "Command Error Status: Not Found".

The EeeBook is deficient, and doesn't provide an UEFI shell, but I
suspect it fails for the same reason.

Re: Zotac ZBOX-CI540

[Michael McConville]

bluesun08 wrote:
> I own a Zotac ZBOX-CI540. The installation of 5.8 works without any
> problems.
> 
> But when i reboot the ZBOX the system won't start. The HDD light
> appear but the system don't find the HDD and hangs.

Please share a dmesg. Let us or Freenode know if you need help with
that.

Re: Unable to sufficiently clean up softraid metadata

[Patrik Lundin]

On Wed, Dec 02, 2015 at 01:26:10AM +0100, Patrik Lundin wrote:
> 
> I have a custom installer script which automatically creates RAID
> devices and assembles an sd1 CRYPTO device before the ordinary installer
> continues (making the installer use sd1 for the rest of the
> installation).
> 

I forgot to mention this on OpenBSD 5.8.

-- 
Patrik Lundin

Unable to sufficiently clean up softraid metadata

[Patrik Lundin]

Hello,

I have a custom installer script which automatically creates RAID
devices and assembles an sd1 CRYPTO device before the ordinary installer
continues (making the installer use sd1 for the rest of the
installation).

This works well, other than needing this patch since the keydisk is on
the same harddrive:
http://marc.info/?l=openbsd-misc=141450636905550=2

The fdisk/disklabel magic does the following steps:
===
echo "Creating MBR partition on physical disk"
fdisk -iy sd0

echo "Creating crypto disklabel"
disklabel -E sd0 

Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?

[Raul Miller]

On Tue, Dec 1, 2015 at 5:21 PM, Tinker  wrote:
> So your current solution is *NOT* data-safe toward "mis-write":s and other
> write errors that go unnoticed at write time.
>
> While I agree that the probability that the writes to both disks and to
> their checksum areas would fail are really low, the "hash tree"/"100% hash"
> way of ZFS must be said to be a big enabler because it's an integrity
> preservation/data safety scheme of a completely other, higher level:

Anything can fail - you need numbers describing the failure rates (and
describing the performance and resource costs of the associated
features) to make an intelligent comparison.

Thanks,

-- 
Raul

Re: a little help with ipsec

[Stuart Henderson]

On 2015-12-01, Marko Cupać  wrote:
> Hi,
>
> I am trying to setup IPsec VPN between fixed-ip central location and
> dynamic-ip branch office. It works well once established, but when
> public ip of branch office changes, it never re-establishes again. I
> guess I misunderstood "dead peer detection" mechanism, which I hoped
> will take care of realising the other side is dead, and try to
> re-negotiate.
>
> Is my ipsec.conf below optimal for such setup? Is it ok to use
> "dynamic" on both sides or should i use "passive" in central office?
> Should I go for "agressive" instead of "main" in branch office?

Do not use aggressive mode.

> I can re-establish VPN by restarting no-ip2 on branch host, manually
> restarting isakmpd, flushing SAs and reloading ipsec.conf with
> ipsecctl after both hosts become aware that gate.noip.me points
> to a new address. Should I script this with some pinger, or is there a
> better way to accomplish my goal?
>
> Thank you in advance.
>
> ipsec.conf:
> # central config
> lan_central = "192.168.33.0/24"
> lan_branche = "10.30.8.0/22"
> gw_central =  "vpn.example.org" # <--- static
> gw_branche =  "gate.noip.me" # <--- noip dynamic dns
>
>
> ike dynamic esp from $gw_central to $gw_branche \
>main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>psk "hackme"
> 
> ike dynamic esp from $lan_central to $lan_branche peer $gw_branche \
>main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>psk "hackme"

Neither isakmpd nor iked tracks DNS changes. On the central side use
"passive" not "dynamic". Remove the "peer $gw_branche" to set this for the
'default peer' (i.e. to avoid matching on IP address).

Do you really need the first flow? It will simplify things if you can restrict
yourself to $lan_branche addresses and just have the second flow. (Otherwise
because you want to use the 'default peer' you'll need to collapse these into
a single rule with "to any").

It might be easier to get the basic setup working with psk first, but when
you have that up and running, see the PUBLIC KEY AUTHENTICATION section
in isakmpd(8) and get that setup, it is pretty simple to use and much safer
than psk.

> # branch config
> lan_central = "192.168.33.0/24"
> lan_branche = "10.30.8.0/22"
> gw_central =  "vpn.example.org" # <--- static
> gw_branche =  "pppoe0"
>
>
> ike dynamic esp from $gw_branche to $gw_central \
>main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>psk "hackme"

See above "Do you really need the first flow?". (If you do, you're going
to need to at least monitor addresses on pppoe0 on the client side and
restart; it won't track changes automatically).

The aim is to avoid having anything in config files which references the
dynamic address.

> ike dynamic esp from $lan_branche to $lan_central peer $gw_central \
>main auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>quick auth hmac-sha2-256 enc aes-256 group modp4096 lifetime 3600 \
>psk "hackme"

Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?

[Tinker]

On 2015-12-02 07:14, Raul Miller wrote:

On Tue, Dec 1, 2015 at 5:21 PM, Tinker  wrote:
So your current solution is *NOT* data-safe toward "mis-write":s and 
other

write errors that go unnoticed at write time.

While I agree that the probability that the writes to both disks and 
to
their checksum areas would fail are really low, the "hash tree"/"100% 
hash"

way of ZFS must be said to be a big enabler because it's an integrity
preservation/data safety scheme of a completely other, higher level:


Anything can fail - you need numbers describing the failure rates (and
describing the performance and resource costs of the associated
features) to make an intelligent comparison.


Raul,

At least as for me, I'd be happy to go with the merkle tree hash-based 
solution even if the overhead was extremely large, like anywhere up to 
80% lower IO performance would be fine with me. I would guess that that 
not is the case though, I think we're talking about something more like 
5-15% overhead.


I guess the choice of going with a merkle-fulldisk hash or not should be 
guided by practical need rather than performance.


But I agree local configuration options within such a setup are 
interesting to study, and also how the general design of such a setup 
affects performance.




As for failure rates and comparison with features, did you think of 
anything in particular -


What do you say of simply looking at what you need within what you do in 
particular, and then look at what overhead that would imply, and act 
from there?



I don't find anything particularly interesting in harddrive failure 
studies such as the following, feel free to correct me if you see 
anything else.


http://static.googleusercontent.com/media/research.google.com/sv//archive/disk_failures.pdf
https://www.usenix.org/legacy/events/fast07/tech/schroeder/schroeder.pdf
https://users.ece.cmu.edu/~omutlu/pub/flash-memory-failures-in-the-field-at-facebook_sigmetrics15.pdf
https://storagemojo.com/2007/02/20/everything-you-know-about-disks-is-wrong/

Best regards,
Tinker

Re: Unable to sufficiently clean up softraid metadata

[Nathan Wheeler]

I have a similar sort of setup during installs and I clear out the
first 10m before setting up the CRYPTO disk and it works for me. I
don't think you're zeroing out enough at the beginning of the disk.

dd if=/dev/zero of=/dev/rsd0c bs=10m count=1

On Tue, Dec 1, 2015 at 4:33 PM, Patrik Lundin  wrote:
> On Wed, Dec 02, 2015 at 01:26:10AM +0100, Patrik Lundin wrote:
>>
>> I have a custom installer script which automatically creates RAID
>> devices and assembles an sd1 CRYPTO device before the ordinary installer
>> continues (making the installer use sd1 for the rest of the
>> installation).
>>
>
> I forgot to mention this on OpenBSD 5.8.
>
> --
> Patrik Lundin

Re: OpenBSD 5.8 on VMware 5.5

[Ed Ahlsen-Girard]

> From:   Fabio Almeida 
> Date:   2015-12-01 16:18:43
> Message-ID: CAGd5O8LpM3Dz8N7fq8edWmuqnxnBEVgN16QETsOtHo69Ote_-w ()
> mail ! gmail ! com [Download message RAW]
> 
> Hi Felipe,
> 
> I'm running OpenBSD VMWare guests without problem, both as Firewall,
> IPSec VPN and FTP/SFTP servers.
> If you plan to run H.A systems with CARP, just be sure to enable
> "promiscuous mode" on the carp interfaces, both on the VM and the
> Hypervisor side.
> 
> Everything else you can let the default options, I have both 'vic'
> and 'em' interfaces without problem.
> 
> regards,
> Fabio Almeida

Please remember, Mr. Almeida, that if your firewall is a virtual
machine, the host is on the outside.
-- 

Edward Ahlsen-Girard
Ft Walton Beach, FL

Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?

[Raul Miller]

On Tue, Dec 1, 2015 at 6:49 PM, Tinker  wrote:
> At least as for me, I'd be happy to go with the merkle tree hash-based
> solution even if the overhead was extremely large, like anywhere up to 80%
> lower IO performance would be fine with me. I would guess that that not is
> the case though, I think we're talking about something more like 5-15%
> overhead.
..
> http://static.googleusercontent.com/media/research.google.com/sv//archive/disk_failures.pdf
> https://www.usenix.org/legacy/events/fast07/tech/schroeder/schroeder.pdf
> https://users.ece.cmu.edu/~omutlu/pub/flash-memory-failures-in-the-field-at-facebook_sigmetrics15.pdf
> https://storagemojo.com/2007/02/20/everything-you-know-about-disks-is-wrong/

This gives essentially zero information with which to compare the
relative failure rates between file system implementations. (Except to
point out that the hardware failure rates claimed by various
distributors of hard disks is something like an order of magnitude
better than observed failure rates - which suggests the underlying
market has become monopolistic in practice even if it superficially
appears to be something different.)

But I guess it's good to hear how you would be happy?

Thanks,

-- 
Raul

Re: A branded USB stick as an alternative to the CD set?

[Theo de Raadt]

>"All I can do is buy the CD's and give some $ to the
>foundation.  Any other suggestion is not productive."
>
>I don't think that quite covers it. Those of us who have the choice
>can send checks or Paypal money directly to Theo, as described on the
>Donations page. I think checks are preferable, because they eliminate
>Paypal skimming its credit-card-like fees, at the cost of a stamp. The
>CDs also involve paying a middle-man.

Completely true.  Also it is a 20 minute walk each way to the bank,
and keyboard folk need to do more walks.

>Checks to Theo get the maximum amount of money to the place where it
>will do the project the most good, which includes providing Theo with
>the money he needs to continue doing what he's doing.

On a personal "hate ramen noodles and tuna" level, I agree.

But my good-for-project-good-for-the-world side says the OpenBSD
Foundation is more effective at growing the contribution pie and
in particular funding the hackathons where great work happens.

Re: Zotac ZBOX-CI540

[メット]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 2015年12月2日 8:28:53 JST, Michael McConville  wrote:
>bluesun08 wrote:
>> I own a Zotac ZBOX-CI540. The installation of 5.8 works without any
>> problems.
>>
>> But when i reboot the ZBOX the system won't start. The HDD light
>> appear but the system don't find the HDD and hangs.
>
>Please share a dmesg. Let us or Freenode know if you need help with
>that.

Sorry for top posting...
iQE9BAEBCgAnBQJWXkcYIBxNZXR0IEhlbF9LZWl0YWkgPG1ldHRAcG1hcnMuanA+
AAoJEPao4OPC92Nkgd4H/0B/8qw46yn10BByCITO1srFCd2jiC8ez16ZP7EdTt5/
+ejyDBU9oxsxiywNkErGVNXLc94Ff1GxfU6g0amksCMTA4Xy5eAU004Zkpe8Jj3c
fWDteqGOlaT2avpoDnAap/QeBiAnTsXABQNeMjfsDvi7uQwE1hfrKnDB2rqpNUSK
J/DE3YBexwylfgp7/dWcBmH+EI0G+zzS4wRFJus+7mK/O0qEsn7AV2cdCYN4JatT
hkeMH+/G9xouI2QrTViOc9ZE/fIj9WfqPtMtH6R5duaZrFKXjNbtgYXQU7zYsMbF
QdsooNket3QhPmOXRxYM0AWM6T1h2MHNNb4neKzBlJk=
=hEft
-END PGP SIGNATURE-

Re: Zotac ZBOX-CI540

[メット]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

efi certificate?

On 2015年12月2日 8:28:53 JST, Michael McConville  wrote:
>bluesun08 wrote:
>> I own a Zotac ZBOX-CI540. The installation of 5.8 works without any
>> problems.
>>
>> But when i reboot the ZBOX the system won't start. The HDD light
>> appear but the system don't find the HDD and hangs.
>
>Please share a dmesg. Let us or Freenode know if you need help with
>that.
iQE9BAEBCgAnBQJWXkY1IBxNZXR0IEhlbF9LZWl0YWkgPG1ldHRAcG1hcnMuanA+
AAoJEPao4OPC92Nk/lMIAL4NeMyUQG5LhFh298vys2NdjTVZmL0ankxZ3cIyrtEp
eYP5Myyo8LrfdXfwQ/A1BLpX0OAdfXrfmZiqxDFgCIjXdVW2den9L7zyvV0vISEX
92GEQ4meyr8viMczWppmgxAdKE3Kmyw+N7swZes7nXtG9WUQ6NO6ljODiU/T3jyk
fTMFqov0JnuG+wb297HTg5V3NHL/xM3PREVXoi9m8ybp4pQj9zRWVhRfLzBpf2pl
zDmN5p9yvBIXhy6TuKRwxzgWhRJXx4gUojtjuEA0LeJrZiIp+kY8dd01wVGgU96Z
CmeChxZL5NeOXDzdcQ1oSn5bsBVEnHW3SWBByWcOO9A=
=RopI
-END PGP SIGNATURE-

Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Tinker]

Hi!

I heard someone was working with implementing Fletcher checksums in 
softraid.


Do you know any updates on this?



Fletcher checksums are how OpenBSD would guarantee that the data you 
read from disk actually has integrity. What makes it different from 
traditional checksumming is that it not only guarantees that a 
sector/block of data read has integrity within itself, but also that it 
actually belonged in the place on the disk that it was read from.


This is of particular importance when having sensitive information on 
disks with sector mapping, like all SSD:s (and even magnet disks, or??) 
have, which can break down.


For this reason, with ordinary filesystems, reading file contents could 
give you just about any data that's anywhere on the disk, while a 
Fletcher-based disk would give you a read error.


So it's really like a night and day difference.

https://en.wikipedia.org/wiki/Fletcher%27s_checksum

Thanks!
Tinker

Re: A branded USB stick as an alternative to the CD set?

[Kevin Chadwick]

> > Now to be clear Theo, are donation via the paypal on the donations page
> > are directly to you and you can do as you see fit, and/or only checks
> > would be best?  
> 
> Correct, as I see fit.  I try to use it for the Project for things the
> Foundation doesn't fund.  I declared it that way on the web site.  I
> have not used it much for my own needs.

I'd guess this has been thought of and just throwing in lame ideas
on the off chance it's of any use and maybe it's just extra site coding
work and there are legal complications, if not then are the people in
charge of the foundation website/operation privvy to this list? Is Bob
part of that?


I wonder if it would gain any traction if there was a separate donation
box and cheque address with a statement along the lines of The OpenBSD
project leader works full time and receives no support from donations
to the foundation. If you would like to also support The project
leader directly then you can do so here or by sending a cheque to.

___Made up example, Don't send here

Theo De Raadt
The OpenBSD project leader
8101 160 Street
Edmonton, Alberta, Canada
T5R 2G9
___

Alternatively but perhaps more complex behind the scenes?.. a percentage
box so everytime someone makes a donation they can choose a percentage
of their donation to the foundation from 0%-?% that goes to support the
project leader.

That way I guess the project leader could choose to waive it if the
foundation is ever in trouble financially should they wish so long as
the site foundation site discloses that possibility for legal reasons I
guess?

-- 

KISSIS - Keep It Simple So It's Securable

Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

[Tinker]

Hi!

I heard someone was working with implementing Fletcher checksums in 
softraid.


Do you know any updates on this?



Fletcher checksums are how OpenBSD would guarantee that the data you 
read from disk actually has integrity. What makes Fletcher checksums 
different from traditional checksumming e.g. CRC is that CRC only 
guarantees that a sector/block of data read has integrity within itself, 
while Fletcher also guarantees that the data read actually belongs in 
the place on the disk that it was read from.


The latter is of particular importance when having sensitive information 
on disks that have sector mapping implemented in them ( 
https://en.wikipedia.org/wiki/Flash_memory_controller#Flash_Translation_Layer_.28FTL.29_and_Mapping 
), like all SSD:s (and even magnet disks??) have, which can break down.


Also a disk could write to the wrong place because of firmware bugs or 
because it's getting worn out. The possible ways an SSD can break down 
are endless.



For this reason, with ordinary filesystems, fread() could give you just 
about any data from anywhere on the disk, while a Fletcher-based disk 
would give you a read error immediately on failure, so you're prompted 
to use backups, instead of going into processing broken information, 
which could have unlimitedly bad consequences (crash programs, 
compromise information, etc.).


So it's really like a night and day difference.

https://en.wikipedia.org/wiki/Fletcher%27s_checksum

Thanks!
Tinker

Re: OpenBSD 5.8 on VMware 5.5

[Bruno Flueckiger]

On 01.12.2015 16:50, Felipe Gomes wrote:

Folks,

I've been trying to search for more information on OpenBSD as a VMWare
guest, but I wasn't able to find much... and the information is pretty 
much

outdated.

What are the recommendations for OpenBSD 5.8 (amd64) as a guest on 
VMware

5.5?

Guest Operating System: should I pick "Other (64bit)" or FreeBSD?

How does OpenBSD work with "virtual sockets" and "cores per virtual 
socket"?


What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?

What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic 
SAS

or VMware Paravirtual?

I'd believe that all of these options work... I just don't know which 
is

more stable or perform better.

Any other tips on fine tunning or special setting?

I'm planning on migrating a few Soekris boxes to virtual machines. Is 
this

reliable? Is anyone running production OpenBSD servers on VMware?

Thanks in advance!


I run a productive SMTP server with OpenBSD 5.8-stable on VMware 5.5 for 
some
months and so far I didn't experience any problems. Guest OS is FreeBSD, 
NIC

is VMXNET3 and the controller is LSI Logic Parallel.

There are plans for more OpenBSD servers on VMware in the company I work 
for
due to the small footprint of the OS and the very good experience we 
have so

far.

Cheers,
Bruno

Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?

[Tinker]

Raul,

What do you mean?


What I wanted to say here has been that with respect to data safety, 
there are two classes of filesystems around, and that is those with a 
whole-disk hash on the one hand (ZFS, and I think maybe btrfs and 
Hammer2), and those who don't on the other.



I agree with you that "in practise, how do filesystems break" is an 
essential question to be asked, as that question impies "in practise, 
how do filesystems lose their data safety".


But also, that question is analogous to "is it worthwhile to download 
and check the SHA256/MD5 hash separately when downloading a file from 
the web".



In comparison, Karel's RAID1C in its present form would be like 
downloading the file twice, and per-block CRC32 hashes twice, and then 
comparing both copies to know you got the right thing.


That's nice as it provides some automatic healing, but, that has a 
limitation in the extra space used, and yet it's not safe to misdirected 
writes, not even across the time that it's mounted continuously.


Just hashing the whole disk (and also keeping that hash in RAM fort he 
whole period that it's in use) seems like a pretty inexpensive and "lean 
and mean" way to data safety guarantees to me.



We do know that what is happening is that disks do fail in all kinds of 
ways, some less and some more incredible, we do see that ordinary 
filesystems not would detect misdirected writes at the location where, 
and the question I wanted to pose by this conversation was how to 
maximize data safety -


Sorry for kind of pushing a particular way of thinking here, but, to 
some extent this is an algorithmical conversation where the exact way 
physical disks fail predominantly does not matter.


I agree that how widely it's worth to use this kind of hashing is an 
interesting question, both in understanding what overhead it implies 
performance-wise, and how frequently its unique safety benefits actually 
are of practical value -


I guess maybe the only way to get that answered would be by actually 
implementing it, and then maybe implementing also a routine to detect 
when it was uniquely beneficial to find a fault, as that can be easily 
detected (complementing sysctl diskhashing.detected_breach with a sysctl 
diskhashing.was_i_uniquely_needed).


This way, the performance overhead can be evaluated over ordinary 
non-hashed FS by ordinary IO tests, and its practical use can be done by 
users by monitoring the two sysctl:s and measuring how often 
diskhashing.was_i_uniquely_needed is set when 
diskhashing.detected_breach is set.



And finally of course an important question is exactly how the disk 
hashing scheme would be implemented best, and how disks break in 
practice would be central in answering that. But, at least as for me, if 
I just know there's strong hashing (and I can get a copy of the disk's 
total hash at unmount and mount time), I trust that enough and that's 
all I need -


I just want a catch-all data safety mechanism that safeguards against 
every type of disk breakdown, that's all.



What do you say about this way of reasoning?


Thanks,
Tinker


https://en.wikipedia.org/wiki/Btrfs#Checksum_tree_and_scrubbing

https://en.wikipedia.org/wiki/HAMMER



On 2015-12-02 10:17, Raul Miller wrote:

This gives essentially zero information with which to compare the
relative failure rates between file system implementations.

..


But I guess it's good to hear how you would be happy?

Thanks,

Re: OpenBSD 5.8 on VMware 5.5

[Nick Holland]

On 12/01/15 13:32, Stuart Henderson wrote:
> On 2015-12-01, Reyk Floeter  wrote:
>>> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS
>>> or VMware Paravirtual?
>>
>> LSI Logic SAS- mpi(4)
>> VMware Paravirtual   - vmwpvs(4)
>>
>> Use LSI Logic SAS.  The VMware Paravirtual has bugs that might corrupt
>> your data (seen with fsck).
> 
> It just seems to drop the first write to a virtual disk. (Not "first after 
> boot",
> but "first ever after attaching the disk to vmwpvs" (where "attaching" 
> includes
> "changing the disk from another controller type to paravirtual").
> 
> YMMV but I interrupt the installer early and 'fdisk -iy sd0', resume the 
> install
> and haven't run into any problem.

Well, I have.
Nasty, repeatable problems bringing up VMs after hard shutdowns/restarts
(my local power company gave me a couple of those right after setting up
my VMware host).

fsck runs...writes changes out to disk, but misses the first write, so
root ends up being read-only, which is quite annoying when not expecting
it.  Not impossible (though a little odd) to fix, certainly, but much
more than you want to do in a production environment after an "event",
and heaven (and a few assistants) help you if you have 100 VMs to bring
back up.

Not to mention the idea of having the first write of an fsck just
mysteriously not happen kinda creeps me out.  I'm not a FS guy, but it
seems to me that skipping the occasional write, or even just the first
write, isn't going to improve data integrity. :)

Nick.

Re: OpenBSD 5.8 on VMware 5.5

[Stuart Henderson]

On 2015-12-01, Reyk Floeter  wrote:
>> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS
>> or VMware Paravirtual?
>
> LSI Logic SAS - mpi(4)
> VMware Paravirtual- vmwpvs(4)
>
> Use LSI Logic SAS.  The VMware Paravirtual has bugs that might corrupt
> your data (seen with fsck).

It just seems to drop the first write to a virtual disk. (Not "first after 
boot",
but "first ever after attaching the disk to vmwpvs" (where "attaching" includes
"changing the disk from another controller type to paravirtual").

YMMV but I interrupt the installer early and 'fdisk -iy sd0', resume the install
and haven't run into any problem.

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

[Chris Cappuccio]

Tinker [ti...@openmailbox.org] wrote:
> Hi!
> 
> I heard someone was working with implementing Fletcher checksums in
> softraid.
> 
> Do you know any updates on this?
> 

Karel Gardas was working on an implementation of RAID1C for softraid

Last I remember, it needs to be pulled out into smaller pieces

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid? (+better phrasing)

[Tinker]

On 2015-12-02 02:27, Chris Cappuccio wrote:

Tinker [ti...@openmailbox.org] wrote:

Hi!

I heard someone was working with implementing Fletcher checksums in
softraid.

Do you know any updates on this?



Karel Gardas was working on an implementation of RAID1C for softraid

Last I remember, it needs to be pulled out into smaller pieces



Chris,

I see Karel's patch from September at 
http://marc.info/?t=14410531914=1=2 . It says CRC32 only there, 
nothing about Fletcher - am I missing something?


That's the one you meant right?



I guess "RAID1-Checksummed" is neat as the RAID1 logic has low overhead 
in itself, and you can sandwhich that RAID with other RAID:s.


Thanks!
Tinker

Re: A branded USB stick as an alternative to the CD set?

[Theo de Raadt]

> > > Now to be clear Theo, are donation via the paypal on the donations page
> > > are directly to you and you can do as you see fit, and/or only checks
> > > would be best?  
> > 
> > Correct, as I see fit.  I try to use it for the Project for things the
> > Foundation doesn't fund.  I declared it that way on the web site.  I
> > have not used it much for my own needs.
> 
> I'd guess this has been thought of and just throwing in lame ideas
> on the off chance it's of any use and maybe it's just extra site coding
> work and there are legal complications, if not then are the people in
> charge of the foundation website/operation privvy to this list? Is Bob
> part of that?
> 
> 
> I wonder if it would gain any traction if there was a separate donation
> box and cheque address with a statement along the lines of The OpenBSD
> project leader works full time and receives no support from donations
> to the foundation. If you would like to also support The project
> leader directly then you can do so here or by sending a cheque to.
> 
> ___Made up example, Don't send here
> 
> Theo De Raadt
> The OpenBSD project leader
> 8101 160 Street
> Edmonton, Alberta, Canada
> T5R 2G9
> ___
> 
> Alternatively but perhaps more complex behind the scenes?.. a percentage
> box so everytime someone makes a donation they can choose a percentage
> of their donation to the foundation from 0%-?% that goes to support the
> project leader.
> 
> That way I guess the project leader could choose to waive it if the
> foundation is ever in trouble financially should they wish so long as
> the site foundation site discloses that possibility for legal reasons I
> guess?

We've heard numerous times that the OpenBSD Foundation avoids
targetted contributions.  I don't think what you suggest is the
right method of solving this (essentially, splitting a pie).

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Karel Gardas]

I don't know about fletcher, but I'm working on crc32 based
checksumming for soft raid1. The basic implementation is ready but I'm
not satisfied with write performance in some cases: small files, lots
of collisions in chksumming blocks etc. Worst case I see 6-7x slower
performance here in comparison with plain RAID1. I've tried to make
that situation better with the chksumming blocks cache on which I've
been working last few weekends, but still this is not right and while
using 32k blocks fs the improvements are not worth the much higher
complexity of the code, so I'll probably switch to scrub hacking which
is something you usually need in case of chksumming anyway. :-)
On the bright side: code "self-heal" bad block happily and refuse to
push you bad data in case of errors on all chunks. Also due to
simplicity of design if something runs really badly you still can
detach drive and attach it as a plain RAID1 and get your data out.
W.r.t. performance read is on 70-80% of plain RAID1 and write of big
data (>=32k on 32k block fs) is about 70% of plain RAID1. Also
PostgreSQL pgbench is about 70% of speed of RAID1 (again on 32k block
fs). Just small files sucks. W.r.t. fletcher, I think we don't need it
and still will be able to detect moved block. That's due to layout
which is really simple: .

Are you willing to test the code on your setup? If so, I can save the
patch somewhere for you but well, my tree is month old or so if you
don't mind...

PS: all performance figures got on haswell based server with 2 WD Re
512 bytes sector (physical size) drives. So your numbers may vary and
I'm certainly interested to know them -- if you benchmark.

On Tue, Dec 1, 2015 at 6:31 PM, Tinker  wrote:
> Hi!
>
> I heard someone was working with implementing Fletcher checksums in
> softraid.
>
> Do you know any updates on this?
>
>
>
> Fletcher checksums are how OpenBSD would guarantee that the data you read
> from disk actually has integrity. What makes it different from traditional
> checksumming is that it not only guarantees that a sector/block of data read
> has integrity within itself, but also that it actually belonged in the place
> on the disk that it was read from.
>
> This is of particular importance when having sensitive information on disks
> with sector mapping, like all SSD:s (and even magnet disks, or??) have,
> which can break down.
>
> For this reason, with ordinary filesystems, reading file contents could give
> you just about any data that's anywhere on the disk, while a Fletcher-based
> disk would give you a read error.
>
> So it's really like a night and day difference.
>
> https://en.wikipedia.org/wiki/Fletcher%27s_checksum
>
> Thanks!
> Tinker

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Tinker]

Hi Karel,

Glad to talk to you.

Why the extra IO expense?


About the Fletcher vs not Fletcher thing, can you please explain to me 
what happens in a setup where I have one single disk with one single 
RAID partition on it using your disciple, and..


 1) I write a sector/block on some position X

 2) My disk's allocation table gets messed up so it's moved to another 
random position Y


 3) I read sector/block on position Y

 4) Also I read sector/block on position X


Maybe an advantage with the Fletcher thing is that, as I understood it, 
it's in a way like a "tree-ed checksum" structure so the disk has a 
"root checksum" that's for all the disk, which is updated at write time 
too i.e. 1) (and any hash tree levels between the root and the position 
X which is written).


This means that not only would 3) here report failure, but also 4), 
which is perfect, i.e. the Fletcher thing would catch *any* 
inconsistency anywhere on the disk.


Maybe it could be argued that it's "too picky" for some less data-safe 
environments, but, in a place where you have good backups and you value 
100.0% fread() correctness, it's awesome!!



Looking forward to your response, thanks!
Tinker

On 2015-12-02 03:22, Karel Gardas wrote:

I don't know about fletcher, but I'm working on crc32 based
checksumming for soft raid1. The basic implementation is ready but I'm
not satisfied with write performance in some cases: small files, lots
of collisions in chksumming blocks etc. Worst case I see 6-7x slower
performance here in comparison with plain RAID1. I've tried to make
that situation better with the chksumming blocks cache on which I've
been working last few weekends, but still this is not right and while
using 32k blocks fs the improvements are not worth the much higher
complexity of the code, so I'll probably switch to scrub hacking which
is something you usually need in case of chksumming anyway. :-)
On the bright side: code "self-heal" bad block happily and refuse to
push you bad data in case of errors on all chunks. Also due to
simplicity of design if something runs really badly you still can
detach drive and attach it as a plain RAID1 and get your data out.
W.r.t. performance read is on 70-80% of plain RAID1 and write of big
data (>=32k on 32k block fs) is about 70% of plain RAID1. Also
PostgreSQL pgbench is about 70% of speed of RAID1 (again on 32k block
fs). Just small files sucks. W.r.t. fletcher, I think we don't need it
and still will be able to detect moved block. That's due to layout
which is really simple: .

Are you willing to test the code on your setup? If so, I can save the
patch somewhere for you but well, my tree is month old or so if you
don't mind...

PS: all performance figures got on haswell based server with 2 WD Re
512 bytes sector (physical size) drives. So your numbers may vary and
I'm certainly interested to know them -- if you benchmark.

On Tue, Dec 1, 2015 at 6:31 PM, Tinker  wrote:

Hi!

I heard someone was working with implementing Fletcher checksums in
softraid.

Do you know any updates on this?



Fletcher checksums are how OpenBSD would guarantee that the data you 
read
from disk actually has integrity. What makes it different from 
traditional
checksumming is that it not only guarantees that a sector/block of 
data read
has integrity within itself, but also that it actually belonged in the 
place

on the disk that it was read from.

This is of particular importance when having sensitive information on 
disks
with sector mapping, like all SSD:s (and even magnet disks, or??) 
have,

which can break down.

For this reason, with ordinary filesystems, reading file contents 
could give
you just about any data that's anywhere on the disk, while a 
Fletcher-based

disk would give you a read error.

So it's really like a night and day difference.

https://en.wikipedia.org/wiki/Fletcher%27s_checksum

Thanks!
Tinker

Re: ansible openbsd_rcctl module

[Alexander Salmin]

On 2015-12-01 09:54, Sarevok Anchev wrote:

Hello,

Recently I submitted openbsd_rcctl to ansible. In order to speed up the
process of having it included by default, I'm asking the community to
review/test the module and drop a comment at
https://github.com/ansible/ansible-modules-extras/pull/1296

Let me know if there are other OpenBSD-specific modules you'd like to see
for ansible.

p.s: not subscribed to the list, cc me

Hey Sarevok,

Much appreciated. If you have the time I'd really like improved gre 
interface support for the tunnel configuration. See below example. 
Thanks for asking and for offer your help.


## TEST OPENBSD MACHINE

# uname -a
OpenBSD test46.local.lan 5.8 GENERIC#1534 amd64

# ifconfig gre0 create 1.2.3.4 5.6.7.8 tunnel 11.22.33.44 55.66.77.88

# ifconfig gre0
gre0: flags=9011 mtu 1476
priority: 0
groups: gre
tunnel: inet 11.22.33.44 -> 55.66.77.88
inet 1.2.3.4 --> 5.6.7.8 netmask 0xff00


## LAPTOP WITH ANSIBLE

# ansible -m setup test46.local.lan -a 'filter=ansible_gre0'

test46.local.lan | success >> {
"ansible_facts": {
"ansible_gre0": {
"device": "gre0",
"flags": [
"UP",
"POINTOPOINT",
"LINK0",
"MULTICAST"
],
"ipv4": [
{
"address": "1.2.3.4",
"broadcast": "0xff00",
"netmask": "5.6.7.8",
"network": "1.2.3.0"
}
],
"ipv6": [],
"macaddress": "unknown",
"mtu": "1476",
"type": "unknown"
}
},
"changed": false
}

Re: Any news on Fletcher checksums (=ZFS-style checksums) in softraid?

[Tinker]

Just to illustrate the case. This is just how I got that it works, 
please pardon the amateur level on algorithm details here.


With the Fletcher checksumming, say that you have the Fletcher checksum 
in a tree structure of two levels: One at the disk root, one for every 
100MB of data on the disk.


When you read any given sector on the disk, it will be checked for 
consistency with those two checksums, and if there's a failure, fread() 
will fail.



Example: I write to sector/block X which is at offset 125MB.

That means the root checksum and the 100MB-200MB branch checksums are 
updated.



I now shut down and start my machine again, and now block/sector X 
changed mapping with some random block/sector Y located at offset 
1234MB.


Consequently, any fread() both of sector X and of sector Y will fail 
deterministically, because both the root checksum and the 100-200MB 
checksum and the 1200-1300MB checksum checks would fail.



Reading other parts of the disk would work though.


On 2015-12-02 03:31, Tinker wrote:

Hi Karel,

Glad to talk to you.

Why the extra IO expense?


About the Fletcher vs not Fletcher thing, can you please explain to me
what happens in a setup where I have one single disk with one single
RAID partition on it using your disciple, and..

 1) I write a sector/block on some position X

 2) My disk's allocation table gets messed up so it's moved to another
random position Y

 3) I read sector/block on position Y

 4) Also I read sector/block on position X

Re: procmap prints ?VNODE?

[Ted Unangst]

Stefan Berger wrote:
> hi,
> 
> with the command 'procmap pid', I often/always get ?VNODE? instead of
> the actual filename.  My question is, whether this is on purpose because
> on similary BSDs (pmap on NetBSD) , I don't get ?VNODE? but the actual 
> filename.  Any ideas what went wrong?

digging filenames out of the kernel is not well supported. in the general
case, it's not even possible since a file may have more than one name, or even
none.

Re: vmmctl and vmd problem

[Mike Larkin]

On Sat, Nov 28, 2015 at 09:46:36AM +, freeu...@ruggedinbox.com wrote:
> 26 Nov 2015 at 21:10:06, Norman Golisz  wrote:
> >This is expected. vmm(4) is not yet enabled in the default kernel
> >configuration.
> 
> Thanks for your hints:)
> 
> I tried "config -e -f /bsd", then "list" & "find vm".
> no result, OpenBSD amd64 snapshots in 28 Nov 2015.
> 
> vmm, vmd, vmmctl on snapshots, and "man vmd" said "vmd_flags=" in
> "/etc/rc.conf.local".
> but, "/etc/rc.conf" and "/etc/rc" didn't in these codes. :<
> 
> 
> I should be compile the kernel "-current" ?
> 

This is still under active development. You would need to use
a custom config to enable the option.

Re: Recommended Industrial PCs?

[Clint Pachl]

Martin Haufschild wrote on 08/26/15 12:11:
can someone recommend me an Industrial PC (IPC) to use with OpenBSD? I 
would like to have a lot of hardware supported from this IPC by OpenBSD.


I've had great luck with Lanner (http://www.lannerinc.com/).

I've been running a LEC-2280 and FW-7541 for almost 2 years now for my 
business. They run the local network and public ecommerce website. The 
LEC is the main server with an Intel Core i7-3555LE @ 2.5GHz. The FW is 
the firewall/gateway running other light services, like DNS and NTP, 
with an Atom D525 1.8GHz.


I set them on top of a cabinet in a closet and just forget about them; 
nobody knows they exist. I haven't physically touched them since I 
installed them almost 2 years ago. The ambient temperature ranges from 
about 70-90F. These two boxes always stay cool regardless of the temp; 
plus these machines are fanless so they don't suck dust.


I interconnected them with serial cables to assist with out-of-band 
maintenance. For instance, I SSH into one machine, then connect via 
serial to the other for console access. That's been working out really 
well through 3 or 4 upgrade cycles now.


With the maximum RAM and best CPUs at the time, the LEC-2280 and FW-7541 
were about $1200 USD and $400 USD, respectively. I would highly 
recommend them. Plus, their customer support was very helpful. Their 
tech support even tests and runs OpenBSD, which is what sealed the deal 
for me.


http://www.lannerinc.com/products/embedded-box-pcs/industrial-automation/lec-2280
http://www.lannerinc.com/products/x86-network-appliances/desktop/fw-7541

Re: bridge fails to broadcast ARP from gif tunnel

[Kazuya GODA]

Hi,

It seems to bridge doesn't forward broadcast/multicast frames from gif.
This pathc will fix this problem, so would you try it?

Thanks,

- Goda

Index: net/if_bridge.c
===
RCS file: /cvs/src/sys/net/if_bridge.c,v
retrieving revision 1.270
diff -u -p -r1.270 if_bridge.c
--- net/if_bridge.c 7 Nov 2015 12:42:19 -   1.270
+++ net/if_bridge.c 1 Dec 2015 08:44:42 -
@@ -1337,18 +1337,21 @@ bridge_process(struct ifnet *ifp, struct
if (mc == NULL)
goto reenqueue;

-   bridge_ifinput(ifp, mc);
 #if NGIF > 0
if (ifp->if_type == IFT_GIF) {
TAILQ_FOREACH(ifl, >sc_iflist, next) {
if (ifl->ifp->if_type != IFT_ETHER)
continue;

-   bridge_ifinput(ifl->ifp, m);
-   return;
+   bridge_ifinput(ifl->ifp, mc);
+   break;
}
-   }
+   if (!ifl)
+   m_freem(mc);
+   } else
 #endif /* NGIF */
+   bridge_ifinput(ifp, mc);
+   
bridgeintr_frame(sc, ifp, m);
return;
}




On 2015/11/28 15:33, Rolf Sommerhalder wrote:

Using the simple Layer-2 bridge setup below, an ICMP Ping 172.17.1.5
from HostA does not get to HostB while using EtherIP encapsulation with
gif(4) at its tunnel end points.

The Ping's initial Ethernet broadcasts with the ARP Requests make it
through the gif tunnel to BridgeB, to both its bridge0 and vio2
interfaces (check with tcpdump, tshark).

However, vio2 never re-broadcasts those ARP Requests on the wire to
HostB!? E.g. the physical egress interface vio2, which is member of a
bridge(4) on BridgeB, receives the ARP Requests, but it fails
re-broadcast them to HostB so that Host could answer with ARP Reponses.

Also, BridgeB does not learn the source MAC from HostA (and of course it
can not learn the MAC of HostB, because ARP Requests never get there).

However, pinging the (numbered) vio2 on BridgeB succeeds (Ping
172.17.1.2 from HostA), e.g. the gif tunnel is OK.

Also, HostA can ping HostB after removal of the gif tunnel, e.g. after
deleting gif0 from bridge0 on both BridgeA and BridgeB, and adding vio1
to them instead.

Testing conditions:
- default installs of OpenBSD i386 snapshot from yesterday
- pf is disabled
- no L2 filter rules on the bridge member interfaces
- set sysctl net.inet.etherip.allow=1 to enable EtherIP on gif()
- the observation is the same on both VirtualBox with vio() interfaces,
as well as on a real hardware with APU2 that have em() interfaces.

Currently, experimenting with pf enabled on BridgeB, I found that ARP
Requests apparently do not generate state with a very basic rule-set,
such as 'pass log all'.

What did I miss?  Or, is there "just a bug" in the gif/bridge combo that
is haunting me?
Would it be worthwhile to try with -stable or an older version of
OpenBSD?  Years ago, I had such a setup working with 4.3, and I can make
configuration files available (although they are very minimal, mostly
running default install) ...

Thanks for any hints and suggestions!
Rolf


*HostA*
vio1 172.16.0.5/22
  |
  v
vio2 172.16.0.2/22
*BridgeA*
bridge0 add vio2 add gif0
gif0 tunnel 10.10.1.2 10.10.1.3
vio1 10.10.1.2/24
  |
  v
vio1 10.10.1.3/24
gif0 tunnel 10.10.1.3 10.10.1.2
bridge0 add vio2 add gif0
*BridgeB*
vio2 172.16.1.2/22
  |
  v
vio1 172.16.1.5/22
*HostB*

ansible openbsd_rcctl module

[Sarevok Anchev]

Hello,

Recently I submitted openbsd_rcctl to ansible. In order to speed up the
process of having it included by default, I'm asking the community to
review/test the module and drop a comment at
https://github.com/ansible/ansible-modules-extras/pull/1296

Let me know if there are other OpenBSD-specific modules you'd like to see
for ansible.

p.s: not subscribed to the list, cc me

Re: ansible openbsd_rcctl module

[Antoine Jacoutot]

On Tue, Dec 01, 2015 at 08:54:25AM -, Sarevok Anchev wrote:
> Hello,
> 
> Recently I submitted openbsd_rcctl to ansible. In order to speed up the
> process of having it included by default, I'm asking the community to
> review/test the module and drop a comment at
> https://github.com/ansible/ansible-modules-extras/pull/1296
> 
> Let me know if there are other OpenBSD-specific modules you'd like to see
> for ansible.

Isn't there support for rcctl in ansible already?

-- 
Antoine

Re: bridge fails to broadcast ARP from gif tunnel

[Rolf Sommerhalder]

Hi Goda,

On Tue, Dec 1, 2015 at 10:07 AM, Kazuya GODA  wrote:
> It seems to bridge doesn't forward broadcast/multicast frames from gif.
> This pathc will fix this problem, so would you try it?

Indeed, your patch fixes the problem! Excellent, thank you very much.

Now, I will go on and try IPsec with the same bridge test setup... :-)

Will you merge the fix into -current?

Thanks again,
Rolf

Re: A branded USB stick as an alternative to the CD set?

[Gareth Nelson]

Theo: like others in this thread I find it quite shocking and disappointing
how poorly you are doing financially from your hard work.

I apologise if this is too obvious a suggestion but if the foundation is
making a sufficient income is it not possible for you to draw a salary as
an employee?

As to the original topic, there are companies out there that will do USB
sticks, but it's probably simpler for end users to just download the
installer and stick it on a stick themselves.
On 1 Dec 2015 4:05 am, "Theo de Raadt"  wrote:

> > The good news if any, is that Gifts are tax free in Canada, so that part
> > is helpful and users should fell they get more out of their money freely
> > given as a gift.
> >
> > http://www.taxtips.ca/personaltax/giftsandinheritances.htm
>
> Correct, but be careful it will not be interpreted later as a non-gift.
>
> > Now to be clear Theo, are donation via the paypal on the donations page
> > are directly to you and you can do as you see fit, and/or only checks
> > would be best?
>
> Correct, as I see fit.  I try to use it for the Project for things the
> Foundation doesn't fund.  I declared it that way on the web site.  I
> have not used it much for my own needs.
>
> Please don't assume that a lot arrives in that account.  It seems most
> contributions are towards the OpenBSD Foundation.
>
> > I know that was discuss a few times on this list, just try to be clear
> > as it is now, and I can setup paypal and do recurring gifts to
> > compensate some for the sad CD sales reduction and if so, I sure would
> > encourage users to do the same so that you can continue to do what you
> > love and what we all benefit from obviously!
>
> Thanks.  I feel there have been too many attempts at rebooting this model
> every few years and it isn't going to work out long term.

Re: A branded USB stick as an alternative to the CD set?

[Theo de Raadt]

> Theo: like others in this thread I find it quite shocking and disappointing
> how poorly you are doing financially from your hard work.

Join the club :)

> I apologise if this is too obvious a suggestion but if the foundation is
> making a sufficient income is it not possible for you to draw a salary as
> an employee?

That is not my choice.  I think the Foundation is being wise.  The
relationship is that I keep them apprised of wants & needs the project
faces, and they fund those initiatives.  So take a look at their
report and books once they finish this year, to see if the money was
spent well, and where that leaves them.

I have not asked for their assistance, becuase I think they are making
the right decisions.

I don't know if there is enough left over a year to help me out.

I will not ask them.

pfstat - bits or bytes

[Christer Solskogen]

Hi!

In pfstat(8) the example is:
collect 1 = interface "sis0" pass bytes in ipv4 diff

but it also have this:
graph 1 bps "in" "bits/s" color 0 192 0 filled

Does pfstat record bits or bytes?

-- 
chs

Re: bridge fails to broadcast ARP from gif tunnel

[Reyk Floeter]

On Tue, Dec 01, 2015 at 10:07:12AM +0100, Kazuya GODA wrote:
> Hi,
> 
> It seems to bridge doesn't forward broadcast/multicast frames from gif.
> This pathc will fix this problem, so would you try it?
> 
> Thanks,
> 
> - Goda
> 

that matches the behaviour of -r1.239 before the enqueue changes.

OK reyk@

> Index: net/if_bridge.c
> ===
> RCS file: /cvs/src/sys/net/if_bridge.c,v
> retrieving revision 1.270
> diff -u -p -r1.270 if_bridge.c
> --- net/if_bridge.c   7 Nov 2015 12:42:19 -   1.270
> +++ net/if_bridge.c   1 Dec 2015 08:44:42 -
> @@ -1337,18 +1337,21 @@ bridge_process(struct ifnet *ifp, struct
>   if (mc == NULL)
>   goto reenqueue;
> 
> - bridge_ifinput(ifp, mc);
>  #if NGIF > 0
>   if (ifp->if_type == IFT_GIF) {
>   TAILQ_FOREACH(ifl, >sc_iflist, next) {
>   if (ifl->ifp->if_type != IFT_ETHER)
>   continue;
> 
> - bridge_ifinput(ifl->ifp, m);
> - return;
> + bridge_ifinput(ifl->ifp, mc);
> + break;
>   }
> - }
> + if (!ifl)
> + m_freem(mc);
> + } else
>  #endif /* NGIF */
> + bridge_ifinput(ifp, mc);
> + 
>   bridgeintr_frame(sc, ifp, m);
>   return;
>   }
> 
> 
> 
> 
> On 2015/11/28 15:33, Rolf Sommerhalder wrote:
> >Using the simple Layer-2 bridge setup below, an ICMP Ping 172.17.1.5
> >from HostA does not get to HostB while using EtherIP encapsulation with
> >gif(4) at its tunnel end points.
> >
> >The Ping's initial Ethernet broadcasts with the ARP Requests make it
> >through the gif tunnel to BridgeB, to both its bridge0 and vio2
> >interfaces (check with tcpdump, tshark).
> >
> >However, vio2 never re-broadcasts those ARP Requests on the wire to
> >HostB!? E.g. the physical egress interface vio2, which is member of a
> >bridge(4) on BridgeB, receives the ARP Requests, but it fails
> >re-broadcast them to HostB so that Host could answer with ARP Reponses.
> >
> >Also, BridgeB does not learn the source MAC from HostA (and of course it
> >can not learn the MAC of HostB, because ARP Requests never get there).
> >
> >However, pinging the (numbered) vio2 on BridgeB succeeds (Ping
> >172.17.1.2 from HostA), e.g. the gif tunnel is OK.
> >
> >Also, HostA can ping HostB after removal of the gif tunnel, e.g. after
> >deleting gif0 from bridge0 on both BridgeA and BridgeB, and adding vio1
> >to them instead.
> >
> >Testing conditions:
> >- default installs of OpenBSD i386 snapshot from yesterday
> >- pf is disabled
> >- no L2 filter rules on the bridge member interfaces
> >- set sysctl net.inet.etherip.allow=1 to enable EtherIP on gif()
> >- the observation is the same on both VirtualBox with vio() interfaces,
> >as well as on a real hardware with APU2 that have em() interfaces.
> >
> >Currently, experimenting with pf enabled on BridgeB, I found that ARP
> >Requests apparently do not generate state with a very basic rule-set,
> >such as 'pass log all'.
> >
> >What did I miss?  Or, is there "just a bug" in the gif/bridge combo that
> >is haunting me?
> >Would it be worthwhile to try with -stable or an older version of
> >OpenBSD?  Years ago, I had such a setup working with 4.3, and I can make
> >configuration files available (although they are very minimal, mostly
> >running default install) ...
> >
> >Thanks for any hints and suggestions!
> >Rolf
> >
> >
> >*HostA*
> >vio1 172.16.0.5/22
> >  |
> >  v
> >vio2 172.16.0.2/22
> >*BridgeA*
> >bridge0 add vio2 add gif0
> >gif0 tunnel 10.10.1.2 10.10.1.3
> >vio1 10.10.1.2/24
> >  |
> >  v
> >vio1 10.10.1.3/24
> >gif0 tunnel 10.10.1.3 10.10.1.2
> >bridge0 add vio2 add gif0
> >*BridgeB*
> >vio2 172.16.1.2/22
> >  |
> >  v
> >vio1 172.16.1.5/22
> >*HostB*
> 

-- 

HP LaserJet Problem

[bluesun08]

Hi,

i connected my HP LaserJet 1320 to a USB-Port. The message is:

ulpt0 at uhub1
openbsd /bsd:  port 4 configuration 1 interface 0 "Hewlett-Packard hp
LaserJet 1320 series" rev 1.10/1.00 addr 4
openbsd /bsd: ulpt0: using bi-directional mode

After the command "textfile" > /dev/ulpt0
i get the message:

cannot create /dev/ulpt0: Device busy

The printer don't print. What goes wrong here?

Regards Alex



--
View this message in context: 
http://openbsd-archive.7691.n7.nabble.com/HP-LaserJet-Problem-tp284216.html
Sent from the openbsd user - misc mailing list archive at Nabble.com.

OpenBSD 5.8 on VMware 5.5

[Felipe Gomes]

Folks,

I've been trying to search for more information on OpenBSD as a VMWare
guest, but I wasn't able to find much... and the information is pretty much
outdated.

What are the recommendations for OpenBSD 5.8 (amd64) as a guest on VMware
5.5?

Guest Operating System: should I pick "Other (64bit)" or FreeBSD?

How does OpenBSD work with "virtual sockets" and "cores per virtual socket"?

What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?

What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS
or VMware Paravirtual?

I'd believe that all of these options work... I just don't know which is
more stable or perform better.

Any other tips on fine tunning or special setting?

I'm planning on migrating a few Soekris boxes to virtual machines. Is this
reliable? Is anyone running production OpenBSD servers on VMware?

Thanks in advance!

Re: A branded USB stick as an alternative to the CD set?

[Peter J. Philipp]

On 12/01/15 10:20, Anthony Campbell wrote:
> On 30 Nov 2015, Bryan Vyhmeister wrote:
>> Let's not waste any more of Theo's time. USB sticks are not the magic
>> device that some seem to think. Some are not very reliable and prone to
>> failure. I've had very mixed results with budget USB sticks in
>> particular. Going with a more expensive USB stick like a major brand
>> name *usually* turns out better but that's still no guarantee. If you
>> don't want a CD set, simply donate the amount the CD set costs directly
>> to the project. That provides funding for OpenBSD while also not wasting
>> anyone's time.
>>
>> http://www.openbsd.org/donations.html
>>
>> Bryan
> As a UK resident, buying the CDs (which I don't need) would require me
> to pay VAT and delivery, neither of which (obviously) would benefit
> Theo. I therefore prefer to make a donation to the project.
>
>

I buy the CD's every time as I want to have a material reminder of every
release.  Thanks for pointing out the donation link it made me realize
that deraadt too takes paypal, so I don't need to shift to paypal'ing
the foundation, as I'm sick of doing bank transfers.

As far as the USB stick goes, I think it's a good idea, I'd buy it but
if it's too much effort and cost then don't worry about it.

-peter

Re: kernel panic - panic: ehci_device_clear_toggle: queue active

[Donald Allen]

The crash I reported a few days ago is the same:
ehci_device_clear_toggle: queue active

Re: A branded USB stick as an alternative to the CD set?

[Anthony Campbell]

On 30 Nov 2015, Bryan Vyhmeister wrote:
> Let's not waste any more of Theo's time. USB sticks are not the magic
> device that some seem to think. Some are not very reliable and prone to
> failure. I've had very mixed results with budget USB sticks in
> particular. Going with a more expensive USB stick like a major brand
> name *usually* turns out better but that's still no guarantee. If you
> don't want a CD set, simply donate the amount the CD set costs directly
> to the project. That provides funding for OpenBSD while also not wasting
> anyone's time.
> 
> http://www.openbsd.org/donations.html
> 
> Bryan

As a UK resident, buying the CDs (which I don't need) would require me
to pay VAT and delivery, neither of which (obviously) would benefit
Theo. I therefore prefer to make a donation to the project.


-- 
Anthony Campbellhttp://www.acampbell.uk

Re: 5.8 freezes on Shuttle DS87, anybody else?

[Harald Dunkel]

I migrated this openBSD setup to a 5 years old network
appliance. Its running for more than a week without problems.

This means I don't have a test setup to chase the problem
anymore.

Regards
Harri

Re: bridge fails to broadcast ARP from gif tunnel

[Kazuya GODA]

Hi, Rolf,

> Will you merge the fix into -current?
This fix was merged into -current.

Thanks,
- Goda

On 2015/12/01 11:20, Rolf Sommerhalder wrote:

Hi Goda,

On Tue, Dec 1, 2015 at 10:07 AM, Kazuya GODA  wrote:

It seems to bridge doesn't forward broadcast/multicast frames from gif.
This pathc will fix this problem, so would you try it?


Indeed, your patch fixes the problem! Excellent, thank you very much.

Now, I will go on and try IPsec with the same bridge test setup... :-)

Will you merge the fix into -current?

Thanks again,
Rolf

Re: OpenBSD 5.8 on VMware 5.5

[James Shupe]

On 2015-12-01 09:50, Felipe Gomes wrote:

Folks,

I've been trying to search for more information on OpenBSD as a VMWare
guest, but I wasn't able to find much... and the information is pretty 
much

outdated.

What are the recommendations for OpenBSD 5.8 (amd64) as a guest on 
VMware

5.5?

Guest Operating System: should I pick "Other (64bit)" or FreeBSD?

How does OpenBSD work with "virtual sockets" and "cores per virtual 
socket"?


What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?

What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic 
SAS

or VMware Paravirtual?

I'd believe that all of these options work... I just don't know which 
is

more stable or perform better.

Any other tips on fine tunning or special setting?

I'm planning on migrating a few Soekris boxes to virtual machines. Is 
this

reliable? Is anyone running production OpenBSD servers on VMware?

Thanks in advance!



It runs just fine for me. I use "Other (64bit)" and change the NICs to 
vmxnet3. Everything else remains the default.


--
James Shupe

Re: HP LaserJet Problem

[Stefan Sperling]

On Tue, Dec 01, 2015 at 07:37:05AM -0700, bluesun08 wrote:
> Hi,
> 
> i connected my HP LaserJet 1320 to a USB-Port. The message is:
> 
> ulpt0 at uhub1
> openbsd /bsd:  port 4 configuration 1 interface 0 "Hewlett-Packard hp
> LaserJet 1320 series" rev 1.10/1.00 addr 4
> openbsd /bsd: ulpt0: using bi-directional mode
> 
> After the command "textfile" > /dev/ulpt0
> i get the message:
> 
> cannot create /dev/ulpt0: Device busy
> 
> The printer don't print. What goes wrong here?
> 
> Regards Alex

Some HP LaserJet printers need firmware. See the ulpt(4) man page.
Your model is not listed there.

Does your printer need firmware?

Re: OpenBSD 5.8 on VMware 5.5

[Fabio Almeida]

Hi Felipe,

I'm running OpenBSD VMWare guests without problem, both as Firewall, IPSec
VPN and FTP/SFTP servers.
If you plan to run H.A systems with CARP, just be sure to enable
"promiscuous mode" on the carp interfaces, both on the VM and the
Hypervisor side.

Everything else you can let the default options, I have both 'vic' and 'em'
interfaces without problem.

regards,
Fabio Almeida


On Tue, Dec 1, 2015 at 1:50 PM, Felipe Gomes  wrote:

> Folks,
>
> I've been trying to search for more information on OpenBSD as a VMWare
> guest, but I wasn't able to find much... and the information is pretty much
> outdated.
>
> What are the recommendations for OpenBSD 5.8 (amd64) as a guest on VMware
> 5.5?
>
> Guest Operating System: should I pick "Other (64bit)" or FreeBSD?
>
> How does OpenBSD work with "virtual sockets" and "cores per virtual
> socket"?
>
> What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?
>
> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS
> or VMware Paravirtual?
>
> I'd believe that all of these options work... I just don't know which is
> more stable or perform better.
>
> Any other tips on fine tunning or special setting?
>
> I'm planning on migrating a few Soekris boxes to virtual machines. Is this
> reliable? Is anyone running production OpenBSD servers on VMware?
>
> Thanks in advance!

Re: OpenBSD 5.8 on VMware 5.5

[Reyk Floeter]

Hi,

On Tue, Dec 01, 2015 at 01:50:57PM -0200, Felipe Gomes wrote:
> I've been trying to search for more information on OpenBSD as a VMWare
> guest, but I wasn't able to find much... and the information is pretty much
> outdated.
> 
> What are the recommendations for OpenBSD 5.8 (amd64) as a guest on VMware
> 5.5?
> 
> Guest Operating System: should I pick "Other (64bit)" or FreeBSD?
> 

I usually pick FreeBSD 64 bit.  It doesn't make a big difference, as
there aren't any defaults that fit OpenBSD.  VMware never dares to add
OpenBSD, and we are not using their drivers but reimplementations.

> How does OpenBSD work with "virtual sockets" and "cores per virtual socket"?
> 

?  I think OpenBSD doesn't care.  GENERIC.MP will show you numbered
cpus, no matter if they are cores or sockets.  

> What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3?
> 

Name- OpenBSD driver:

e1000*  - em(4) (supports VLANs, but is kind of slow)
vmxnet2 - vic(4)(older NIC, no VLANs)
vmxnet3 - vmx(4)(emulates 10GbaseT, supports VLANs)

Use vmxnet3.

> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS
> or VMware Paravirtual?
> 

LSI Logic SAS   - mpi(4)
VMware Paravirtual  - vmwpvs(4)

Use LSI Logic SAS.  The VMware Paravirtual has bugs that might corrupt
your data (seen with fsck).

> I'd believe that all of these options work... I just don't know which is
> more stable or perform better.
> 

You will also have vmt(4) for limited VMware tools support.

> Any other tips on fine tunning or special setting?
> 

Tuning?  No, everything should work by default and is enabled in GENERIC[.MP]

> I'm planning on migrating a few Soekris boxes to virtual machines. Is this
> reliable? Is anyone running production OpenBSD servers on VMware?
> 

Many of them.

Reyk

Re: HP LaserJet Problem

[Stefan Sperling]

On Tue, Dec 01, 2015 at 05:12:26PM +0100, Stefan Sperling wrote:
> On Tue, Dec 01, 2015 at 07:37:05AM -0700, bluesun08 wrote:
> > cannot create /dev/ulpt0: Device busy
> > 
> > The printer don't print. What goes wrong here?
> > 
> > Regards Alex
> 
> Some HP LaserJet printers need firmware. See the ulpt(4) man page.
> Your model is not listed there.
> 
> Does your printer need firmware?

It's also possible that another program has already opened the ultp0 device.