Gina/Adityha, followup on donation request re OpenPower devices Re: Interest in POWER platform?

[Mikael]

Hi IBM,

This email followup was mostly to reflect that a member of the general
public has asked for the support that a donation of OpenPower devices would
lead to.

Feel free to forward it to the person at your company who has the power to
sign off on a donation e.g. your CEO.

This is a pro bono email.

I personally think Power9/Power8 support is a useful thing and if
10x2,850USD would have been less than say 0.2% of my wealth, I'd simply
have taken this matter through a web shopping cart.

Have a good day.

Thanks!,
Mikael More

2017-05-25 13:52 GMT+08:00 Mikael <mikael.ml...@gmail.com>:

> Hi Kai and IBM,
>
> Yes I did the attempts to contact IBM to get Power donations, as quoted by
> axon below. A guy by the name Benjamin Herrenschmidt at IBM used the word
> 'execrable' about me in PM, that was weird for a fund(hardware)raiser, and
> sincerely quite disturbing to me.
>
> (My 'attitude' - after 8 months and 70 emails they delegated the matter
> from their main office in the US, to Australia.)
>
> I think with respect to IBM donations, the relevant path would be to
> enquire directly with their CEO Ginni Rommety, e.g.
> https://www.ibm.com/ibm/ginni/ , also with reference to the massive
> commercial value they have from OpenSSH. Anyone below does not have the
> power to authorize donations.
>
> Gina and Adithya at IBM on copy for reference. If you can forward this to
> your CEO would be great. Your cheapest multi-CPU Power9 or Power8 device
> should be around 2000 USD production cost max, meaning this is a 20,000 USD
> donation request, or for 20,000 / 80,000,000,000 = 0.000,025% of your
> annual turnover, as a marketing and goodwill thing it couldn't be cheaper.
>
>
> If you have 6 to 10 devices - just any, preferably multi-CPU - to donate,
> please ship them over and we'll likely see support happen.
>
> If they need to be shopped, Tyan was selling them for 2850 USD a piece
> recently, https://web.archive.org/web/20160118065359/http://
> www.tyan.com/campaign/openpower/index.html . Maybe we're closer to Power9
> now.
>
> I like Power as it's server-grade hardware that I personally deem
> preferable to AMD64. IBM's attitude about my hardware-raiser initiative
> last year was execrable though.
>
> This email is a response to suggest a next step. I think everyone involved
> has been personally well-intended and there was a certain sense of friction
> in the realization that noone involved in the emails at IBM had the
> authority to sign off on a donation.
>
> People like you and me are free to shop and donate.
>
> Mikael
>
> 2017-05-25 1:57 GMT+08:00 Kai Wetlesen <kwetle...@mac.com>:
>
>> Hi all,
>>
>> What is the current community interest in getting OpenBSD running on the
>> newer POWER processors? I have a number of POWER based systems at work
>> which run various Linux flavors, but it would be nice to bring OpenBSD to
>> these systems as we’re been trying it out in different spaces throughout
>> our division. What would it take to get a POWER port up and kicking?
>>
>> Regards,
>> Kai
>>
>
> 2017-05-25 8:42 GMT+08:00 Ax0n <a...@h-i-r.net>:
>
>> In summary: There are 3 people who have been quite vocal about getting a
>> POWER port recently. None of them are developers with the knowledge or
>> resources to port it.
>>
>> Big thread from late last year:
>> https://marc.info/?l=openbsd-misc=147680858507662=2
>>
>> A follow-up (late December 2016):
>> https://marc.info/?l=openbsd-misc=148246956710299=2
>>
>> Search link with some scattered and often-unrelated results:
>> https://marc.info/?l=openbsd-misc=2=1=IBM+POWER=b
>>
>
> 2017-01-03 14:52 GMT+08:00 Benjamin Herrenschmidt <b...@au1.ibm.com>:
> ..
>
>> Right, and as I mentioned, we hope to have reasonably soon much more
>> affordable machines available as well, which will make it easier for us
>> to sponsor community projects with HW donations.
>>
>> Cheers,
>> Ben.
>>
>
>

Re: Interest in POWER platform?

[Mikael]

Hi Kai and IBM,

Yes I did the attempts to contact IBM to get Power donations, as quoted by
axon below. A guy by the name Benjamin Herrenschmidt at IBM used the word
'execrable' about me in PM, that was weird for a fund(hardware)raiser, and
sincerely quite disturbing to me.

(My 'attitude' - after 8 months and 70 emails they delegated the matter
from their main office in the US, to Australia.)

I think with respect to IBM donations, the relevant path would be to
enquire directly with their CEO Ginni Rommety, e.g.
https://www.ibm.com/ibm/ginni/ , also with reference to the massive
commercial value they have from OpenSSH. Anyone below does not have the
power to authorize donations.

Gina and Adithya at IBM on copy for reference. If you can forward this to
your CEO would be great. Your cheapest multi-CPU Power9 or Power8 device
should be around 2000 USD production cost max, meaning this is a 20,000 USD
donation request, or for 20,000 / 80,000,000,000 = 0.000,025% of your
annual turnover, as a marketing and goodwill thing it couldn't be cheaper.


If you have 6 to 10 devices - just any, preferably multi-CPU - to donate,
please ship them over and we'll likely see support happen.

If they need to be shopped, Tyan was selling them for 2850 USD a piece
recently, https://web.archive.org/web/20160118065359/http://
www.tyan.com/campaign/openpower/index.html . Maybe we're closer to Power9
now.

I like Power as it's server-grade hardware that I personally deem
preferable to AMD64. IBM's attitude about my hardware-raiser initiative
last year was execrable though.

This email is a response to suggest a next step. I think everyone involved
has been personally well-intended and there was a certain sense of friction
in the realization that noone involved in the emails at IBM had the
authority to sign off on a donation.

People like you and me are free to shop and donate.

Mikael

2017-05-25 1:57 GMT+08:00 Kai Wetlesen <kwetle...@mac.com>:

> Hi all,
>
> What is the current community interest in getting OpenBSD running on the
> newer POWER processors? I have a number of POWER based systems at work
> which run various Linux flavors, but it would be nice to bring OpenBSD to
> these systems as we’re been trying it out in different spaces throughout
> our division. What would it take to get a POWER port up and kicking?
>
> Regards,
> Kai
>

2017-05-25 8:42 GMT+08:00 Ax0n <a...@h-i-r.net>:

> In summary: There are 3 people who have been quite vocal about getting a
> POWER port recently. None of them are developers with the knowledge or
> resources to port it.
>
> Big thread from late last year:
> https://marc.info/?l=openbsd-misc=147680858507662=2
>
> A follow-up (late December 2016):
> https://marc.info/?l=openbsd-misc=148246956710299=2
>
> Search link with some scattered and often-unrelated results:
> https://marc.info/?l=openbsd-misc=2=1=IBM+POWER=b
>

2017-01-03 14:52 GMT+08:00 Benjamin Herrenschmidt <b...@au1.ibm.com>:
..

> Right, and as I mentioned, we hope to have reasonably soon much more
> affordable machines available as well, which will make it easier for us
> to sponsor community projects with HW donations.
>
> Cheers,
> Ben.
>

Any efforts to make sector size configurable file/disk IO subsystem locking?

[Mikael]

Hi!

Are there any efforts to decrease the internal locking in the file/disk IO
subsystem, and perhaps make the sector size configurable?

My quest is to get anything in the ballpark of the speeds I get on
/dev/rsd0c (~575MB/sec on aligned random 16KB) on ordinary noncached
filesystem access (which I experience as capped around 120MB/sec
independent of everything, in my tests on a single SATA or NVME SSD).

I will need to make additional benchmarks where I try to max two disks
concurrently to see if the peak read speed I get is still 120MB/sec then
i.e. that is system-wide, however I think that is the case.

(First someone suggested in chatroom that the ~120MB/sec cap on all threads
on disk IO is because of the absence of multiqueueing support, but then if
I understood David Gwyne right, he's saying it's not but instead somehow
about the overhead of the IO subsystem itself and the solution to that is
general multithread-ization of the code.

I was asking myself if overhead also may be due to that the whole
filesystem always leads to per-512-byte accesses to the underlying media
too. My SSD benchmarks seem to suggest that aligned 16KB reads perform the
best both in both random and sequential modes.)

Thanks!
Mikael

Re: OpenBSD 6.1 - bravo

[Mikael]

Also from me, big thanks!

2017-04-12 16:45 GMT+08:00 Clément.J :

> Thank you OpenBSD team for this new release 6.1
> OpenBSD makes me happy every day for so many usages
> so thank you so much everyone for your great work.


2017-04-12 16:45 GMT+08:00 Clément.J :

> Thank you OpenBSD team for this new release 6.1
> OpenBSD makes me happy every day for so many usages
> so thank you so much everyone for your great work.
>
> have a good day
> vive OpenBSD
>
>
> Le 12-04-2017 10:27, Stuart Henderson a écrit :
>
>> On 2017-04-12, Jordon  wrote:
>>
>>> rcctl enable dhcrelay
 rcctl set dhcrelay flags -i athn0 192.168.1.1 "assuming that is
 your routers

>>> address"
>>>
 rcctl start dhcrelay

 and possibly add -d (log to stderr) to see what its doing.


>>> Thank you!  That got it working!  So why is that necessary?  Doesnt
>>> the bridge
>>> just forward everything?  Or are DHCP requests broadcasts that dont
>>> get
>>> forwarded?
>>>
>>
>> It shouldn't be necessary, dhcrelay is normally used when you have a
>> subnet behind a router, and the DHCP server is a separate machine on
>> a
>> different subnet.
>>
>> Could it be a PF rule problem?
>>
>> Normally you would only have an IP address on one member of the
>> bridge,
>> just "up" on the others..

Re: Per-device multiqueuing would be fantastic. Are there any plans? Are donations a matter here?

[Mikael]

Bump

2017-02-10 22:11 GMT+08:00 Mikael <mikael.ml...@gmail.com>:

> 2017-02-10 18:39 GMT+08:00 David Gwynne <da...@gwynne.id.au>:
>
>> > 2017-02-09 16:41 GMT+08:00 David Gwynne <da...@gwynne.id.au>:
>>
>  ..
>
>> i can go into more detail if you want.
>>
>> cheers,
>> dlg
>>
>
> Hi David,
>
> Thank you - yes please go into more detail!
>
> Also on a more concrete level I would be the most curious to understand
> how far away we would be from having concurrent IO access (e.g. on various
> distances from the hardware: via some lowlevel block device API that would
> be mp-safe / via open("/dev/rsd0c") + read() that would be mp-safe / that
> the filesystem would be mp-safe so open("myfile") + read() would be all
> concurrent)?
>
> Then, just to get an idea of what's going on, regarding how the system is
> doing concurrent IO, or how you like to call it: While I had a
> pre-understanding that the kernel's big lock could be playing in so that it
> would be a performance constraint, the people I talked to seemed to suggest
> that the kernel is doing IO in a procedure-call-based code-blocking,
> synchronous way today, which only runs one command in the sata comand queue
> concurrently, and so I got an impression that concurrency-friendly IO and
> higher IOPS would require a reconstruction of the IO system where IO
> operations are represented internally rather by data structures run by an
> asynchronous mechanism. How is it actually, and where is it going?
>
> Regarding mp-safe drivers, I would guess ahci.4 and nvme.4 are the most
> commonly used interfaces for SSD:s.
>
> Best regards,
> Mikael

Re: Per-device multiqueuing would be fantastic. Are there any plans? Are donations a matter here?

[Mikael]

2017-02-10 18:39 GMT+08:00 David Gwynne <da...@gwynne.id.au>:

> > 2017-02-09 16:41 GMT+08:00 David Gwynne <da...@gwynne.id.au>:
>
 ..

> i can go into more detail if you want.
>
> cheers,
> dlg
>

Hi David,

Thank you - yes please go into more detail!

Also on a more concrete level I would be the most curious to understand how
far away we would be from having concurrent IO access (e.g. on various
distances from the hardware: via some lowlevel block device API that would
be mp-safe / via open("/dev/rsd0c") + read() that would be mp-safe / that
the filesystem would be mp-safe so open("myfile") + read() would be all
concurrent)?

Then, just to get an idea of what's going on, regarding how the system is
doing concurrent IO, or how you like to call it: While I had a
pre-understanding that the kernel's big lock could be playing in so that it
would be a performance constraint, the people I talked to seemed to suggest
that the kernel is doing IO in a procedure-call-based code-blocking,
synchronous way today, which only runs one command in the sata comand queue
concurrently, and so I got an impression that concurrency-friendly IO and
higher IOPS would require a reconstruction of the IO system where IO
operations are represented internally rather by data structures run by an
asynchronous mechanism. How is it actually, and where is it going?

Regarding mp-safe drivers, I would guess ahci.4 and nvme.4 are the most
commonly used interfaces for SSD:s.

Best regards,
Mikael

Re: Per-device multiqueuing would be fantastic. Are there any plans? Are donations a matter here?

[Mikael]

2017-02-09 16:41 GMT+08:00 David Gwynne <da...@gwynne.id.au>:
..

> hey mikael,
>
> can you be more specific about what you mean by multiqueuing for disks?
> even a
> reference to an implementation of what you’re asking about would help me
> answer this question.
>
> ill write up a bigger reply after my kids are in bed.
>
> cheers,
> dlg


Hi David,

Thank you for your answer.

The other OpenBSD:ers I talked to also used the wording "multiqueue". My
understanding of the kernel's workings here is too limited.

If I would give a reference to some implementation out there, I guess I
would to the one introduced in Linux 3.13/3.16:

"Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems"
http://kernel.dk/blk-mq.pdf

"Linux Multi-Queue Block IO Queueing Mechanism (blk-mq)"
https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mech
anism_(blk-mq)

"The multiqueue block layer"
https://lwn.net/Articles/552904/

Looking forward a lot to your followup.

Best regards,
Mikael

Re: SSD read performance benchmark OpenBSD 6.0 vs. Linux 4.7: OpenBSD will benefit of multiqueueing and also a speedup for sequential reads, and Linux' mmap() is extremely slow for random reads.

[Mikael]

2017-02-09 Mikael <mikael.ml...@gmail.com>:

> Dear misc@,
>
> *## Intro, environment*
> Find below a comparative benchmark of OpenBSD 6.0 vs Linux 4.7 read speeds
> on a 3.3Ghz Xeon E3 server with a Samsung 850 Pro 256GB SATA SSD, which is
> one of the very fastest SSD:s in the sub-1000USD/TB price range. dmesg
> below.
>
[..]

Someone reminded me to benchmark rsd0c vs. sd0c on OpenBSD, and there are
some great surprises in there!:

Random multithreaded actually goes up to 198MB/sec at 4KB, and 578MB/sec at
16KB, and sequential multithreaded follows the same curve.

So OpenBSD's current IO subsystem gives ~31,000 IOPS for 16KB random and
sequential reads, multithreaded.

This is excellent and shows that a custom database program on current
OpenBSD indeed can utilize all bandwidth of a SATA SSD!

The 4K multithreaded random and sequential read performs at 2/3 of Linux,
which is also not bad.

The natural next step in understanding SSD performance would be to
benchmark two SATA SSD:s (on separate SATA plugs), on OpenBSD rsd0c,
OpenBSD sd0c, and Linux sd0 .


*Benchmark details*

Unlike sd0c , rsd0c cannot be mmap():ed.

Also, rsd0c requires reads to be aligned? - which is fine for any database
usecase as these are based on internal pages anyhow.

Sequential singlethreaded: 4KB is 121MB/sec, 16KB is 225MB/sec, and 64KB+
is 342MB/sec.

Sequential 10-threaded: 4KB is 19MB/sec per process so 190MB/sec total (vs.
sd0c's 48MB/sec total), 16KB is 50MB/sec per process so 500MB/sec total
(vs. sd0c's 150MB/sec total), 64KB is 53MB/sec so 530MB/sec total (vs.
sd0c's 288MB/sec total).

Random singlethreaded: 4KB is 45MB/sec (so same as sd0c), 16KB is 132MB/sec
(vs. sd0c's 52MB/sec), 32KB is 198MB/sec (vs. sd0c's 67MB/sec), 64KB is
264MB/sec (vs. sd0c's 85MB/sec).

Random 10-threaded 4KB is 19.6MB/sec per process so 196MB/sec total (yey,
that's 4x sd0c!), 16KB is 48MB/sec per process so 480MB/sec total, 32KB is
51MB/sec per process so 510MB/sec, and 64KB is 53MB/sec per process so
530MB/sec.

Random 20-threaded 4KB is 9.9MB/sec per process so 198MB/sec, 16KB is
24.8MB/process so 496MB/sec total, 32KB is 25.9MB/process so 518MB/sec.

Random 40-threaded 4KB is 4.95MB/process so 198MB/sec total, 16KB is
14.45MB/process so 578MB/sec total, 32KB is 12.99MB/process so 519MB/sec.

Per-device multiqueuing would be fantastic. Are there any plans? Are donations a matter here?

[Mikael]

Hi misc@,

The SSD reading benchmark in the previous email shows that per-device
multiqueuing will boost multithreaded random read performance very much
e.g. by ~7X+, e.g. the current 50MB/sec will increase to ~350MB/sec+.

(I didn't benchmark yet but I suspect the current 50MB/sec is system-wide,
whereas with multiqueuing the 350MB/sec+ would be per drive.)

Multiuser databases, and any parallell file reading activity, will/would
see a proportional speedup with multiqueing.


Do you have plans to implement this?

Was anything done to this end already, any idea when multiqueueing can
happen?


Are donations a matter here, if so about what size of donations and to who?

Someone suggested that implementing it would take a year of work.

Any clarifications of what's going on and what's possible and how would be
much appreciated.


Thanks,
Mikael

SSD read performance benchmark OpenBSD 6.0 vs. Linux 4.7: OpenBSD will benefit of multiqueueing and also a speedup for sequential reads, and Linux' mmap() is extremely slow for random reads.

[Mikael]

ular sysctl:s or other kernel settings speed up random
   or sequential reading?


   - It's interesting to see that an SSD can be made to deliver so much
   random read performance simply by using multiqueues (e.g. 10X higher). It
   reflects that SSD:s as devices are largely parallellized internally.


   - The OpenBSD buffer cache containing a lot of irrelevant data will not
   affect read speeds negatively (that is on this AMD64 with its 32bit buffer
   size limit, Theo said otherwise about Sparc64 though) -

   I conclude this as benchmarks gave the same results with
   kern.bufcachepercent=5 and kern.bufcachepercent=90 settings (on this
system
   with 16GB RAM on a 256GB SSD and the benchmark being constructed to
   minimize buffer cache utilization through random lseek()).


Details follow below.

Best regards,
Mikael


*## Detail benchmark results*

*# OpenBSD results*
System: GENERIC.MP, 6.0, AMD64, dmesg below.

Sequential reads (single-threaded) [1] are around 120MB/sec all the way
down to 128-byte reads, and below it it dumps quickly (33MB/sec at 64byte
reads etc.) [2].

Random reads, single-threaded are: 512B/read [3] is ~5.8MB/sec, 4KB/read
[4] is ~45MB/sec,  and 32KB/read [5] is ~67MB/sec.

read() and mmap() give essentially the same performance in these two tests.

Random reads, 10-threaded, 20-threaded and 40-threaded, give the same total
throughput as the single-threaded results above i.e. there is no benefit at
all in running multiple concurrent IO operations:

Random reads, 10-threaded are: 512B/read [6] is 5.0MB/sec total (0.50MB/sec
per process), 4KB/read [7] is 44MB/sec (4.4MB/sec per process), and
32KB/read [8] is 72MB/sec (7.2MB/sec per process).

Random-read 20-threaded yielded somehow similar results (1.79MB/sec per
process meaning 35.8MB/sec at 4KB) [9].

Random-read 40-threaded yielded somehow similar results (0.81MB/sec per
process meaning 32.4MB/sec - that's [9] but doubled).

We also do a silly "10-threaded sequential read" (with all reads
originating at different offsets) [10], yielding 3.9MB/sec per process
meaning 39MB/sec total, so that's similar to single-threaded sequential
read.

*# Linux results*
System: "Linux version 4.6.0-kali1-amd64 (de...@kali.org) (gcc version
5.4.0 20160609 (Debian 5.4.0-6) ) #1 SMP Debian 4.6.4-1kali1 (2016-07-21)",
installed from
https://images.offensive-security.com/kali-linux-2016.2-amd64.torrent .

The first observation on Linux is that mmap() performance for random reads,
is WAY below that of read(): Singlethreaded random read: 512B: 1.45MB/sec
mmap() vs. 6.38MB/sec read(), 4K: 11MB/sec mmap) vs. 49MB/sec read(), 32KB:
86MB/sec mmap() vs. 230MB/sec read(), 10-threaded 4K random read:
1.79MB/sec per process mmap() vs. 35MB/sec per process read() [0].

In sequential reads on Linux however, mmap() performance is a bit better
than that of read() - on >=128byte reads, performance is equal, and on
reads below 100 bytes, mmap() access speed is much closer to the 128byte
performance profile, than that of read(). Proportionally speaking OpenBSD
is faster here.

All the following tests are done with read():

Sequential reads (single-threaded) [1] are around 525MB/sec all the way
down to 128-byte reads, and below it it dumps quickly (around 340MB/sec at
64byte reads etc.) [2].

Random reads, single-thread are: 512B/read [3] is 6.38MB/sec, 4KB/read [4]
is 50MB/sec, and 32KB/read [5] is 231MB/sec.

Random reads, 10-threaded are: 512B/read [6] is 46MB/sec total (4.6MB/sec
per process), 4KB/read [7] is 357MB/sec (35.7MB/sec per process), and
32KB/read [8] is 528MB/sec (52.8MB/sec per process).

Random-read 20-threaded yielded somehow similar results (19.8MB/sec per
process meaning 396MB/sec at 4KB) [9].

Random-read 40-threaded yielded somehow similar results (10MB/sec per
process meaning 400MB/sec at 4KB - that's [9] but doubled).

We also do a silly "10-threaded sequential read" (with all reads
originating at different offsets) [10], yielding 53.6MB/sec per process
meaning 536MB/sec total, so that's similar to single-threaded sequential
read.


*## Benchmark commands*
The main difference between OpenBSD test and Linux tests is that OpenBSD
uses "sd0c" while Linux uses "sda".

*# OpenBSD benchmark commands*
[1]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096

[2]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 64 64

[3]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512

[4]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096

[5]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768

[6]
sysctl kern.bufcachepercent=5
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \
nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1

IBM Power roundup for the year: Inaction, likely due to stupid greed from budget managers

[Mikael]

t if I-we do this-whatever then they will be impelled to, and so
on. They are perfectly entitled to suggest anyone to go through any hoops
for them to care or maybe care, however, that does not inspire any sense of
material goodwill or reason to care, with me.

They admitted that there is a real lack of organization and structure
within IBM to deal with anything else than selling hardware, as seen by the
total ignorance and cluelessness that I was met by for months on end.

My conclusion up to this point is that all the people in charge of IBM's
Power architecture are strapped in a very hierarchic slow structure that
gives them no mandate or space for anything.

At the same times, they are the very executives and managers who are
supposed to make the architecture fly.

Meaning, technically they are well intended, but they don't have any powers
whatsoever.

Me suggesting these particular people to donate hardware in a situation
like that is obviously like asking a starving child on the savannah to hand
you a sausage and an icecream - torturous.

While admittedly the hardware donation request I presented to them is
unusual, it's by no means extraordinary, and it's important to see that in
a multifactorial world that presents paramount competition, a structure
must provide some flexibility beyond that of ceramic, to really live.

A strict doing-anything-for-profit is not a winner. A café whose policy is
to force people to order before sitting down on their chairs, will not
thrive.


*Impressions of stupid greed from stakeholders*
My conclusion for the time being is that IBM's stakeholders have basically
strapped their organization and hierarchy of everything - a massive love
deprivation - yet expect their child to be singing and performing.

And so currently they cherish themselves as winners in their practice of
"blind greed", and, totally miss slightly more subtle qualities in the
market, such as the power of the influencer role that all people who
actually use their stuff, have.

And in this line of reasoning, they think it's OK to use OpenSSH from
OpenBSD, across the board, and reap benefit of that, while giving
"f**k-you":s to the same people when asked for coffee money.

These are the people who sense entitlement to force a sausage or icecream
from anyone.


I will be glad to follow how IBM handles this situation, and indeed ten
devices might just show up on the mail someday.

They have perfectly free will and are under no obligation to do anything,
and I perfectly respect that.

With that said, I do have a sense of "blind greed" going on among the
people at IBM who do have the authority to donate, and I suggest that the
universal honor system indeed takes careful notice of that, and damages
that are difficult-to-impossible are incurred all over the place, in
various forms such as negative internal marketing at IBM itself (i.e.
waking up to the realization that unnice people live upstream in your
building), bad PR with people for whom IBM matters such as myself who carry
these experiences on everywhere including to those who make buying
decisions, and finally strategically, in the way that the fierce
competition is getting advantages with every day that goes. This is the
very force that makes old business go out of business, and new humble
business thrive.

So, shame on you, people managing IBM's budgets, for being so short-sighted
that you break things for yourselves and others, without need.

"Me-me-me" strategies may have been compatible with business success some
decades ago. You are naturally entitled to run such strategies, but I don't
see the viability as lasting.

Please do improve in 2017.


*Summary*
I do like the Power architecture as a useful alternative or replacement to
crappy AMD64.

I have donated something like 70 emails and 10 hours of time this year to
suggest to them that donating some hardware to get more architecture
support is the very cheapest marketing they'll get ever in perpetuity.

The Power tech is useful. It would be nice to see donations happen. I will
be the happiest to post public thanks to IBM for doing the right thing. As
of yet I have no indication whatsoever that they will do that.

I would be glad to be positively surprised in this respect in 2017. Of
course others can donate too. In either case, the world will go on.

Mikael

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-20 1:15 GMT+08:00 Ralph Siegler :
..

> Their ecosystem?
>
> closed source softwares including for x86-64 like Websphere, DB2, MQ,
>
..

> Hardware platforms limited to Power ($11,000 and up), Z series ($60,000
>

A silly example of interest in the Power architecture that's certainly not
typical IBM enterprise apps and chassis:
http://www.theregister.co.uk/2016/04/07/open_power_summit_power9/

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-20 1:15 GMT+08:00 Ralph Siegler <rsieg...@rsiegler.org>:

> On Wed, 19 Oct 2016 23:50:11 +0800, Mikael wrote:
>
> > 2016-10-19 23:18 GMT+08:00 Ralph Siegler <rsieg...@rsiegler.org>:
> >
> >> On Wed, 19 Oct 2016 15:18:02 +0200, Otto Moerbeek wrote:
> >>
> >> > Director of the Power(8) Ecosystem & Alliances,
> >>
> >>
> >> > "It would be helpful to know where you are seeing requests for
> >> > OpenBSD
> >> on
> >> > Power and what applications on top of OpenBSD are being requested. We
> >> have
> >> > not seen any requests as of yet from our target clients. "
> >>
> >> I really don't think the eternal optimists here in openbsd.misc
> >> understand the real meaning behind her words, so I will be the cruel
> >> and heartless one that translates those kind and diplomatic words to
> >> Mikael into the common tongue.  What she was really saying was:
> >>
> >>  (colloquial vernacular for self-pleasuring as obtainable alternative
> to receiving IBM donations)
> >>
> > I see what you mean and I am unconvinced that you are correct about your
> > suggestion.
> >
> > They are a for-profit though. I guess we'll see over time how much their
> > heart is on their ecosystem (vs. only on individual large customers), as
> > in, if it is, then OpenBSD is very interesting for them, and if not,
> > not.
>
> Their ecosystem?
>
> closed source softwares including for x86-64 like Websphere, DB2, MQ,
> Domino, various compilers etc.  None of that will run on OpenBSD (maybe
> slim chance of getting some of it running on FreeBSD as kiddie science
> project but would be unsupported by IBM so no business would even
> consider that)
>
> Hardware platforms limited to Power ($11,000 and up), Z series ($60,000
> and up), LinuxOne (power with containers by usage, $5,800 a month and
> up)
>
> IBM storage
>
> IBM Services - tech agnostic  consulting
>
> I think they know  their ecosystem.  Not seeing anything for or related
> to OpenBSD in there.
>
>
Ralph,

My best understanding is that IBM have widened the scope of their Power
platform, to become something of a mainstream server platform.

This is why they founded the OpenPower foundation [1], and why they are
allying with lots of otherwise unrelated companies such as Tyan, and why
the Talos desktop is coming up.

I have not kept track of them in detail, these are my best guesses.

If that's their central strategy, OpenBSD is a natural part of that and
also it should be natural for them to donate some hw.

[1] https://en.wikipedia.org/wiki/OpenPOWER_Foundation
https://openpowerfoundation.org/

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-19 23:18 GMT+08:00 Ralph Siegler <rsieg...@rsiegler.org>:

> On Wed, 19 Oct 2016 15:18:02 +0200, Otto Moerbeek wrote:
>
> > Director of the Power(8) Ecosystem & Alliances,
>
> >
> > "It would be helpful to know where you are seeing requests for OpenBSD
> on
> > Power and what applications on top of OpenBSD are being requested. We
> have
> > not seen any requests as of yet from our target clients. "
>
> I really don't think the eternal optimists here in openbsd.misc
> understand the real meaning behind her words, so I will be the cruel and
> heartless one that translates those kind and diplomatic words to Mikael
> into the common tongue.  What she was really saying was:
>

I see what you mean and I am unconvinced that you are correct about your
suggestion.

They are a for-profit though. I guess we'll see over time how much their
heart is on their ecosystem (vs. only on individual large customers), as
in, if it is, then OpenBSD is very interesting for them, and if not, not.

Re ARM64 server hw availability and low-end Sparc64

[Mikael]

Splitting this to a separate thread just not to pollute the Power8 thread.

This relates to OpenBSD in the way that it's on the topic of what server
hardware exists that we possibly could run it on. So bordering to off-topic.

2016-10-19 14:34 GMT+08:00 Karel Gardas <gard...@gmail.com>:

> On Wed, Oct 19, 2016 at 7:23 AM, Mikael <mikael.ml...@gmail.com> wrote:
> >
> > Oracle have been talking about making a low-end server model of their new
> > Sparc64 chip, I guess that one will sell at around 5000 USD too.
>
> I guess you talk about so called Sonoma/scale-out SPARCs, well those
> were already unveiled and the prices start on > $11k -- see for
> example S7-2 box configurations.
>

Sonoma was what I was talking about yes.

My best understanding is that S7-2 is their high-end line and those are
>$11k indeed, and that no Sonoma devices have been released to the market
yet, only pre-announced e.g. http://www.theregister.co.uk/2015/08/24/oracle_
sonoma_processor_sparc/ .


I didn't see any prices.

Since their normal price point is >$11k and this should be "much cheaper",
then my wild guess was that that probably should mean something like $5k.



> > (I didn't see any convincing ARM64 servers on the market yet.)
>
> I'm not sure, but various of those already looks quite nice, will not
> break the account nor power-bill and still be even comparable to
> POWER7/8 in terms of raw C compilation power. By comparable I mean
> they are not 10 times (or more) slower or so
>

The closest to a existing, usable, buyable ARM64 ECC server I saw is
Gigabyte's Cavium and X-Gene1 offers:

Cavium:
http://b2b.gigabyte.com/products/product-page.aspx?pid=5926 1U
http://b2b.gigabyte.com/products/product-page.aspx?pid=5925 1U
http://b2b.gigabyte.com/products/product-page.aspx?pid=5927 2U

X-Gene1:
http://b2b.gigabyte.com/products/product-page.aspx?pid=5912 mobo

But, I didn't see them actually for sale anywhere, neither did I see a
concrete price tag.

If you saw anything interesting feel free to share.

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-19 12:59 GMT+08:00 Ralph Siegler :
..

>  too expensive to have for development, too expensive to run, to
> expensive for a userbase while businesses waited for a mature version, no
> compelling use case in the open source world that couldn't be done with
> Xeon drawing half to a third the power.
>

Ralph,

At 2850 to 5300 USD for rack chassi+mobo+CPU, I think Power8 servers do
make sense - it seems like a great way to diversify from AMD64, while still
in a robust server architecture.

Oracle have been talking about making a low-end server model of their new
Sparc64 chip, I guess that one will sell at around 5000 USD too.

So then there's two alternative architectures to AMD64, great!

(I didn't see any convincing ARM64 servers on the market yet.)

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-19 6:51 GMT+08:00 Ralph Siegler :
 ..

> no one is going to buy box from product line that starts at $11,000 (non-
>

Power8 machine offers start at USD 2,850:
http://www.tyan.com/campaign/openpower/index.html

And their standard prices are USD 5,530 and up, that is
http://www.tyan.com/Barebones_TN71-BP012_BSP012T71V14HR-4T-3 .

Re: Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

2016-10-19 0:48 GMT+08:00 Kapetanakis Giannis :
>
> pf, relayd, bgpd ;)
>
> G
>
> ps. after the unlocking
>

Giannis, this is too little info to be useful.

Please describe the practical and technical utility and value, the
organization/social context, scope, duration, anything that is relevant to
motivate them.

Right now they have no idea so this is to inform them.

Would you use OpenBSD on Power8, and if so what applications? (IBM asks! They're thinking about donating hw.)

[Mikael]

Hi everyone,

I asked IBM to donate 4-10 Power8 servers to the OpenBSD Foundation, for
adding support for this arch. After 6 months this got all the way to their
Director of the Power(8) Ecosystem & Alliances, that is the highest
executive for the whole arch. Just right now, she's asking for a motivation
for IBM to donate - she asks:


"It would be helpful to know where you are seeing requests for OpenBSD on
Power and what applications on top of OpenBSD are being requested. We have
not seen any requests as of yet from our target clients. "


Can you please collect answers to this question and post them here in this
thread, or PM them to me. I'll forward your responses and they'll decide
whether to donate Power8 devices to OpenBSD, based on them.

** Please tell the next 6-7 days!

Thanks!
Mikael

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

Thanks everyone for clarifying that the sectors 0 to 79 should be
considered reserved on OpenBSD MBR partitions (and thus within the context
of the "disklabel" tool).


There is just one thing I don't understand now, and that is what
dysfunction-potential there is in inappropriate handling of sectors 64 to
79.

I'll specify my Q below - so what do you say?

Last, I agree that the concept of "binary blob partitions" is slightly
archaic, but not by any standards obsolete.

Probably the most common blob-level use of partitions you do would be
"dd"-based backuping - and we learned here now, that restoring such a
partition backup could trash the disklabel if it started at less than
offset = 512 * 8 !

Thanks



*Impression:*
Based on what Benny and I think someone else said, I got an approximative
impression something like that the whole disklabelling system is actually
designed with the intention that every disklabel is required to

1) Have an "a" partition that
2) Starts on sector 64 and continues at least up to and including sector 79
and
2) Be of the 4.2BSD or RAID type,

Because otherwise something bad happens.

*Example, for clarity:*
I mean, if on an 999-sector disk where I want to put an 9-sector
binary blob in its own partition, and then give all the remaining space to
a "main" UFS-partition, if it wasn't for this conversation I'd just have
gone into the "disklabel" tool, wipe the disklabel and then first have
added my blob partition at the beginning of the disk and then the "main"
partition after it (i.e. blob partition would be sectors 64 to 100063 and
the UFS partition would be sectors 100064 to 998).

This conversation shows that the binary blob partition should be put at
sectors >= 80 (i.e. sector 80 to 100078 would have worked for it, so then
the UFS partition at sectors 100079 to 998), BUT,

Because of what possible dysfunction is it that I should put the UFS
partition in the beginning and the binary blob partition after it (i.e. UFS
partition on sector 64 to 981, and blob partition on sector 990 to
998)?


2015-10-07 2:48 GMT+08:00 Benny Lofgren :
...

> If you place a 4.2BSD or a RAID partition from the first sector (64) of
> the BSD usable part of the disk (as limited by the "boundstart" and
> "boundend" disklabel parameters) you will never get in trouble. Anything
> else and you need to know more about the system internals to be safe.
>
> You ask about saved butts. Just a couple of example scenarios:
>
> Consider the option to encasing the partition metadata in the first
> partition (and again, first physical address-wise, not first partition
> letter-wise), which is to leave a gap in the beginning: First of all,
> I'm not even sure the boot code is able to cope with that scenario, but
> let's say it is. Whenever you want to add a new partition, disklabel
> will suggest you populate the first free area on your disk - guess where
> that is going to be...
>
> If you wreck your disklabel and have done something non-standard, such
> as move the first partition from its expected position, you're pretty
> much in the dark when it comes to actually *find*, not to mention
> hopefully recover, your partitions.
>
> It is well known and understood since decades what's on these first
> sectors of a) a disk, b) of the BSD usable area and c) of each partition
> (type). Why are you having trouble accepting that things are the way
> they are and that they WORK as they are, if you don't turn everything on
> its head? Well, at least they work until you start messing with things
> you have not yet had any experience with. The upside is you'll quickly
> gain useful experience, of how not to do. :-)

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 1:38 GMT+08:00 Ted Unangst <t...@tedunangst.com>:

> Mikael wrote:
> > 2015-10-07 0:58 GMT+08:00 Ted Unangst <t...@tedunangst.com>:
> > >
> > > the disklabel is the second sector of the openbsd part of the disk.
> > >
> > > *3: A6  0   1   2 - 243200 254  63 [  64:  3907024001 ]
> OpenBSD
> > >
> > > so, if you overwrite sector 65, you will overwrite disklabel. normally
> the
> > > 'a'
> > > partition overlaps the disklabel, but you made 'e' first.
> > >
> >
> > Ugh, ok - just to settle this one forever I hope, four brief Q:s:
> >
> > 1) Does this mean that on an ordinary disk (where the "a" partition is
> the
> > disk's first partition, and starts at the offset autogenerated as default
> > option by the "disklabel" tool), the start of the "a" partition" actually
> > overlaps with disklabel-internal data?
>
> Yes. So, the metadata on a x86 disk will look like:
>
> |start -- end|
> |MBR . somestuff | OpenBSD partition |
> | .. | disklabel |
> | .. | FFS 'a'  | swap 'b' --- | FFS 'd' |
>
> FFS knows it should not use the first few sectors of a partiton, because
> other
> things may live there.
>
> This may not be obvious to start, but if you think about it, these data
> structures have to live *somewhere*. If you cannot see reserved space
> between
> areas of disk, then the reserved space must be somewhere inside those
> areas.
>

Ah sure.

Perhaps I misunderstood the level of "foolproofness" that the disklabel
tool's autogenerated default value was intended to give -

Just curious, now that structural things like this are at stake (i.e. some
user making a seemingly reasonable presumption based on the "UI" should
give you partitions where you can really have completely
user-defined/custom stuff based on an expectation that the autogenerated
values guarantee non-overlapping with other partitions or the disk
partition structure itself i.e. the disklabel), what's the motivation for
not increasing the disklabel tool's minimum autogenerated offset value from
64 to 80?

The tradeoff would be that the world would lose 8KB per disk, and win the
absence of needing to debug a complete disk crash, that I just underwent.

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

Aha got it.

So then I'll just learn that sector 80 and up are "safe" for "user data",
and it's up to all users to take care that any non-UFS/swap/RAID partitions
never go below 80.



But how does the behavior of the first added partition by default
overlapping the disklabel "save butts" -

Does this behavior fill any practical function today, and also when the
user (me) makes there be no overlap, could/do I break anything?

Just to really understand the practical impact.


2015-10-07 1:21 GMT+08:00 Theo de Raadt :

> > Wait, sorry - so the disklabel tool says that the c partition starts at
> > offset 0 , that's logical indeed as data always starts at offset 0.
> >
> > By some reason, the disklabel tool however doesn't want partitions on the
> > first 64 sectors (console dump below), i.e. on the first 32KB (why?).
>
> Modern disks are showing up with sector sizes > 512 bytes.  This is an
> alignment strategy, to future proof things.

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 1:44 GMT+08:00 Mikael <mikael.tr...@gmail.com>:
>
> Ah sure.
>
> Perhaps I misunderstood the level of "foolproofness" that the disklabel
> tool's autogenerated default value was intended to give -
>
> Just curious, now that structural things like this are at stake (i.e. some
> user making a seemingly reasonable presumption based on the "UI" should
> give you partitions where you can really have completely
> user-defined/custom stuff based on an expectation that the autogenerated
> values guarantee non-overlapping with other partitions or the disk
> partition structure itself i.e. the disklabel), what's the motivation for
> not increasing the disklabel tool's minimum autogenerated offset value from
> 64 to 80?
>
> The tradeoff would be that the world would lose 8KB per disk, and win the
> absence of needing to debug a complete disk crash, that I just underwent.
>

In all cases, as long as *not* having any partition covering any of the 16
sectors #64 to #79 won't break anything, I feel this is clear & thanks for
clarifying!

Re: dd if=/dev/zero of=/dev/mykeydisk; bioctl -k /dev/mykeydisk ... = will use 0x00 as key, or will generate a secure key?

[Mikael]

2015-10-06 19:25 GMT+08:00 Jiri B <ji...@devio.us>:

> On Tue, Oct 06, 2015 at 07:17:19PM +0800, Mikael wrote:
> > You
> >
> > 1) Fill your keydisk with zeroes and
> >
> > 2) Apply "bioctl -k" on it.
>
> ^^^ this is not exact cmd arg, is it?
>
> j.
>

No, exact key argument is bioctl -C force -c C -l thedrive -k keydrive
softraid0 , but as I got it that's -k's single intened use anyhow so was
kind of implicit?

2015-10-06 19:27 GMT+08:00 Stefan Sperling <s...@stsp.name>:

> Perhaps this will answer your questions:
> http://www.openbsd.org/papers/eurobsdcon2015-softraid-boot.pdf
>

That one mentions nothing of what the keydisk is supposed to contain.

Perhaps that was omitted for brevity?

dd if=/dev/zero of=/dev/mykeydisk; bioctl -k /dev/mykeydisk ... = will use 0x00 as key, or will generate a secure key?

[Mikael]

You

1) Fill your keydisk with zeroes and

2) Apply "bioctl -k" on it.

Does this mean your key is now zeroes, meaning completely unsafe, or did
bioctl make a key for you?


The keydisk gets some "OPENBSDSR KEYDISK005" header but it says nowhere if
it actually made a key for you.

If it generates it, then there is no mentioning in the man page of how to
use one keydisk for multiple volumes. Perhaps that means it doesn't
generate it afterall?

Also it says nowhere how big the keydisk needs to be, and if it's any
benefit of if it's bigger than needed.

Re: dd if=/dev/zero of=/dev/mykeydisk; bioctl -k /dev/mykeydisk ... = will use 0x00 as key, or will generate a secure key?

[Mikael]

2015-10-06 19:54 GMT+08:00 Stefan Sperling <s...@stsp.name>:

> On Tue, Oct 06, 2015 at 07:32:45PM +0800, Mikael wrote:
> > 2015-10-06 19:27 GMT+08:00 Stefan Sperling <s...@stsp.name>:
> > > Perhaps this will answer your questions:
> > > http://www.openbsd.org/papers/eurobsdcon2015-softraid-boot.pdf
> > >
> >
> > That one mentions nothing of what the keydisk is supposed to contain.
> >
> > Perhaps that was omitted for brevity?
>
> Indeed, it's not explained in detail.
> The mask key is contained in an optional softraid meta data item.
>
> If no softraid meta date exists (as is the case when you zero the disk)
> fresh meta data with a fresh mask key is written to the key disk slice.
>
> Does that answer your question?
>

Aha. So at "-k" time, if there's no key on the keydisk structure already,
it'll make one. So this is how you can use one and the same keydisk for
multiple volumes.


I guess by "mask key" you mean "stored encryption key" i.e. the whole point
with the keydisk.

Is that one generated by bioctl, or does it just take the bytes that happen
to be at those positions already i.e. zeroes??



Also how big should a keydrive be? No docs say.

"dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

The script below includes extra considerations to see through any kernel
caching of the disklabel, by rebooting between every relevant step.

"dd if=/dev/srandom of=/dev/rwd0e bs=1024 count=1" does also wipe the
disklabel.

"dd if=/dev/srandom of=/dev/wd0a bs=1024 count=1" does not wipe the
disklabel.

Find below 1) Description/how reproduce, and 2) Full console interaction
log + dmesg.

Did I misunderstand anything or is this a bug?






*1) Description/how reproduce*
Boot the OBSD 5.7 AMD64 installer in QEMU 2.2.

Go into (S) Shell.

dd if=/dev/zero of=/dev/wd0c bs=1024 count=102400
fdisk -iy wd0

disklabel wd0


And diskabel correctly shows there's only a c partition i.e.


# /dev/rwd0c:
[...]
16 partitions:
#size   offset  fstype [fsize bsize  cpg]
  c:   39070291680  unused


Reboot (reboot).

Go into (S) Shell. Set up the disklabel:

disklabel -E /dev/wd0c

a e
[offset - enter for default]
[size: 1M]
[type: RAID - the keydisk has type RAID]
a a
[offset - enter for default]
[size - enter for all remaining space]
[type - RAID]
w
q


And then check that it's actually written (by "disklabel wd0"), and it is:


16 partitions:
#size   offset  fstype [fsize bsize  cpg]
  a:   390700800016065RAID
  c:   39070291680  unused
  e:16001   64RAID

Reboot (reboot).

Go into (S) Shell, and do:

dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1


Now check that the disklabel has integrity, and it has, i.e. "disklabel
wd0" shows the same output as above.

Reboot (reboot).

Go into (S) Shell, and do "disklabel wd0", and the disklabel is now reset!

# disklabel wd0
# /dev/rwd0c:
type: ESDI
disk: ESDI/IDE disk
label: QEMU HARDDISK
duid: 
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 243201
total sectors: 3907029168
boundstart: 64
boundend: 3907024065
drivedata: 0

16 partitions:
#size   offset  fstype [fsize bsize  cpg]
  c:   39070291680  unused









*2) Full console interaction log + dmesg*

Marked in bold the parts where I actually change anything.

# fdisk wd0
Disk: wd0   geometry: 243201/255/63 [3907029168 Sectors]
Offset: 0   Signature: 0xAA55
Starting Ending LBA Info:
 #: id  C   H   S -  C   H   S [   start:size ]
---
 0: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
 1: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
 2: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
*3: A6  0   1   2 - 243200 254  63 [  64:  3907024001 ] OpenBSD
# disklabel wd0
# /dev/rwd0c:
type: ESDI
disk: ESDI/IDE disk
label: QEMU HARDDISK
duid: 
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 243201
total sectors: 3907029168
boundstart: 64
boundend: 3907024065
drivedata: 0

16 partitions:
#size   offset  fstype [fsize bsize  cpg]
  c:   39070291680  unused
*# dd if=/dev/zero of=/dev/wd0c bs=1024 count=102400*
*102400+0 records in*
*102400+0 records out*
*104857600 bytes transferred in 56.595 secs (1852752 bytes/sec)*
*#*
*# fdisk -iy wd0*
*Writing MBR at offset 0.*
# fdisk wd0
Disk: wd0   geometry: 243201/255/63 [3907029168 Sectors]
Offset: 0   Signature: 0xAA55
Starting Ending LBA Info:
 #: id  C   H   S -  C   H   S [   start:size ]
---
 0: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
 1: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
 2: 00  0   0   0 -  0   0   0 [   0:   0 ] unused
*3: A6  0   1   2 - 243200 254  63 [  64:  3907024001 ] OpenBSD
# disklabel wd0
# /dev/rwd0c:
type: ESDI
disk: ESDI/IDE disk
label: QEMU HARDDISK
duid: 
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 243201
total sectors: 3907029168
boundstart: 64
boundend: 3907024065
drivedata: 0

16 partitions:
#size   offset  fstype [fsize bsize  cpg]
  c:   39070291680  unused
# reboot
syncing disks... done
rebooting...


(After reboot:)

Welcome to the OpenBSD/amd64 5.7 installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
# fdisk wd0
Disk: wd0   geometry: 243201/255/63 [3907029168 Sectors]
Offset: 0   Signature: 0xAA55
Starting Ending LBA Info:
 #: id  C   H   S -  C   H   S [   start:size ]
---
 0: 00  0   0   0 -  0   

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 0:58 GMT+08:00 Ted Unangst :
>
> the disklabel is the second sector of the openbsd part of the disk.
>
> *3: A6  0   1   2 - 243200 254  63 [  64:  3907024001 ] OpenBSD
>
> so, if you overwrite sector 65, you will overwrite disklabel. normally the
> 'a'
> partition overlaps the disklabel, but you made 'e' first.
>

Ugh, ok - just to settle this one forever I hope, four brief Q:s:

1) Does this mean that on an ordinary disk (where the "a" partition is the
disk's first partition, and starts at the offset autogenerated as default
option by the "disklabel" tool), the start of the "a" partition" actually
overlaps with disklabel-internal data?


What's the proper approach here - I thought partitions' contents were
expected to be completely separate from disklabel-internal data and was
completely unaware that the "disklabel" tool's default behavior is to
autogenerate offsets (and sizes?) that overlap those.


2) Does this mean that for my partitions never to overlap the disklabel, I
must ensure that they never start at a sector or byte offset lower than a
given one, if so which one?
 (Disklabels are constant-size so this is a constant that can be
learned?)


3) Is this some how a feature ("and not a bug") so that some frequently
used tools actually rely on this overlapping between disklabel sectors and
partition-a-sectors, for instance "installboot"?


4) If this is documented anywhere feel free to mention.

Also sorry for the fuss but me and some others found it surprising.

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 1:07 GMT+08:00 Theo de Raadt :

> > > Have I (and some others) misunderstood anything about how BSD
> disklabelling
> > > works?
> >
> > the disklabel is the second sector of the openbsd part of the disk.
> >
> > *3: A6  0   1   2 - 243200 254  63 [  64:  3907024001 ]
> OpenBSD
> >
> > so, if you overwrite sector 65, you will overwrite disklabel. normally
> the 'a'
> > partition overlaps the disklabel, but you made 'e' first.
>
> The ffs and swap code knows not to touch the first 8K of the disk, for
> historic reasons, and this has saved many a butt.  I suspect softraid
> also does this.
>
> But your fingers don't know it.
>
>
Right, time for fingers to learn.

Will look forward to learn how it "saved many a butt" and what's the lowest
"safe" offset (..64 + 8*2 = 81+?..) (if that will actually make sense when
understanding the whole thing) through the Q:s in my last post?


Very interesting; thanks!

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 0:45 GMT+08:00 Ted Unangst <t...@tedunangst.com>:

> Mikael wrote:
> > The script below includes extra considerations to see through any kernel
> > caching of the disklabel, by rebooting between every relevant step.
> >
> > "dd if=/dev/srandom of=/dev/rwd0e bs=1024 count=1" does also wipe the
> > disklabel.
> >
> > "dd if=/dev/srandom of=/dev/wd0a bs=1024 count=1" does not wipe the
> > disklabel.
>
> > 16 partitions:
> > #size   offset  fstype [fsize bsize  cpg]
> >   a:   390700800016065RAID
> >   c:   39070291680  unused
> >   e:16001   64RAID
>
> yes, if you overwrite the disklabel (which lives at the start of the disk)
> you
> will wipe it. don't do that.
>

But.. wd0e is the "e" partition within the BSD disklabel, and that's what I
wrote to (and even if it's not relevant here, also its writes should be
"boundary checked" so they're foolproof in respect of overwriting
extra-partition data)?

With "dd if=/dev/srandom of=/dev/rwd0*c* bs=1024 count=1" I'd have
understood that I wiped something sensitive, but this is "dd
if=/dev/srandom of=/dev/rwd0*e* bs=1024 count=1"??


Have I (and some others) misunderstood anything about how BSD disklabelling
works?

Re: "dd if=/dev/srandom of=/dev/wd0e bs=1024 count=1" WIPES my wd0 disklabel. Is this intended, bug, how come, how workaround ??? Incl reproduction script+console output+dmesg

[Mikael]

2015-10-07 1:14 GMT+08:00 Theo de Raadt :

> > > But your fingers don't know it.
> > >
> > >
> > Right, time for fingers to learn.
> >
> > Will look forward to learn how it "saved many a butt" and what's the
> lowest
> > "safe" offset (..64 + 8*2 = 81+?..) (if that will actually make sense
> when
> > understanding the whole thing) through the Q:s in my last post?
>
> 8K is the smart offset.
>

Wait, sorry - so the disklabel tool says that the c partition starts at
offset 0 , that's logical indeed as data always starts at offset 0.

By some reason, the disklabel tool however doesn't want partitions on the
first 64 sectors (console dump below), i.e. on the first 32KB (why?).

The 8KB you talk about now, is that *in addition* to the first 32KB then,
so >40KB i.e all sectors starting with and above 80 are safe?


And, does it break anything or otherwise require administrative
consideration, that there is no partition covering those 8KB of disklabel
data (i.e. "installboot" won't misbehave or alike)?


# disklabel -E wd0
Label editor (enter '?' for help at any prompt)
> p
OpenBSD area: 64-3907024065; size: 3907024001; free: 3907024001
#size   offset  fstype [fsize bsize  cpg]
  c:   39070291680  unused
> a
partition: [a]
offset: [*64*]

Re: disklabel fs types, where can I find the whole list of supported types?

[Mikael]

Dear Dusan and Ingo,


I meant disklabel's fs type, not fdisk's partition type;


Disklabel filesystem type is different altogether from fdisk partition
type, isn't it?

And, the disklabel filesystem type is requested as a string (unlike the
fdisk partition type which is an 8-bit unsigned integer typically entered
in hex) and hence you need to know which options are available:

# *disklabel -E wd0*
> *a a*
offset: [0]
size: [12345]
FS type: [4.2BSD] *?*

Filesystem type (usually 4.2BSD or swap)
FS type: [4.2BSD] *help*

Unrecognized filesystem type 'help', treating as 'unknown'


And "e a" asks nothing about fs type.

What do you say?

Mikael


2015-10-05 15:03 GMT+08:00 Ingo Schwarze <schwa...@usta.de>:

> Hi Mikael,
>
> Mikael wrote on Mon, Oct 05, 2015 at 01:44:36PM +0800:
>
> > Hi,
> >
> > Where can I see a complete list of disklabel fs types?
>
> https://en.wikipedia.org/wiki/Partition_type
>
> But note that such a list can never be authoritative, never complete,
> and by definition of the concept, almost all of it is irrelevant
> for no matter which task.
>
> It may seem highly unusual that *I* am sending people to the
> web for documentation.  However, there is a reason in this
> particular case.
>
> > It must be documented somewhere,
>
> No, there is no need to.
>
>  1) It is in no way related to OpenBSD.  So why should OpenBSD
> document it?  All that is relevant to OpenBSD is A6, and
> that is documented in fdisk(8), where it belongs.
>
>  2) We do want our documentation to be correct and complete,
> but we also want it to be concise, so we don't include
> irrelevant information.
>
>  3) OpenBSD does support multibooting, though we clearly
> discourage it for most use cases.  So you may have a need
> to know another partition type ID.  But that's a property
> of your *other* system, so you should look it up in the
> documentation of your other system.
>
> > but I can't find it neither in the
> > "disklabel" tool itself, nor in its man pages.
>
> It has nothing to do with disklabel(8), it's a property of the
> machine-dependent so called "master boot record".
>
> > In disklabel's "a" command,
>
> I guess you are talking about fdisk(8) -e.
>
> > typing "?" is interpreted as invalid input, and
> > typing "help" is interpreted as choosing the fs type "unknown".
>
> I don't know which version of OpenBSD or which snapshot you
> are using, and right now, i don't have the time or hardware
> to figure out whether there was or is a bug of that kind.
> Maybe someone else can answer this part.
>
> Yours,
>   Ingo

Re: disklabel fs types, where can I find the whole list of supported types?

[Mikael]

Right, I am fully aware of that (i.e. that you can type in MBR partition
type as HEX code in the fdisk tool) - please correct me if I'm wrong, but
that is specific to the FDISK (and the MBR partition table only), and the
BSD disklabel and hence what you're working with in the disklabel tool, is
separate altogether from that;

My question was (third time now), which FS types are available in the
disklabel tool?

Is it "4.2BSD", "swap", "RAID" and "unknown" only, or are there any more?


2015-10-05 15:41 GMT+08:00 Dusan Sukovic <dusan.suko...@gmail.com>:

> Yes, but beside ffs HEX id inside fdisk prompt you have also ffs partition
> id values in plain English..
>
> Regards,
>
> Dusan
>
> 2015-10-05 9:28 GMT+02:00 Mikael <mikael.tr...@gmail.com>:
>
> >
> > And, the disklabel filesystem type is requested as a string (unlike the
> > fdisk partition type which is an 8-bit unsigned integer typically entered
> > in hex) and hence you need to know which options are available:

Include "wsconsctl" on installer CD so keyboard repeat can be disabled = make VNC KVM install smooth on laggy connection?

[Mikael]

VNC KVM install means some keypresses will be interpreted as seconds-long,
ordinarily leading to multiple unintended "enter" or character key presses
which easily seriously breaks things, when the connection is not perfect,
which it many times is not.

I believe there is no way to disable keyboard repeat on the installer CD -

What do you say about including "wsconsctl" on it?

Thanks,
Mikael

Re: disklabel fs types, where can I find the whole list of supported types?

[Mikael]

First peripherally:
2015-10-05 17:58 GMT+08:00 Jason McIntyre <j...@kerhand.co.uk>:

> we're not talking about the list in fstab(5)?
>
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/fstab.5?query=fstab=i386

Nice but it misses the 4.2BSD and RAID types.
Then centrally:



2015-10-05 17:14 GMT+08:00 Ingo Schwarze <schwa...@usta.de>:

> Mikael, why do you ask?  What is the actual problem you want to
> solve?  Read a disk that nobody touched for 35 years?  Write a disk
> that can be read by a machine which hadn't its operating system
> upgraded for 35 years?
>
> If it's just "i have no special needs but i can configure something
> here but i don't know the options", i'd recommend you just stop
> worrying and don't touch the defaults.  In general, in particular
> with low-level tools like this, in particular when it's not even
> documented, the message the developers are trying to send is "use
> the defaults unless you have very special needs and you know what
> you are doing".  Nothing interesting to see here, move on...
>

2015-10-05 18:25 GMT+08:00 Ted Unangst <t...@tedunangst.com>:

> the lack of documentation may be an oversight, but i can't imagine why one
> would neet to know. if curious, refer to the source. if trying to solve a
> problem, picking the obvious name will likely work. if there are values you
> don't know about, you probably don't have a use for them.
>

Indeed I didn't get to this question with a "good" reason - I wanted to
learn backwards how keydisks *might* be loaded, as that is undocumented
(too). However, with this said,


In such a central aspect of the system as the disk slicing/labelling, I
think it's reasonable to have some kind of suggestion or documentation of
at least what would be included in normal usecases;

The way it is now is that as user you learn to know that "4.2BSD" and
"swap" are possible options, because the "a" option will insinuate those
through its default setting.

And, as soon as you dig into the softraid docs, you get mentionings that
there's a label of type "RAID" too, and you try to type it in and it works,

And last you type in gibberish and it gives you "undefined", so that way
you learn of in total four options.


I don't know about you but I can feel this as a bit much "black box"
behavior, so just a little bit more documentation somewhere would be
fantastic.

Just the "man disklabel" saying something like

"FS type" is generally "BSD4.2" for all partitions except for b which
should "swap", and you might want "RAID" too, and if you need a raw blob
then use "[ SOMETHING ]". (Also CD ISO99.. and FAT, noo???) Other than this
don't experiment as there's a lot of legacy about this setting.


(and perhaps typing in "?" in the "FS type" input in the disklabel program)
would be helpful!

(Without getting into undefined territory where you'd risk messing up 35
year old exotic conventions etc.)

Re: Include "wsconsctl" on installer CD so keyboard repeat can be disabled = make VNC KVM install smooth on laggy connection?

[Mikael]

Neat, yes !!


(Getting all kinds of unintended behavior just because a network lag
happens to incur an extra couple of \n:s or other chars of keyboard console
input for you is such a PITA.)


(As it is right now, it cannot be disabled in the installer right?)


2015-10-05 18:28 GMT+08:00 Ted Unangst <t...@tedunangst.com>:

> Mikael wrote:
> > VNC KVM install means some keypresses will be interpreted as
> seconds-long,
> > ordinarily leading to multiple unintended "enter" or character key
> presses
> > which easily seriously breaks things, when the connection is not perfect,
> > which it many times is not.
> >
> > I believe there is no way to disable keyboard repeat on the installer CD
> -
> >
> > What do you say about including "wsconsctl" on it?
>
> do people need autorepeat in the installer? (for what?) i think it'd be
> simpler to just disable it in all ramdisks.

disklabel fs types, where can I find the whole list of supported types?

[Mikael]

Hi,

Where can I see a complete list of disklabel fs types?

It must be documented somewhere, but I can't find it neither in the
"disklabel" tool itself, nor in its man pages.

In disklabel's "a" command, typing "?" is interpreted as invalid input, and
typing "help" is interpreted as choosing the fs type "unknown".

Thanks,
Mikael

Re: when SSDs are not so solid or why no TRIM support can be a good thing :)

[Mikael]

For having a *guaranteedly intact* storage, what is the way then?

This is with the background of recent discussions that touched on
https://www.usenix.org/legacy/events/fast08/tech/full_papers/bairavasundaram/bairavasundaram_html/index.html
and https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ .



What about having *two SSD*:s in softraid RAID1, and as soon as any IO
failure is found on either SSD, that one would be replaced?

If the underlying read operations are made from both SSD:s each time and
the machine has ECC RAM (??and UFS is checksummed enough??), then at least
the OS would be able to detect corruption (??, fix anything??) and return
proper read failures (or sigsegv) properly.

Mikael

2015-06-18 16:23 GMT+07:00 Karel Gardas gard...@gmail.com:

 On Thu, Jun 18, 2015 at 9:08 AM, David Dahlberg
 david.dahlb...@fkie.fraunhofer.de wrote:
  Am Donnerstag, den 18.06.2015, 02:15 +0530 schrieb Mikael:
 
  2015-06-18 2:07 GMT+05:30 Gareth Nelson gar...@garethnelson.com:
  No I meant, you plug in a 2TB SSD and a 2TB magnet HD, is there any way
 to
  make them properly mirror each other [so the SSD performance is
 delivered
  while the magnet disk safeguards contents] - would you use softraid
 here?
 
  No. If you use a RAID1, you'll get the performance of the worse of both
  disks. To support multiple disks with different characteristics and to
  get the most out of it was AFAIK one of motivations for Matthew Dillon
  to write HAMMER.
 

 I'm not sure about RAID1 in general, but I'm reading softraid code
 recently and based on it I would claim that you get write performance
 of the slowest drive (assuming OpenBSD schedule writes to different
 drives in parallel), but read performance slightly higher than slower
 drive since the read is done in round-robin fashion hence SSD will
 speed it a little bit.

 Anyway, the interesting question is if it makes sense to balance this
 interleaving reading based on actual drive performance. AFAIK this
 should be possible, but IMHO it'll not be that reliable, i.e. it'll
 not provide that much of added reliability. Since reliability is my
 concern, I'm more looking forward to see kind of virtual drive with
 implemented block checksumming in OpenBSD, that IMHO will provide some
 added reliability when run for example in RAID1 setup.

 Karel

Re: when SSDs are not so solid or why no TRIM support can be a good thing :)

[Mikael]

Wait, just for my (and I guesss some others') clarity, three questions here:

1) From the article, what can we see that Ext4/Linux actually did wrong
here? - Is it that the TRUNCATE command should be abandoned completely, or
was it how it matched supported/unsupported drives, or something else?


2) General on SSD: When an SSD starts to shrink because it starts to wear
out, how is this handled and how does this appear to the OS, logs, and
system software?


3) On OBSD, how would you generally suggest to make a magnet-SSD hybrid
disk setup where the SSD gives the speed and maget storage security?


Thanks!



2015-06-17 23:17 GMT+05:30 Mariano Ignacio Baragiola 
mari...@baragiola.com.ar:

 On 17/06/15 08:05, frantisek holop wrote:

 https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/

 also note the part relating to ext4:

 I have to admit, I slept better before reading the
 changelog.


 fast, features, realiable: pick any 2.

 -f


 I don't think TRIM is to blame here. I don't understand
 why someone on their sane mind would use latests versions
 of Ubuntu and Linux for servers. And yes, I know Ubuntu
 for Servers is a thing, and yes, I know the fight this
 instability with redundancy, but stil...

 About EXT4: it is not exactly the most trust-worthy filesystem
 there is.

 Interesting reading, though.

Re: when SSDs are not so solid or why no TRIM support can be a good thing :)

[Mikael]

2015-06-18 2:07 GMT+05:30 Gareth Nelson gar...@garethnelson.com:

 On point 3, hybrid SSD drives usually just present a standard IDE
 interface - just use a SATA controller and you don't need to worry about it


No I meant, you plug in a 2TB SSD and a 2TB magnet HD, is there any way to
make them properly mirror each other [so the SSD performance is delivered
while the magnet disk safeguards contents] - would you use softraid here?

Re: when SSDs are not so solid or why no TRIM support can be a good thing :)

[Mikael]

2015-06-18 0:53 GMT+05:30 Theo de Raadt dera...@cvs.openbsd.org:

  2) General on SSD: When an SSD starts to shrink because it starts to wear
  out, how is this handled and how does this appear to the OS, logs, and
  system software?

 Invisible.  Even when a few drives make it visible in some way, it is
 highly proprietary.


What is then proper behavior for a program or system using an SSD, to deal
with SSD degradation?:

So say you have a program altering a file's contents all the time, or you
have file turnover on a system (rm f123; echo importantdata  f124). At
some point the SSD will shrink and down the line reach zero capacity.


The degradation process will be such that there will be no file content
loss as long as the shrinking doesn't exceed the FS total files size right?
(Spontaneously I'd presume the SSD informs the OS of shrinking at sector
write time through failing sector writes, and the OS registers shrunk parts
in the FS as broken sectors.)


Will the SSD+OS reflect the degradation status in getfsstat(2) f_blocks /
df(1) blocks so the program just ensure there's some f_bavail / avail all
the time by simply shrinking (ftruncate etc.) its files accordingly and
when f_blocks is too small shuts down completely?


3) On OBSD, how would you generally suggest to make a magnet-SSD hybrid
disk setup where the SSD gives the speed and maget storage security?

SSD failure handling

[Mikael]

2015-06-18 8:57 GMT+05:30 Nick Holland n...@holland-consulting.net:

  What is then proper behavior for a program or system using an SSD, to
 deal
  with SSD degradation?:

 replace drive before it is an issue.

  So say you have a program altering a file's contents all the time, or you
  have file turnover on a system (rm f123; echo importantdata  f124). At
  some point the SSD will shrink and down the line reach zero capacity.

 That's not how it works.

 The SSD has some number of spare storage blocks.  When it finds a bad
 block, it locks out the bad block and swaps in a good block.

..

 Neither SSDs nor magnetic disks shrink to the outside world.  The
 moment they need a replacement block that doesn't exist, the disk has
 lost data for you and you should call it failed...it has not shrunk.

 Now, in both cases, this is assuming the drive fails in the way you
 expect -- that the flaw will be spotted on immediate read-after-write,
 while the data is still in the disk's cache or buffer.  There is more
 than one way magnetic disks fail, there's more than one way SSDs fail.
 People tend to hyperventilate over the one way and forget all the rest.

 Run your SSDs in production servers for two or three years, then swap
 them out.



Thank you very much for your remarks.

In particular for the distinctions that for production purposes a disk is
good or bad only, and that SSD failure can surface in any nasty way
imaginable.

I guess in a system where you have some kind of data duplication between
disks already, and for economical reasons you want to use the SSD til it
fails, you have bad disk detection trig on any write or read IO error, both
directly reported (fopen/fwrite/fread/fclose misbehavior) and appearing
indirectly through IO op taking more time than usual .. and perhaps a
checksum?

Could broken SSD corrupt binaries without UFS checksumming and application
executable checksumming denying a program to start at all, so random
SIGSEGV:s count as broken disk too - what checksumming does UFS and
executables have, how reliable are they?

Re: mac mini - virtualbox - openbsd amd64?

[Mikael]

2015-03-24 22:17 GMT+05:30 Jeremiah Ford m...@jeremiahford.com:

 On 2015-03-24 11:48, frantisek holop wrote:

 has anybody tried running openbsd in virtualbox on a
 mac mini?  is X11, etc feasable?

 -f


 Never on a macmini, but I have on imac and many others.  If you are
 seeking a virtual environment, I do not recommend using OpenBSD as the
 guest. Aside from the obvious security concerns, You will end up with
 frustrations such as mouse troubles and no ability to resize screen running
 X env.


Is this the same reason as why XVFB can't change [virtual] screen
resolution too?

xvfb run with the -xrandr argument e.g. Xvfb -screen 0 1600x1200x32 +extension
RANDR , and trying to change screen resolution using xrandr e.g. xrandr
-display :0 --size 1280x1024 and hooked in graphically with x11vnc.

It seems that the XRANDR extension is magically absent because the xrandr
tool terminates with error RandR extension missing?!

Re: OpenBSD as base OS for Virtualization

[Mikael]

If someone wanted to hack together a Bhyve OBSD port would be complete
awesomeness.

Even as a custom patch only for the stupid guys like me who love this
unsafe virtualization stuff that so many use now. It would be awesome.

I know the whole virtualization thing is crap from a strict security point
of view but I like to take the risk, and OBSD certainly is a better
codebase to do this host stuff in than other systems. Probably some people
would be happy to donate some bucks to the person who wanted to implement
and maintain it.

NetMap in OpenBSD

[Mikael]

NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
great idea. It's a simple API for solving an important problem, at least
its core parts.

Is OBSD's kernel structure suited as it is for NetMap?

What's the interest out there for NetMap on OBSD?

Thanks,
Mikael

Re: NetMap in OpenBSD

[Mikael]

Dear Henning,

Thank you for your thoughtful response.

2014-10-14 11:02 GMT+02:00 Henning Brauer hb-open...@ml.bsws.de:

 * Mikael mikael.tr...@gmail.com [2014-10-14 10:24]:
  NetMap (http://info.iet.unipi.it/~luigi/netmap/) in OpenBSD would be a
  great idea.



We kinda like our stack.


Of course, OBSD has a very good stack as it is, but it has no NetMap
functionality i.e. there's no way for a userland application to do high
speed packet-level IO.



 for what?


There is a whole world of need of network monitoring and manipulation and
other specialized networking software.

The benefit with doing this on OBSD is that it's more robust than other
systems and at least in my world that's hard currency.



Chris got NetMap to compile on OBSD though I have no clue if/how well
present subsystems deliver for it.


Mikael

Re: NetMap in OpenBSD

[Mikael]

2014-10-14 16:15 GMT+02:00 Henning Brauer hb-open...@ml.bsws.de:

  Of course, OBSD has a very good stack as it is, but it has no NetMap
  functionality

 yeah, and that is good. netmap bypasses teh stack and you look at
 reimplementing the stack in userland, repeating mistakes, bugs and
 whatnot from many decades.

  i.e. there's no way for a userland application to do high speed
  packet-level IO.

 there are plenty of methods actually.


Like what?



 userland reimplementing the stack[...]


I didn't necessarily/specifically suggest that.


 There is a whole world of need of network monitoring and manipulation and
  other specialized networking software.

 I read a collection of buzzwords with nothing specific.

 A solution in dire need of a problem.


Will be more clear on this one following your response. Last for completing
reflections -



Most devices in a system can be accessed with good performance from
userland as it is now, for instance block devices, USB, serial ports, video
and audio.

Ethernet is a rare exception and NetMap solved this in a neat way -

Prior to NetMap, those who wanted to make high-performance ethernet IO in
userland would run their app as root and effectively implement NIC hardware
drivers in userland. NetMap generalized this entire problem to one
hardware-agnostic interface.

Re: low power device

[Mikael]

Zé,

What does the abbreviation OP stand for?

Thanks.


2014-09-12 17:22 GMT+02:00 Zé Loff zel...@zeloff.org:

 On Fri, Sep 12, 2014 at 04:28:46PM +0200, Lars wrote:
  On 12.09.2014 15:27, Martijn van Duren wrote:
  Hello misc@,
 
  Hi,
 
  Currently I have an old desktop PC running as a home server/media
  center, which runs OpenBSD. Most of the time it's idling, but does run
  (open)ssh/(open)smtp/imap(dovecot)/http(nginx/apache
  +subversion)/minidnla, which I want to keep available.
  
  Is there any board/device known which can support these requirements and
  is fully (within the requirements) supported by OpenBSD?
  
 
  As a personal preference I would avoid using ARM boards and would try to
 go
  with x86/amd64 boards instead. I don't know how well those ARM devices
 are
  supported on OpenBSD (I have only a little experince with running Linux
 on
  those), but the performance was pretty disappointing(with Linux). *I*
 would
  decide for the APU from pcengines.
  http://pcengines.ch/apu.htm

 My first thought too, but it has no video, which is probably required by
 the OP.

  I haven't used these either but they are x86/amd64 and well supported
 with
  OpenBSD as a lot of devs use them. I can not say anything about the
  performance, but I would *assume* they have at least the same
 performance as
  your mentioned pandaboard (if not better) and probably cost the same.
 
  I use an HP Microserver N54L for the same purpose. It is roughly the same
  price but probably doesn't fit your power consumption requirements
 (roughly
  40 Watts with 2 drives)
 
  Sincerely,
  
  Martijn van Duren
 
  Have a nice day
  Lars
 

 --

Re: OpenBSD on Utilite pro device

[Mikael]

2014-07-30 16:05 GMT+02:00 Maciej Jan Broniarz gau...@gausus.net:

 On Wed, Jul 30, 2014 at 03:48:23PM +0200, Mikael wrote:
  Maciej,
 
  At least there is an iMX6 port of OBSD. Please let the ML know what you
 got
  to.
 

 Hi,

 Where can i find the instalaltion files?


26 Jan there was a related post check it.

The tree with the patches for iMX6 are on github and the author's name is
Nick, if i recall right.

Google.


 Sorry, but who is ML ?


Mailing list.



 All best,
 mjb

  Mikael
 
 
  2014-07-30 13:31 GMT+02:00 Maciej Jan Broniarz gau...@gausus.net:
 
   Hi,
  
   Did anyone tried to run OpenBSD on Utilite Pro device?
   http://utilite-computer.com/web/utilite-pro-specifications
  
   All best,
   mjb

Re: DNS resolver retries configurable? (or: Anything to make DNS resolves always work!)

[Mikael]

Hi Matthew,

Aha so the files of reference are asr.c (
http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/asr/asr.c?rev=1.31;content-type=text%2Fplain
)
and res_send_async.c (
http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/asr/res_send_async.c?rev=1.19;content-type=text%2Fplain
 )


Do I understand it right that when the DNS resolver fails with a resolve,
it actually retries with the *next* DNS server of resolv.conf,

and thus for a resolve to fail, 4 DNS servers in a row must be unresponsive
within the timeout which is 5 seconds per attempt (ac_nstimeout)?

(this is implemented in iter_ns at the bottom of res_send_async.c.)


So then, the resolve failures I experienced cannot have been due to failure
about an individual DNS server being down, but instead, that it took 4 of
them in a row each more than 5 seconds to complete the resolve - and that
in turn must have been because of some extremely serious lag about the
resolved domain's DNS server?

I guess to safeguard for such problems, it would help to have only one DNS
server, this being 127.0.0.1, run a local named/dnsmasq/etc., so then that
one would get all the four retries, thus relatively increasing the chance
by 4x that the resolve will actually look, as the server does caching
between resolves so if the first resolve took 17 seconds then the fourth
one will be the succeeding one?


Thanks,
Mikael



2013/12/20 Matthew Dempsky matt...@dempsky.org

 On Thu, Dec 19, 2013 at 2:36 PM, Mikael mikael.tr...@gmail.com wrote:
  a) OpenBSD's resolver configured to retry 999 times before failing, and
  [...]
  If so, is there any way to do a)?

 In src/lib/libc/asr/asr.c, change ac-ac_nsretries = 4; to
 ac-ac_nsretries = 999;, recompile, and reinstall.

 However, I wouldn't recommend you actually do this.  You should
 instead focus on figuring out why your DNS queries are failing in the
 first place, and/or fix your downstream users to handle HTTP errors
 correctly.

DNS resolver retries configurable? (or: Anything to make DNS resolves always work!)

[Mikael]

Seems that by default OpenBSD's DNS resolver rotates what DNS server from
/etc/resolv.conf it uses, and when the DNS server used for a resolve does
not work, the DNS resolve fails;

Perhaps not exactly like this, but clearly in this direction: If there's
any imperfection about any of the listed DNS servers, there will be DNS
resolves that fail.

This creates an extended issue that errors propagate, like, HTTP connects
fail because the remote host can't be found, causing all kinds of weird
errors.

I want this fixed.

I guess a setup with

a) OpenBSD's resolver configured to retry 999 times before failing, and
b) Running a DNSMasq and using that as only DNS server on a machine

should be a perfect safeguard agains failing resolves?


If so, is there any way to do a)?

Thanks,
Mikael

Re: Should Android have used OpenBSD instead of Linux?

[Mikael]

Hi Frank,

I heard this argument that OBSD is slow circulated a little bit.

Can you please clarify and quantify?

Really performance is of a very secondary importance, however unless
there's a good reason for it not to be there, it is nice that it is there,
hence my question for you to clarify and quantify now -

Thanks,
Mikael




2013/11/29 frantisek holop min...@obiit.org

 hmm, on Tue, Nov 26, 2013 at 02:00:53PM -0800, Chris Cappuccio said that
  So the next question is, why would someone want to switch to OpenBSD
  on one of these platforms?
 
  1. Concise ecosystem (less maintenance of your own distribution)
 
  2. High quality code
 
  3. Increasing attention to areas that matter (ARMv7, KMS, etc)

 just like everyone else, i would love to see an openbsd
 powered android phone.  but i think the elephant in
 the room no one is talking about is performance.
 without getting into running bad code faster vs
 running good code slower, openbsd is simply slow.

 -f
 --
 there's no second chance for a good first impression.

How handle multiple G++ versions' compiled code in one and the same process (without SIGSEGV)?

[Mikael]

Dear list,

I've seen issues where a process links to one library compiled with the
OS-bundled G++ version and another that's compiled with a newer G++ version
(4.7 etc.). Libraries include boost, QT and their C++-based dependencies.

I raise this question as there are instances when a newer G++ version is
required for a project to work at all (because of compiler version
specifics, C++X11 support etc).


The typical error I've seen, is that exception handling goes bazonkas:

As soon as the default exception handler is trigged, an error message is
printed and then the process SIGSEGV:s.

Also, ordinary exception handling may malfunction and lead to SIGSEGV.


Picking up what others say on this,

 * #gcc on FreeNode say libstdc++ versions are *not* intercompatible with
each others,

   and also they say newer libstdc++ versions are *not* providing backwards
compatibility with older ones

 * FreeBSD published a workaround to the libstdc++ compatibility issue
using their libmap.conf feature:
http://www.freebsd.org/doc/en/articles/custom-gcc/article.html .



What is OpenBSD's take on this?;

What's the best practice?

If the only way is to actually recompile all dependency libraries in the
newer G++ version, then, is there a way to build ports and their
dependencies with a specific G++ version and then install them all in a
separate directory i.e. /usr/local/lib/g++-4.7-compiled/ ?

Thanks!
Mikael

How compile a port and all of its dependencies that use G++ with a specific G++ version (boost/QT etc.) and then install in a separate dir?

[Mikael]

After confirming with someone competent, I'm clear that there is no way
ever to use more than one libstdc++ version concurrently in one OS process.

Therefore, my question is now purely:


How do you compile a port and all of its dependencies that use G++, with a
specific G++ version e.g. /usr/local/bin/eg++?

Also then, how install these in a separate directory/directory structure as
not to mess up other programs by interfering with the ordinary
OS-preinstalled versions of the same libraries, that should indeed remain
compiled with the OS-bundled G++ version



Thanks!
Mikael


2013/11/29 Mikael mikael.tr...@gmail.com

 Dear list,

 I've seen issues where a process links to one library compiled with the
 OS-bundled G++ version and another that's compiled with a newer G++ version
 (4.7 etc.). Libraries include boost, QT and their C++-based dependencies.

 I raise this question as there are instances when a newer G++ version is
 required for a project to work at all (because of compiler version
 specifics, C++X11 support etc).


 The typical error I've seen, is that exception handling goes bazonkas:

 As soon as the default exception handler is trigged, an error message is
 printed and then the process SIGSEGV:s.

 Also, ordinary exception handling may malfunction and lead to SIGSEGV.


 Picking up what others say on this,

  * #gcc on FreeNode say libstdc++ versions are *not* intercompatible with
 each others,

and also they say newer libstdc++ versions are *not* providing
 backwards compatibility with older ones

  * FreeBSD published a workaround to the libstdc++ compatibility issue
 using their libmap.conf feature:
 http://www.freebsd.org/doc/en/articles/custom-gcc/article.html .



 What is OpenBSD's take on this?;

 What's the best practice?

 If the only way is to actually recompile all dependency libraries in the
 newer G++ version, then, is there a way to build ports and their
 dependencies with a specific G++ version and then install them all in a
 separate directory i.e. /usr/local/lib/g++-4.7-compiled/ ?

 Thanks!
 Mikael

Running out of clusters (due to too low kern.maxclusters) destinies you to reboot or should it self-heal? What parameters should guide one's setting of kern.maxclusters?

[Mikael]

Dear list,

Karlis and I experienced the hangup problem and discussed it last month
on the bugs ML ( all the thread at
http://marc.info/?l=openbsd-bugsm=137321082217664w=2 ).

Claudio had the kindness to point out that having no available clusters
(i.e. a kern.maxclusters setting set too low for the machine's network load
in a given moment) would bring a state matching our description of the
hangup problem. Though, it would make sense that the problem (= TCP stack
not accepting new incoming connections, and some more) would *self heal* as
soon as clusters/mbufs had been freed up, and this was not Karlis' and my
experience. So to bring order to this reasoning, I wish to pose the
question:


Is the TCP stack implemented in such a way that clusters running out
[because of a too low kern.maxclusters setting for a particlar network
load] may lead to permanent breakdown of its state and thus may require
reboot, or should it always self-heal?



And then: Now that we see that a too low kern.maxclusters setting for a
particular server load may make the TCP stack fail temporarily - or
permanently? - , we get to the question:


What are the guidelines for choosing kern.maxclusters setting - how should
number of incoming TCP connections per second, number of concurrent TCP
connections, their throughput, and any other relevant parameters, impact
the choice of kern.maxclusters ?


Thanks and best regards,
Mikael

Is there any ordinary/intended code path execution case, where new TCP connections would not be received by their bound processes anymore? Re: Is there any a) TCP network stack state and ...

[Mikael]

Is there any ordinary/intended code path execution case, where new TCP
connections would not be received by their bound processes anymore?

(as in, on all the system suddenly sshd/httpd/smtpd/etc.etc. do not receive
connections made by ssh localhost/telnet localhost 22/telnet localhost 25
etc. but return connection reset by peer, connection refused or just
nothing/blocks)

For instance, is there
 * Some queue that when full would have this effect, or
 * Some malloc() call that could fail in a system-global systemic way if an
application's resident memory quota is full?

Thanks,
Mikael

Re: Is there any ordinary/intended code path execution case, where new TCP connections would not be received by their bound processes anymore? Re: Is there any a) TCP network stack state and ...

[Mikael]

Ah sorry, will try to figure out a more concise one if I'll email again.


Now to the Q - are you aware of any such code path?

I'm trying to understand this behavior I reported at bugs@ ..

Thanks, Mikael

2013/7/11 Jan Stary h...@stare.cz

 Is there any reason why you would annoy everyone
 with a three-line subject?

Re: Is there any a) TCP network stack state and b) mbuf 1) prettyprint-dumper and 2) resetter respectively?

[Mikael]

Hi Alexey,

Awesome. Are you aware if if there's any way to unbind a bound TCP port too?

So, there's a process, defunct or not, that is bound to say port 80 - how
to force it to close?

Thanks,
Mikael

2013/7/8 Alexey E. Suslikov alexey.susli...@gmail.com

 Mikael mikael.trash at gmail.com writes:

   * resetting (wiping) the mbuf:s,

 Not actually for resetting mbufs, but there's a tcpdrop(8),
 with accompanying fstat(1) usage of course.

Is there any a) TCP network stack state and b) mbuf 1) prettyprint-dumper and 2) resetter respectively?

[Mikael]

Dear list,

This email thread is to query for if anyone knows about a tool in OBSD for

 * introspecting the TCP network stack state, preferably by simply getting
a pretty-printed dump of them, and for

 * resetting the TCP network stack state

respectively and any tool for

 * introspecting the present mbufs, preferably by simply getting a
pretty-printed dump of them, and for

 * resetting (wiping) the mbuf:s,

respectively?

I'm looking into an issue described in the bugs@ ML (post mirrored at
http://marc.info/?t=13732109382r=1w=2 ) currently, and my best
understanding is that such a tool could be of direct or indirect value for
figuring out its original reason, the next time it happens.

Also I would wildly guess that such a complex piece of code as the TCP
stack would be worthy of an introspection tool?

Thanks,
Mikael

Re: OpenBSD alternative to cpanel/plesk

[Mikael Österdahl]

I use webmin and it works ok, need a few tweaks, but works. Not in ports though.

/Mikael

31 jul 2011 kl. 00:56 skrev Paolo Aglialoro paol...@gmail.com:

 Hi all,

 I've been through the well known online article:
 http://lordmatt.co.uk/item/1739/
 Also been reviewing ports in www section.

 Among the products in cited article I understand there's just webmin (to
 be downloaded from its site and compiled), while in ports I find none of
 there mentioned products. I'd appreciate to hear some of your experiences.

 Can anybody suggest if there exist working solutions for OpenBSD?
 Renouncing to a full suite (and concentrating on just mail) would be
 mailserv a robust solution?
 In the last case, would there be other products to put aside, e.g. to manage
 webspace for users backup like rsync, ftp, etc.?

 Thanks

Re: Bug Tracking system does not work

[Mikael Österdahl]

Terrible? In what way? I use it in my work and I think it works great.

What ticket software do you think is better?

/Mikael

2011/7/19 Johan Beisser j...@caustic.org:
 On Tue, Jul 19, 2011 at 9:57 AM, Amit Kulkarni amitk...@gmail.com wrote:


 http://openports.se/www/rt
 ?
 written in perl.

 As someone who uses this for ticket tracking, let me be the first to
 say it's terrible.

Re: Lucent Technologies Orinoco Wifi card (PCMCIA) and OpenBSD?

[Mikael Bak]

On Wed, 9 Dec 2009 10:16:28 +0100 (CET)
David Vasek va...@fido.cz wrote:

 On Tue, 8 Dec 2009, David Vasek wrote:
 
  On Mon, 7 Dec 2009, Mikael Bak wrote:
 
  Hi list,
  
  Seems to me that this card isn't recognized at boot time (dmesg pasted at 
  the end of this email). It's a PCMCIA card. Instead of a wifi device 
  OpenBSD tries to allocate a serial port (com3).
 
  I don't think it is necessarily related to the wi(4) driver. I experienced 
  very similar behaviour with an ne(4) PCMCIA card (D-Link DE-650). Sometimes 
  it got detected correctly as ne3, sometimes as a non-working serial port 
  (pccom?). The machine was a ThinkPad T42 and the following part of the 
  dmesg 
  is from 4.2-current. ACPI was not configured. I haven't used that card 
  since 
  then.
 
 It hasn't changed. The Ethernet card gets incorrectly detected as com* in 
 one PCMCIA slot (pcmcia0) and correctly as ne3 in the other (pcmcia1). It 
 still has some probles there from time to time. Don't know what is the 
 cause of this.
 
 Have you tried your wi(4) card in both PCMCIA slots in your laptop? (One 
 slot at a time, of course.)
 
 Regards,
 David
 

Hi David,

No, I hadn't tried the card in both slots. But I did now, and still same error.

Thanks for the tip.

Mikael

Re: Lucent Technologies Orinoco Wifi card (PCMCIA) and OpenBSD?

[Mikael Bak]

Dorian B|ttner wrote:
 Mikael Bak schrieb:
 Hi list,

 Seems to me that this card isn't recognized at boot time (dmesg pasted
 at the end of this email). It's a PCMCIA card. Instead of a wifi
 device OpenBSD tries to allocate a serial port (com3).
   
 Hi Mikael,
 
 some time ago I had a usb wlan adapter from avaya at hand
 http://old.nabble.com/AVAYA-Wireless-USB-Client-%28Gold%29-td21817914.html
 It turned out, then when opening the cover of the desk stand it was
 nothing more than a pcmcia card attached to a usb adapter. Avaya used
 also orinoco chips in their products, and it was back in times of
 802.11b and wep was believed/announced to be totally secure.
 I suspect whether you have anything newer that would be worth
 investigating?
 
 Regards,
 Dorian
 
 

Hi Dorian,

Yes, this is a very old card, and it does only 802.11b and can only do
wep. I know it's not secure, I'd only use it at home for playing around
with and learn OpenBSD. If OpenBSD has no driver for it then I use wired
network. I just thought I'd ask in the list just in case.

I have been using this wifi card successfully in FreeBSD, so I thought
it might work in OpenBSD too.

Anyway. If somebody on the list has made this card work in OpenBSD then
I am all ears :-)

TIA,
Mikael

Lucent Technologies Orinoco Wifi card (PCMCIA) and OpenBSD?

[Mikael Bak]

Hi list,

Seems to me that this card isn't recognized at boot time (dmesg pasted at the 
end of this email). It's a PCMCIA card. Instead of a wifi device OpenBSD tries 
to allocate a serial port (com3).

If I should provide any more info or output, then please let me know.

TIA,
Mikael

My dmesg:

OpenBSD 4.6 (GENERIC) #58: Thu Jul  9 21:24:42 MDT 2009
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Celeron (GenuineIntel 686-class, 128KB L2 cache) 398 MHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR
real mem  = 267923456 (255MB)
avail mem = 250249216 (238MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 07/11/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS 
rev. 2.3 @ 0xf6490 (58 entries)
bios0: vendor Dell Computer Corporation version A14 date 07/11/2002
bios0: Dell Computer Corporation Latitude CPt C400GT
apm0 at bios0: Power Management spec V1.2
apm0: AC on, no battery
acpi at bios0 function 0x0 not configured
pcibios0 at bios0: rev 2.1 @ 0xf/0x1
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfbd80/112 (5 entries)
pcibios0: PCI Interrupt Router at 000:07:0 (Intel 82371 ISA and IDE rev 0x00)
pcibios0: PCI bus #3 is the last bus
bios0: ROM list: 0xc/0xe000 0xce000/0x800 0xce800/0x800 0xcf000/0x800 
0xcf800/0x800
cpu0 at mainbus0: (uniprocessor)
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 Intel 82443BX AGP rev 0x03
intelagp0 at pchb0
agp0 at intelagp0: aperture at 0xf400, size 0x400
ppb0 at pci0 dev 1 function 0 Intel 82443BX AGP rev 0x03
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 Neomagic Magicgraph NM2360 rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
neo0 at pci1 dev 0 function 1 Neomagic MagicMedia 256ZX rev 0x00: irq 5
ac97: codec id not read
audio0 at neo0
cbb0 at pci0 dev 3 function 0 TI PCI1225 CardBus rev 0x01: irq 11
cbb1 at pci0 dev 3 function 1 TI PCI1225 CardBus rev 0x01: irq 11
piixpcib0 at pci0 dev 7 function 0 Intel 82371AB PIIX4 ISA rev 0x02
pciide0 at pci0 dev 7 function 1 Intel 82371AB IDE rev 0x01: DMA, channel 0 
wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: FUJITSU MHH2064AT
wd0: 16-sector PIO, LBA, 6194MB, 12685680 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 ignored (disabled)
uhci0 at pci0 dev 7 function 2 Intel 82371AB USB rev 0x01: irq 11
piixpm0 at pci0 dev 7 function 3 Intel 82371AB Power rev 0x02: SMI
iic0 at piixpm0
spdmem0 at iic0 addr 0x50: 128MB SDRAM non-parity PC100CL3
cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 2 device 0 cacheline 0x8, lattimer 0x20
pcmcia0 at cardslot0
cardslot1 at cbb1 slot 1 flags 0
cardbus1 at cardslot1: bus 3 device 0 cacheline 0x8, lattimer 0x20
pcmcia1 at cardslot1
isa0 at piixpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
usb0 at uhci0: USB revision 1.0
uhub0 at usb0 Intel UHCI root hub rev 1.00/1.00 addr 1
biomask ef45 netmask ef45 ttymask ffdf
mtrr: Pentium Pro MTRR support
com3 at pcmcia1 function 0: can't allocate i/o space
apm0: battery life expectancy 0%
apm0: AC on, battery charge low, estimated 0:00 hours
softraid0 at root
root on wd0a swap on wd0b dump on wd0b

Re: Decrypt email in Sylpheed

[Mikael Bak]

On Wed, 2 Dec 2009 18:21:40 +0100
Mikael Bak mik...@t-online.hu wrote:

 Hi list,
 
 I know. This is not specific to OpenBSD at all. I just thought you guys use 
 gpg to sign and encrypt email a lot :-)
 
 So having OpenBSD 4.6 on my old laptop and one of my favourite email clients 
 (Sylpheed - installed with pkg_add, the flavor supporting gpg), I am able to 
 sign and encrypt email nicely. Sylpheed automagically opens a dialog and I 
 can choose key and enter passphrase.
 
 But when receiving encrypted email I am not able to decrypt.
 Sylpheed simply shows me the gpg encrypted data, and does not open dialog box 
 for entering passphrase.
 
 I folowed this doc (I have no idea if I should have done differently):
 https://help.launchpad.net/ReadingOpenPgpMail#Sylpheed
 
 It states that Sylpheed is unable to decode the message inline. OK. But after 
 following this guide I'm still not able to decrypt. I still do not have a 
 chance to enter my passphrase. The output from the action say bad 
 passphrase.
 
 So if anyone of you uses gpg together with Sylpheed and OpenBSD and got it 
 working, I'm all ears :-)
 

Replying to myself on this one.

Seem it's been a while since I used Sylpheed. I found out that there is a new 
one called claws-mail. It has nice plugins that handles the functionality I 
need.

Thanks and sorry for the noise.
Mikael

Re: Scroll with laptop touchpad

[Mikael Bak]

Dope Ice Apollyon the Third wrote:
 On Tue, Dec 1, 2009 at 3:13 PM, Abel Abraham Camarillo Ojeda
 acam...@the00z.org wrote:
 On Tue, Dec 01, 2009 at 02:36:02PM +0100, Mikael Bak wrote:
 Hi list,
 I'm really new to openbsd, so please forgive me if this is faq or rtfm.

 I did try to search for information on how to be able to scroll with my
 laptop touchpad, but did not find any openbsd specific documentation.
 I'd be happy if someone could point me to any documentation describing how
 to do this in openbsd.
 My system:
 $ uname -a
 OpenBSD neo.my.domain 4.6 GENERIC#58 i386

 My laptop:
 Dell Latitude CPt 400
 (it's an old P2 400MHz)

 In WinXP a driver from synaptics made the scrolling work.

 TIA,
 --
 Mikael Bak mik...@t-online.hu

 First... investigate if the scrolling ins'n in hardware...

 (I read the manual of my eeepc and it said everything about scrolling,
  because the eeepc have some interesting ways to do it)
 
 It's not in hardware. On linux this is supported by the synaptics X
 touchpad driver. I would also really like to see this work on OpenBSD
 but I'm not awesome enough to know why it doesn't. I took a stab at
 compiling the synaptics driver (which you can google for) and it, of
 course, failed miserably (and yes I used gmake).
 
 I know that some features need a multitouch touchpad, but simple
 scrolling should just be able to work with any touchpad that can give
 an x,y coordinate. It's a pity.
 
 -Nick
 

Abel, Nick,
Thanks both of you for responding!

I also have the feeling that basic scrolling shouldn't depend on
specific hardware. I have used this same hw in WinXP and successfully
made the touchpad scroll. Zooming and other features may be hardware
dependent.

Nick, you are telling me that people use openbsd and X and surf the web
as their primary OS without missing this feature? :-) We are the only
ones who whould like to use our old laptops this way? :-)

OK. I know the gig. If I want it to work I should download the source
code and fix it, then post the fix to a list with developers who can
review and submit the patches. I know that. I was just surprised such
basic thing haven't been targeted yet.

I just took a look at my other laptop (running Ubuntu) and its xorg.conf
has this:
Section InputDevice
Identifier  Synaptics Touchpad
Driver  synaptics
Option  SendCoreEventstrue
Option  Device/dev/psaux
Option  Protocol  auto-dev
Option  HorizEdgeScroll   0
EndSection

It seems to be a similar way in FreeBSD (PCBSD):
http://forums.pcbsd.org/viewtopic.php?f=30t=9249

Oh well. If someone had any luck with this on OpenBSD, then please tell us.

TIA,
Mikael

Decrypt email in Sylpheed

[Mikael Bak]

Hi list,

I know. This is not specific to OpenBSD at all. I just thought you guys use gpg 
to sign and encrypt email a lot :-)

So having OpenBSD 4.6 on my old laptop and one of my favourite email clients 
(Sylpheed - installed with pkg_add, the flavor supporting gpg), I am able to 
sign and encrypt email nicely. Sylpheed automagically opens a dialog and I can 
choose key and enter passphrase.

But when receiving encrypted email I am not able to decrypt.
Sylpheed simply shows me the gpg encrypted data, and does not open dialog box 
for entering passphrase.

I folowed this doc (I have no idea if I should have done differently):
https://help.launchpad.net/ReadingOpenPgpMail#Sylpheed

It states that Sylpheed is unable to decode the message inline. OK. But after 
following this guide I'm still not able to decrypt. I still do not have a 
chance to enter my passphrase. The output from the action say bad passphrase.

So if anyone of you uses gpg together with Sylpheed and OpenBSD and got it 
working, I'm all ears :-)

TIA,
Mikael

Scroll with laptop touchpad

[Mikael Bak]

Hi list,
I'm really new to openbsd, so please forgive me if this is faq or rtfm.

I did try to search for information on how to be able to scroll with my laptop 
touchpad, but did not find any openbsd specific documentation.

I'd be happy if someone could point me to any documentation describing how to 
do this in openbsd.

My system:
$ uname -a 
OpenBSD neo.my.domain 4.6 GENERIC#58 i386

My laptop:
Dell Latitude CPt 400
(it's an old P2 400MHz)

In WinXP a driver from synaptics made the scrolling work.

TIA,
-- 
Mikael Bak mik...@t-online.hu

Re: relayd http-https-redirects with sticky-address

[Mikael Jansson]

Thanks for your replies!  I agree that this should be solved in the web-app,
but in the meantime I'll try reyks workaround.

Regards, Mikael

2008/9/17 Michiel van Baak [EMAIL PROTECTED]:
 On 21:39, Wed 17 Sep 08, Reyk Floeter wrote:
 Hi!

 On Wed, Sep 17, 2008 at 05:45:23PM +0200, Mikael Jansson wrote:
  I use relayd with redirects to loadbalance between two webservers
  one redirect is used for http requests and the other for https.
  the redirects looks like the following:
 
  redirect web_http {
listen on $ext_ip1 port http
sticky-address
forward to webservers port http check script /usr/local/sbin/chksrvs
  }
 
  redirect web_https {
listen on $ext_ip1 port https
sticky-address
forward to webservers port https check script /usr/local/sbin/chksrvs
  }
 
  The redirects works fine separately and sticks to the same machine,
  but when the user navigates from http to https the requests sometimes
  move over to the other machine.  I need the same source-ip to always
  stay on the same server regardless of which destination port (http or 
  https)
  is being used.  Any suggestions on how to achive this would be greatly
  appreciated.
 

 it does not work without a patch.  the problem is that the pf Source
 Tracking table includes a reference to the rule but your example above
 will load two different rules into pf - one matching http and another
 one matching https.

 the trick is to combine both statements into one rule.  we don't
 support port tables in pf yet (which whould be very helpful in many
 cases) but there is support for a port range.  so the hack is to
 allow port ranges in the relayd redirection block

 redirect web {
   listen on $ext_ip1 port 80:443
   sticky-address
   forward to webservers port http check script 
 /usr/local/sbin/chksrvs
 }

 note that this will match any traffic in the 80 - 443 port range, make
 sure that you add additional pf rules to filter any other ports except
 80 and 443.  but it works with Source Tracking and should allow your
 clients to move between http and https on the same server.  another
 limitation is that it only runs checks on one of the ports.

 ugh, this looks ugly ;)
 Instead of going this route I would say: find the source of why the
 visitor should access the same host, and solve that.

 We use relayd in front of 6 servers, doing http and https.
 It doesn't matter what backend box the user go. Hell, they can even go
 to another box on a reload.
 This of course means we are storing sessions etc on shared storage (NFS
 in our case, and the new sharedance port looks like an alternative for
 that)

 --

 Michiel van Baak
 [EMAIL PROTECTED]
 http://michiel.vanbaak.eu
 GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0x71C946BD

 Why is it drug addicts and computer aficionados are both called users?

relayd http-https-redirects with sticky-address

[Mikael Jansson]

Hello,

I use relayd with redirects to loadbalance between two webservers
one redirect is used for http requests and the other for https.
the redirects looks like the following:

redirect web_http {
  listen on $ext_ip1 port http
  sticky-address
  forward to webservers port http check script /usr/local/sbin/chksrvs
}

redirect web_https {
  listen on $ext_ip1 port https
  sticky-address
  forward to webservers port https check script /usr/local/sbin/chksrvs
}

The redirects works fine separately and sticks to the same machine,
but when the user navigates from http to https the requests sometimes
move over to the other machine.  I need the same source-ip to always
stay on the same server regardless of which destination port (http or https)
is being used.  Any suggestions on how to achive this would be greatly
appreciated.

Regards, Mikael

Re: Dell RAID controller

[Mikael Kermorgant]

  perc6e/i are sas controllers, you can plug either sata or sas
  disks into them.  They should work fine with 4.3.

Hello,

Is there any way to check raid status without having to reboot and get
into the bios ?

Regards,

-- 
Mikael Kermorgant

2 carp devices for same IP on same host (with 2 nics)

[Mikael Kermorgant]

Hello,

I'm working on testing this network topology :
http://kgt.free.fr/objectif-net2.png

I'm focusing on the inside side of fw1, which is linked (red cables)
to ifw1 and ifw2 for high availability.
These 2 nics are pcn2 and pcn3. I've configured them this way :

pcn2 : 10.1.1.11
pcn3 : 10.1.1.12
# cat /etc/hostname.carp1
inet 10.1.1.1 255.255.255.0 10.1.1.255 vhid 2 carpdev pcn2 advskew 0
# cat /etc/hostname.carp2
inet 10.1.1.1 255.255.255.0 10.1.1.255 vhid 2 carpdev pcn3 advskew 10

When I start the network, carp1 gets MASTER role but carp2 is on INIT
state and not backup.
I've checked /etc/pf.conf and it's ok.

Problem is that carp2 never gets MASTER when I take down pcn2...

Do you know if there's something inherently wrong with setting up 2
carp devices for the same IP on the same host ?

Thanks in advance,

Mikael Kermorgant

-- 
Mikael Kermorgant

Re: 2 carp devices for same IP on same host (with 2 nics)

[Mikael Kermorgant]

  What's the point behind this setup ? It doesn't make any sense!

  John

Well, it makes some sort of sense for me (but as I'm no expert, could
be a sweet dream :) ) so it's best I try to share what I'm looking for
:

There are 2 level of firewalls :
1st with fw1  fw2 protects from internet and manages DMZ
2nd with ifw1  ifw2 manages inter-vlan filtering

I'd like to achive high availability accross these 2 levels, without
the need for a switch between, hence the four red cables.
To be precise, it's also because I want to be able to unplug ifw1
(which leads ifw2 to take over) without having fw2 taking over fw1
(which would be the case if I'd only have one nic toward the inside on
fw1) .

Therefore, if you unplug the link between ifw1 and fw1 (pcn2), pcn3 on
fw1 should be elected as master and talk to the new master on the
other side.

So, have I changed your mind about it ?

Best regards,

-- 
Mikael Kermorgant

Re: 2 carp devices for same IP on same host (with 2 nics)

[Mikael Kermorgant]

On Mon, Apr 14, 2008 at 11:16 PM, Tom Geman [EMAIL PROTECTED] wrote:


   Problem is that carp2 never gets MASTER when I take down pcn2...

  I have never tried the setup you are proposing, but something doesn't seem 
 right.  Shouldn't both NICs belong to the same carp1?  What happens if you 
 try:


  # cat /etc/hostname.carp1
  inet 10.1.1.1 255.255.255.0 10.1.1.255 vhid 2 carpdev pcn2 advskew 0


 inet 10.1.1.1 255.255.255.0 10.1.1.255 vhid 2 carpdev pcn3 advskew 10

Thanks,
I can't try right now but I hope I'll be able tomorrow.

Anyway, it could be that my problem is related to the preempt option.
The man page I should have looked at before posting says this :

 For firewalls and routers with multiple interfaces, it is desirable to
 failover all of the carp interfaces together, when one of the physical
 interfaces goes down.  This is achieved by the preempt option.  Enable it
 on both host A and B

So when I take pcn2 down, preemt probably takes all carp devices down,
including the one that should become MASTER...

If your solution does not work, I thought I'd try a failover trunking
of pcn2 and pcn3, giving some trunk0 interface I could associate with
a single carp device.

I'll keep this updated asap.

Best regards,

-- 
Mikael Kermorgant

Re: 4.2 and em(4)

[Mikael Kermorgant]

Hello,

I'd like to jump on what you said about separate buses because I
haven't looked at this before.

You made me curious to understand this dmesg output :

cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 Intel 82Q965 Host rev 0x02
agp0 at pchb0: aperture at 0xd000, size 0x800
ppb0 at pci0 dev 1 function 0 Intel 82Q965 PCIE rev 0x02: irq 14
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 Intel PCIE-PCIE rev 0x09
pci2 at ppb1 bus 2
ppb2 at pci1 dev 0 function 2 Intel PCIE-PCIE rev 0x09
pci3 at ppb2 bus 3
vga1 at pci0 dev 2 function 0 Intel 82Q965 Video rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
ppb3 at pci0 dev 28 function 0 Intel 82801H PCIE rev 0x02: irq 14
pci4 at ppb3 bus 4
em0 at pci4 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
14, address 00:10:f3:10:7e:68
ppb4 at pci0 dev 28 function 1 Intel 82801H PCIE rev 0x02: irq 10
pci5 at ppb4 bus 5
em1 at pci5 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
10, address 00:10:f3:10:7e:69
ppb5 at pci0 dev 28 function 2 Intel 82801H PCIE rev 0x02: irq 11
pci6 at ppb5 bus 6
em2 at pci6 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
11, address 00:10:f3:10:7e:6a
ppb6 at pci0 dev 28 function 3 Intel 82801H PCIE rev 0x02: irq 15
pci7 at ppb6 bus 7
em3 at pci7 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
15, address 00:10:f3:10:7e:6b
ppb7 at pci0 dev 28 function 4 Intel 82801H PCIE rev 0x02: irq 14
pci8 at ppb7 bus 8
em4 at pci8 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
14, address 00:10:f3:10:7e:6c
ppb8 at pci0 dev 28 function 5 Intel 82801H PCIE rev 0x02: irq 10
pci9 at ppb8 bus 9
em5 at pci9 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
10, address 00:10:f3:10:7e:6d
uhci0 at pci0 dev 29 function 0 Intel 82801H USB rev 0x02: irq 5
uhci1 at pci0 dev 29 function 1 Intel 82801H USB rev 0x02: irq 15
ehci0 at pci0 dev 29 function 7 Intel 82801H USB rev 0x02: irq 5
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1
ppb9 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0xf2
pci10 at ppb9 bus 10
em6 at pci10 dev 14 function 0 Intel PRO/1000MT (82541GI) rev 0x05:
irq 11, address 00:10:f3:10:7e:6e
em7 at pci10 dev 15 function 0 Intel PRO/1000MT (82541GI) rev 0x05:
irq 10, address 00:10:f3:10:7e:6f


Just by reading this :

pci4 at ppb3 bus 4
em0 at pci4 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: irq
14, address 00:10:f3:10:7e:68
ppb4 at pci0 dev 28 function 1 Intel 82801H PCIE rev 0x02: irq 10
pci5 at ppb4 bus 5
em1 at pci5
--
I'd deduce em0 (pci4, bus 4) and em1 (pci5, bus 5) are on separate
buses... but am I right  ?
But em6 and em7 are on the same bus, right ?

Thanks in advance,

Mikael Kermorgant



On Mon, Apr 14, 2008 at 11:14 PM, scott [EMAIL PROTECTED] wrote:
 We've found the best gateway box -- pf, sshd for ssh -w vpn and ipsec
  clients, spamd, etc. -- is non-MP, as follows.

  A) Use a box with the fastest memory bandwidth (and latency) your budget
  -- cash or time spent scrounging -- can afford/acquire.  (e.g.  on a
  P-III 1 GHz machine, we saw meaningful better top-end results on our
  stress tests between using PC133 vs PC100 and again between PC133 CL2.5
  vs CL3 memory sticks.)

  B.1) Server-class motherboards usually have multiple PCI buses (say
  again, buses, not slots).  Opposing the em(4) nics on separate
  buses, with regard to in-to-out flows, helps quite a bit too.  e.g
  internet --- (em0)(bus1)(pf)(bus2)(em1) --- LAN.

  B.2) Once a while back, we did see some positive affect by trying to
  share the driver-IRQ for the like em(4).  But not too sure about this
  one.

  C) We found, on 4.2, if your mb will play nicely, expressly enabling
  ACPI (vs. default APM) functionality seemed to improve the the boxes
  throughput too. In our case, INTEL MOTHERBOARDS.  Your mb may not like
  this, though, so use with care and/or wait to 4.3 release.



  -Original Message-
  From: Stuart Henderson [EMAIL PROTECTED]
  To: misc@openbsd.org
  Subject: Re: 4.2 and em(4)
  Date: Mon, 14 Apr 2008 16:23:24 + (UTC)
  Mailer: slrn/0.9.8.1 (OpenBSD)
  Delivered-To: [EMAIL PROTECTED]

  On 2008-04-14, Joe Warren-Meeks [EMAIL PROTECTED] wrote:
  
   If the box was only doing pf stuff, then that would be correct. If you
   were to put a bunch of ftp-proxys on there too, then MP would help, no?

  very little, the bulk data handling is done in kernel by nat/rdr
  rules added to the anchors, ftp-proxy only touches the control
  connections.





-- 
Mikael Kermorgant

Disable power button

[Mikael Nyström]

Is there a way to disable the power button on OpenBSD, like FreeBSD's 
sysctl hw.acpi.power_button_state=NONE or similar?

I'm running OpenBSD 4.2.

//Micke

Re: nat or routing problem?

[Mikael Fridh]

Mitja wrote:

Mitja wrote:


Andreas Bihlmaier wrote:


On Thu, Dec 07, 2006 at 11:27:11PM +0100, Mitja wrote:


Hello,

I am trying to configure nat from internal network 192.168.1.0/24 to
external nat gateway address 193.189.180.193. The problem is that
packets are not passing from nat gateway to the interface 193.77.12.154
to the internet.

ISP - 193.77.12.154 -- hostA -- 192.168.1.1
  |
193.189.180.193 (em1)
  |
  /27 network



More testing:
I changed my pf.conf to:

# pfctl -s all
TRANSLATION RULES:
nat on bge0 inet from 192.168.1.0/24 to any - (bge0:0)
rdr pass on em1 inet proto tcp from any to any port = 5900 -
192.168.1.111 port 5900


If bge0 is your external interface that nat line now looks correct.
If your internal hosts on the 192.168.1.0/24 net have default gateway 
192.168.1.1 it should be nating properly.


Sounds like you want traffic to traverse:
192.168.1.0/24 - 192.168.1.1 - 193.189.180.193 - 193.77.12.154 - 0/0

I don't yet fully get what you're trying to accomplish.


# pfctl -s all
TRANSLATION RULES:
nat on em1 inet from 192.168.1.0/24 to any - (em1:0)
rdr pass on em1 inet proto tcp from any to any port = 5900 -
192.168.1.111 port 5900

FILTER RULES:
pass in all keep state
pass out all keep state
No queue in use

The same test, with tcpdump on the last interface (bge0;193.77.12.154).

# ping -I 192.168.1.95 209.85.129.147
PING 209.85.129.147 (209.85.129.147): 56 data bytes
--- 209.85.129.147 ping statistics ---
15 packets transmitted, 0 packets received, 100.0% packet loss

# tcpdump -i bge0 icmp
tcpdump: listening on bge0, link-type EN10MB
14:49:16.377482 192.168.1.95  209.85.129.147: icmp: echo request
14:49:17.387437 192.168.1.95  209.85.129.147: icmp: echo request
14:49:18.397398 192.168.1.95  209.85.129.147: icmp: echo request

icmp packets are going out, but it looks like NAT is not working (it
should change my source IP address).


That's because now you are dumping traffic on the internal interface 
where the packets hasn't traversed the NAT yet.
The nat rule you made above has the internal interface where it should 
have the external:

nat on em1:0 from int_net to - em1:0.


# This is a proper simple nat example (that works):
ext_if=rl0 # (or whatever is your external interface)
nat on $ext_if inet from ! ($ext_if) - ($ext_if:0)

--
Fridh

Openbsd and vsftpd with virtual users

[Mikael Jirari]

Hello,

I'd like to install vsftpd with virtual users on my openbsd system.
I read the documentation and at step 2 it says:

Step 2) Create a PAM file which uses your new database.

See the example file vsftpd.pam. It contains two lines:
auth required /lib/security/pam_userdb.so db=/etc/vsftpd_login
account required /lib/security/pam_userdb.so db=/etc/vsftpd_login

This tells PAM to authenticate users using our new database. Copy this
PAM
file to the PAM directory - typically /etc/pam.d/
cp vsftpd.pam /etc/pam.d/ftp

The problem is that there is no pam.d with OpenBsd.

I read elsewhere that some people say that ldap could be used but I'd
like to know if there is a simple way to configure vsftpd with virtual
users that requires a minimal configuration.

Also, I'd like to know how to start the server after I installed it
through the ports because I didn't find so far.

Thanx

Virtual users with openbsd and vsftpd

[Mikael Jirari]

Hello,

I'd like to have vsftpd virtual users on my openbsd system.
I followed the readme on how to set up such feature but on step 2 they
ask to this:

Step 2) Create a PAM file which uses your new database.
See the example file vsftpd.pam. It contains two lines:
auth required /lib/security/pam_userdb.so db=/etc/vsftpd_login
account required /lib/security/pam_userdb.so db=/etc/vsftpd_login
This tells PAM to authenticate users using our new database. Copy this
PAM
file to the PAM directory - typically /etc/pam.d/
cp vsftpd.pam /etc/pam.d/ftp

The problem is that there is no pam.d on openbsd as far as I know.
I read some people would install Ldap but I'm wondering if there was an
easy way to do it without installing ldap.

Thanx