Re: OpenBSD 7.5 - relayd -> vaultwarden - websockets payload not working

[Ollie Strickland]

Pardon me for not sending plain text the first time. Got trigger happy.

Ollie


I have been running the Vaultwarden password manager behind relayd for a couple 
of years now, and have spun up a new 7.5 VM on Vultr to test.

I'm using pkg_add to install the binary package for the 7.5 release - 
vaultwarden-1.30.5, so nothing nonstandard.

The problem - Vaultwarden uses a websockets connection to push changes to user 
data in real time to all connected devices, and on 7.5 with relayd acting as 
reverse proxy, websockets sessions get established successfully, but no payload 
is able to pass from the server to the client.

Here are two images that show the dev console in Firefox - 
https://imgur.com/a/msvyXbX

The first image shows websockets working correctly when public traffic is 
directed to Vaultwarden's Rocket server without using relayd as a reverse proxy.

The second image shows relayd in place; no websockets payload can pass and the 
Vaultwarden application cannot push changes to user data.

Relayd worked great for Vaultwarden in 7.4 and earlier. I saw that relayd got 
touched in the changelogs.

My relayd.conf is:


table  { localhost }

# protocol definition for vaultwarden with tls
http protocol vaultwarden-https {

# forward connections to vaultwarden rocket
match request path "/*" forward to 

# add headers vaultwarden may need
match request header append "Host" value "$HOST"
match request header append "X-Real-IP" value "$REMOTE_ADDR"
match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
match request header append "X-Forwarded-By" value 
"$SERVER_ADDR:$SERVER_PORT"
match request header append "CF-Connecting-IP" value "$REMOTE_ADDR"

# various TCP options
tcp { nodelay, sack, backlog 128 }

# tls config
tls keypair vault.example.com
tls { no tlsv1.0, ciphers HIGH }

# allow websockets - this is nice it handles all the headers no need 
for manual header edits
http websockets
}

# relay definition for vaultwarden - forward inbound 443 tls on the egress 
interface to rocket on default port 8000
relay vaultwarden-https-relay {
listen on egress port 443 tls
protocol vaultwarden-https
forward to  port 8000
}


And dmesg (VM on vultr) is:


OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4278042624 (4079MB)
avail mem = 4127375360 (3936MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC HPET MCFG WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC-Rome Processor, 1996.57 MHz, 17-31-00
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC-Rome Processor, 1996.74 MHz, 17-31-00
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu1: smt 1, core 0, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpihpet0 at acpi0: 1 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xb000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured

OpenBSD 7.5 - relayd -> vaultwarden - websockets payload not working

[Ollie Strickland]

I have been running the Vaultwarden password manager behind relayd for a couple 
of years now, and have spun up a new 7.5 VM on Vultr to test.

I'm using pkg_add to install the binary package for the 7.5 release - 
vaultwarden-1.30.5, so nothing nonstandard.

The problem - Vaultwarden uses a websockets connection to push changes to user 
data in real time to all connected devices, and on 7.5 with relayd acting as 
reverse proxy, websockets sessions get established successfully, but no payload 
is able to pass from the server to the client.

Here are two images that show the dev console in Firefox - 
https://imgur.com/a/msvyXbX

The first image shows websockets working correctly when public traffic is 
directed to Vaultwarden's Rocket server without using relayd as a reverse proxy.

The second image shows relayd in place; no websockets payload can pass and the 
Vaultwarden application cannot push changes to user data.

Relayd worked great for Vaultwarden in 7.4 and earlier. I saw that relayd got 
touched in the changelogs.

My relayd.conf is:

table  { localhost }

# protocol definition for vaultwarden with tls
http protocol vaultwarden-https {

# forward connections to vaultwarden rocket
match request path "/*" forward to 

# add headers vaultwarden may need
match request header append "Host" value "$HOST"
match request header append "X-Real-IP" value "$REMOTE_ADDR"
match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
match request header append "X-Forwarded-By" value 
"$SERVER_ADDR:$SERVER_PORT"
match request header append "CF-Connecting-IP" value "$REMOTE_ADDR"

# various TCP options
tcp { nodelay, sack, backlog 128 }

# tls config
tls keypair vault.example.com
tls { no tlsv1.0, ciphers HIGH }

# allow websockets - this is nice it handles all the headers no need 
for manual header edits
http websockets
}

# relay definition for vaultwarden - forward inbound 443 tls on the egress 
interface to rocket on default port 8000
relay vaultwarden-https-relay {
listen on egress port 443 tls
protocol vaultwarden-https
forward to  port 8000
}


And dmesg (VM on vultr) is:

OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4278042624 (4079MB)
avail mem = 4127375360 (3936MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP APIC HPET MCFG WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC-Rome Processor, 1996.57 MHz, 17-31-00
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC-Rome Processor, 1996.74 MHz, 17-31-00
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 
8-way L2 cache, 16MB 64b/line 16-way L3 cache
cpu1: smt 1, core 0, package 0
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpihpet0 at acpi0: 1 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xb000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0: 0x0010 0x0011 0x
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
pvbus0 at mainbus0: KVM
pvclock0 at pvbus0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 

Re: sysupgrade on pcEngines apu2 boards hangs

[Glen Gunsalus]

Successful upgrades since 6.2

# df -h
Filesystem SizeUsed   Avail Capacity  Mounted on
/dev/sd0a 14.4G4.9G8.8G36%/
/dev/sd1a  3.6G2.0K3.4G 1%/scratch


On 4/7/24 14:28, deich...@placebonol.com wrote:

Do you have enough available space on partition?  You didn't mention how many 
OS upgrades you've done on these systems.

diana


On April 7, 2024 3:04:53 PM MDT, Glen Gunsalus  
wrote:

I have been running OpenBSD on three apu2 boards (as firewalls) for several 
years and doing remote (ssh to boards) with no problems.

Doing the 7.4 to 7.5 upgrades I had an initial success then two failures. 
Not confidence building since these firewalls are at remote sites.

Not sure where the problem lies, here is what I've experienced:

pcEngines apu2 boards, running sysupgrade - have been running OpenBSD
since about 6.2 and doing remote (ssh to board) for several releases
w/o any problems

Did successful sysupgrade on one board the evening before 7.5 release
notification using: url https://cdn.openbsd.org/pub/OpenBSD/ 
 (other
mirrors not yet showing items in 7.5 directory)

Went fine, as per norm.

Next day did on a second board using my default url (as per all previous
remote upgrades) https://mirrors.sonic.net/pub/OpenBSD/ 


after syspugrade pings ok but cannot ssh:
ssh: connect to host 10.42.xx.xx port 22: Connection refused

used minicom to hook up to serial port and was able to see (didn't
require login??):

# ls -al
total 166
drwxr-xr-x 11 root wheel 512 Apr 7 18:34 .
drwxr-xr-x 11 root wheel 512 Apr 7 18:34 ..
-rw-r--r-- 1 root wheel 1770 Mar 20 21:53 .profile
-rw-r--r-- 1 root wheel 194 Apr 7 18:34 auto_upgrade.conf
lrwxr-xr-x 1 root wheel 11 Mar 20 21:53 autoinstall -> install.sub
drwxr-xr-x 2 root wheel 512 Mar 20 21:53 bin
drwxr-xr-x 2 root wheel 2560 Apr 7 18:34 dev
drwxr-xr-x 5 root wheel 512 Apr 7 18:34 etc
lrwxr-xr-x 1 root wheel 11 Mar 20 21:53 install -> install.sub
-rw-r--r-- 1 root wheel 2903 Mar 20 21:53 install.md
-rwxr-xr-x 1 root wheel 64483 Mar 20 21:53 install.sub
drwxr-xr-x 15 root wheel 512 Apr 7 18:36 mnt
drwxr-xr-x 2 root wheel 512 Mar 20 21:53 mnt2
drwxr-xr-x 2 root wheel 1024 Mar 20 21:53 sbin
drwxrwxrwt 4 root wheel 512 Apr 7 18:36 tmp
lrwxr-xr-x 1 root wheel 11 Mar 20 21:53 upgrade -> install.sub
drwxr-xr-x 6 root wheel 512 Mar 20 21:53 usr
drwxr-xr-x 6 root wheel 512 Mar 20 21:53 var

reboot from serial console with minicom

Comes back up (can ssh now) but with only one cpu (normally running mp - 
cpu[0-3] )

reboot

Watching on serial port

reorder_kernel: failed -- see /usr/share/relink/kernel/GENERIC/relink.log

as per log: run 'sha256 -h /var/db/kernel.SHA256 /bsd'

reboot, no error, but still in single cpu mode.

reboot bsd.mp

hang, ssh connection refused

Power cycle to get fresh reboot but still hangs

still on serial port: boot usb with miniroot75.img

reboot - needed to reset - 'stty com0 115200; set tty com0', otherwise hangs

Choose upgrade

Success!

This happened on a third apu2 board; thus, two unsucessful and one 
successful sysupgrades on apu2 boards

Any ideas/pointers? I'd like this not to reoccur.

Re: ipv6 assistance

[Courtney Hicks]

I’ll have to get back to you on this tomorrow, but I JUST got IPv6 working 
again with Comcast. The short answer: 

This helped me: https://lipidity.com/openbsd/router/

Recently Comcast changed how their dhcpv6 servers work in my area and I had to 
change the dhcpv6-server rule to this

pass in on egress inet6 proto udp \
from { fe80::/10 2001:558::/32 } port dhcpv6-server \
to fe80::/10 port dhcpv6-client \
no state

The difference being adding the 2001:558::/32 address, being Comcast’s IPv6 
block they talk over. Before this I had v6 working for years until a 
maintenance window and then it automagically stopped.

Courtney

> On Apr 6, 2024, at 8:12 AM, Sonic  wrote:
> 
> 
> Running -current on my router and finally (after years) decided to move into 
> using ipv6.
> I added "inet6 autoconf" to hostname.em0 (also has "inet autoconf") and I get 
> a link local address:
> =
> # ifconfig em0  
> em0:inet6 fe80::2132:31ff:fe0b:7ea4%em0 prefixlen 64 scopeid 0x1
> inet 69.31.273.6 netmask 0xfc00 broadcast 69.31.273.255
> =
> And an ipv6 default route:
> =
> Internet6:
> Destination Gateway   
>   Flags   Refs  Use   Mtu  Prio Iface
> default fe80::301:5bcf:fe75:2646%em0  
>   UGS0   22 - 8 em0
> =
> Which matches the default router proposal listed by slaacctl:
> =
> em0:
>  index:   1 running: yes temporary: yes
> lladdr: 40:62:31:0b:7e:a4
>  inet6: fe80::2132:31ff:fe0b:7ea4%em0
> Router Advertisement from fe80::201:5cff:fe75:2646%em0
> received: 2024-04-06 10:49:17; 0s ago
> Cur Hop Limit:   0, M: 1, O: 1, Router Lifetime:  1800s
> Default Router Preference: Medium
> Reachable Time:   360ms, Retrans Timer:  1000ms
> prefix: 2001:623:8016:54::/64
> On-link: 0, Autonomous address-configuration: 0
> vltime: 604800, pltime: 302400
> prefix: 2001:623:6007:a5::/64
> On-link: 0, Autonomous address-configuration: 0
> vltime: 604800, pltime: 302400
> prefix: 2001:623:500e:16::/64
> On-link: 0, Autonomous address-configuration: 0
> vltime: 604800, pltime: 302400
> prefix: 2001:623:4020:a5::/64
> On-link: 0, Autonomous address-configuration: 0
> vltime: 604800, pltime: 302400
> Default router proposals
> id:1, state: PROPOSAL_CONFIGURED
> router: fe80::301:5bcf:fe75:2646%em0
> router lifetime:   1800
> Preference: Medium
> updated: 2024-04-06 10:49:17; 0s ago, timeout:   1788s
> =
> However, there's no other ipv6 address on the interface - I suspect an 
> address from one of those 2001: prefix groups needs to be assigned.
> Should not dhcpleased handle this?
> Most of the web posts I find deal with the pre-dhcpleased days.
> 
> I'm on Comcast (Xfinity) in the US.
> 
> Thank you,
> Chris
> 
> 
> 

Libressl verify failure with 3.9.0

[Ted Wynnychenko]

Hello,

I recently updated to -current (about a week ago).

I see that Libressl is at 3.9.1 just now, but I hope that won't be an issue 
(I did not see anything in the release notes that would impact my question).
---
$ openssl version
LibreSSL 3.9.0
---

Over the years, I have made certificates for personal servers/resources on 
my home network.  This is just for me, so I do some things that would be 
frowned on (although, technically, there is nothing "wrong" with them).

In this case, since I have Apple iOS devices that I want to connect to 
https, I backdate any certificates I create to 1/2/2019.  Apple has imposed 
a 300 or 800 day time limit on the validity for certificates created after 
(about) 7/1/2019.  Since I don't want to constantly make new certificates 
for my personal/home network, I have just been setting the certificates' 
"not before" date to early 2019.

Anyway, this had worked fine.
In fact, earlier this year (Jan 2024), I created a new certificate, and all 
is good.

A few weeks ago, I added a new thing to the network - a raspberry pi (I got 
as a gift about 2013 and installed a linux image from 2019 on it) that is 
connected to the home alarm system.

Since I was annoyed that my browser was constantly giving me self-signed 
certificate warnings, I decided to make a certificate for the nginx running 
on this appliance.

I created a key, made a csr, and then signed it with:
openssl ca -startdate 2019010200Z -in pi.csr -out pi.pem -config 
/etc/ssl/openssl.cnf

This all works fine, and a certificate is created

When I check with:
openssl x509 -text -noout -in pi.pem

everything seems as expected, including the not before/after dates:

Validity
Not Before: Jan  2 00:00:00 2019 GMT
Not After : Apr  7 15:39:59 2054 GMT

(yes, it is valid for 35 years - as I said before, if someone breaks into my 
house to secretly do things, I have way bigger problems)

But, if I try to verify this on the openbsd system, I get:

# openssl verify pi.pem
C = US, ST = Illinois, L = ***, O = ***, OU = ***, CN = ***
error 20 at 0 depth lookup:unable to get local issuer certificate
pi.pem: verification failed: 20 (unable to get local issuer certificate)
---

But, if I install this on the raspberry pi, which has a much older version 
of openssl on it:
$ openssl version
OpenSSL 1.1.1c  28 May 2019

The certificate verifies without an issue:
$ openssl verify pi.pem
pi.pem: OK

The last time I created a certificate was in January of this year 
(1/22/2024).
I am thinking the openbsd system was using Libressl 3.8.2 at that point.

I created that certificate in the exact same way, backdating the start date:
openssl ca -startdate 2019010200Z -in 54.csr -out 54.pem -config 
/etc/ssl/openssl.cnf

This previously created certificate also has them same backdated and very 
long valid period:

Validity
Not Before: Jan  2 00:00:00 2019 GMT
Not After : Jan 21 23:49:22 2054 GMT

(Notice the not after date is a little different)
Today, with the new libressl, this certificate verifies OK.

$ openssl verify 54.pem
54.pem: OK

Finally, if I create the new certificate WITHOUT backdating it
e.g.:  openssl ca -in pi.csr -out pi.pem -config /etc/ssl/openssl.cnf

The certificate is created and verifies OK.

So, it seems, there is some sort of issue with backdating the certificate, 
but not an issue with the crazy long validity window, that was not present 
in January of this year.

However, as I said, if I don't backdate, then in about a year the ipad will 
refuse to connect because of the restrictions apple has imposed, unless I 
update the certificate.

I know this is not "best practice," but it should still work, right?

Is there something I am missing?
Otherwise, it appears something has changed in Libressl 3.9.0 but is not 
documented.

Thanks in advance for any suggestions.
Ted

Re: sysupgrade on pcEngines apu2 boards hangs

[deich...@placebonol.com]

Do you have enough available space on partition?  You didn't mention how many 
OS upgrades you've done on these systems.

diana

On April 7, 2024 3:04:53 PM MDT, Glen Gunsalus  
wrote:
>I have been running OpenBSD on three apu2 boards (as firewalls) for several 
>years and doing remote (ssh to boards) with no problems.
>
>Doing the 7.4 to 7.5 upgrades I had an initial success then two failures. Not 
>confidence building since these firewalls are at remote sites.
>
>Not sure where the problem lies, here is what I've experienced:
>
>pcEngines apu2 boards, running sysupgrade - have been running OpenBSD
>since about 6.2 and doing remote (ssh to board) for several releases
>w/o any problems
>
>Did successful sysupgrade on one board the evening before 7.5 release
>notification using: url  https://cdn.openbsd.org/pub/OpenBSD/ (other
>mirrors not yet showing items in 7.5 directory)
>
>Went fine, as per norm.
>
>Next day did on a second board using my default url (as per all previous
>remote upgrades)  https://mirrors.sonic.net/pub/OpenBSD/
>
>after syspugrade pings ok but cannot ssh:
>ssh: connect to host 10.42.xx.xx port 22: Connection refused
>
>used minicom to hook up to serial port and was able to see (didn't
>require login??):
>
># ls -al
>total 166
>drwxr-xr-x  11 root  wheel512 Apr  7 18:34 .
>drwxr-xr-x  11 root  wheel512 Apr  7 18:34 ..
>-rw-r--r--   1 root  wheel   1770 Mar 20 21:53 .profile
>-rw-r--r--   1 root  wheel194 Apr  7 18:34 auto_upgrade.conf
>lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 autoinstall -> install.sub
>drwxr-xr-x   2 root  wheel512 Mar 20 21:53 bin
>drwxr-xr-x   2 root  wheel   2560 Apr  7 18:34 dev
>drwxr-xr-x   5 root  wheel512 Apr  7 18:34 etc
>lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 install -> install.sub
>-rw-r--r--   1 root  wheel   2903 Mar 20 21:53 install.md
>-rwxr-xr-x   1 root  wheel  64483 Mar 20 21:53 install.sub
>drwxr-xr-x  15 root  wheel512 Apr  7 18:36 mnt
>drwxr-xr-x   2 root  wheel512 Mar 20 21:53 mnt2
>drwxr-xr-x   2 root  wheel   1024 Mar 20 21:53 sbin
>drwxrwxrwt   4 root  wheel512 Apr  7 18:36 tmp
>lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 upgrade -> install.sub
>drwxr-xr-x   6 root  wheel512 Mar 20 21:53 usr
>drwxr-xr-x   6 root  wheel512 Mar 20 21:53 var
>
>reboot from serial console with minicom
>
>Comes back up (can ssh now) but with only one cpu (normally running mp - 
>cpu[0-3] )
>
>reboot
>
>Watching on serial port
>
>reorder_kernel: failed -- see /usr/share/relink/kernel/GENERIC/relink.log
>
>as per log: run 'sha256 -h /var/db/kernel.SHA256 /bsd'
>
>reboot, no error, but still in single cpu mode.
>
>reboot bsd.mp
>
>hang, ssh connection refused
>
>Power cycle to get fresh reboot but still hangs
>
>still on serial port: boot usb with miniroot75.img
>
>reboot - needed to reset - 'stty com0 115200; set tty com0', otherwise hangs
>
>Choose upgrade
>
>Success!
>
>This happened on a third apu2 board; thus, two unsucessful and one successful 
>sysupgrades on apu2 boards
>
>Any ideas/pointers?  I'd like this not to reoccur.
>
>
>
>

sysupgrade on pcEngines apu2 boards hangs

[Glen Gunsalus]

I have been running OpenBSD on three apu2 boards (as firewalls) for several 
years and doing remote (ssh to boards) with no problems.

Doing the 7.4 to 7.5 upgrades I had an initial success then two failures. Not 
confidence building since these firewalls are at remote sites.

Not sure where the problem lies, here is what I've experienced:

pcEngines apu2 boards, running sysupgrade - have been running OpenBSD
since about 6.2 and doing remote (ssh to board) for several releases
w/o any problems

Did successful sysupgrade on one board the evening before 7.5 release
notification using: url  https://cdn.openbsd.org/pub/OpenBSD/ (other
mirrors not yet showing items in 7.5 directory)

Went fine, as per norm.

Next day did on a second board using my default url (as per all previous
remote upgrades)  https://mirrors.sonic.net/pub/OpenBSD/

after syspugrade pings ok but cannot ssh:
ssh: connect to host 10.42.xx.xx port 22: Connection refused

used minicom to hook up to serial port and was able to see (didn't
require login??):

# ls -al
total 166
drwxr-xr-x  11 root  wheel512 Apr  7 18:34 .
drwxr-xr-x  11 root  wheel512 Apr  7 18:34 ..
-rw-r--r--   1 root  wheel   1770 Mar 20 21:53 .profile
-rw-r--r--   1 root  wheel194 Apr  7 18:34 auto_upgrade.conf
lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 autoinstall -> install.sub
drwxr-xr-x   2 root  wheel512 Mar 20 21:53 bin
drwxr-xr-x   2 root  wheel   2560 Apr  7 18:34 dev
drwxr-xr-x   5 root  wheel512 Apr  7 18:34 etc
lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 install -> install.sub
-rw-r--r--   1 root  wheel   2903 Mar 20 21:53 install.md
-rwxr-xr-x   1 root  wheel  64483 Mar 20 21:53 install.sub
drwxr-xr-x  15 root  wheel512 Apr  7 18:36 mnt
drwxr-xr-x   2 root  wheel512 Mar 20 21:53 mnt2
drwxr-xr-x   2 root  wheel   1024 Mar 20 21:53 sbin
drwxrwxrwt   4 root  wheel512 Apr  7 18:36 tmp
lrwxr-xr-x   1 root  wheel 11 Mar 20 21:53 upgrade -> install.sub
drwxr-xr-x   6 root  wheel512 Mar 20 21:53 usr
drwxr-xr-x   6 root  wheel512 Mar 20 21:53 var

reboot from serial console with minicom

Comes back up (can ssh now) but with only one cpu (normally running mp - 
cpu[0-3] )

reboot

Watching on serial port

reorder_kernel: failed -- see /usr/share/relink/kernel/GENERIC/relink.log

as per log: run 'sha256 -h /var/db/kernel.SHA256 /bsd'

reboot, no error, but still in single cpu mode.

reboot bsd.mp

hang, ssh connection refused

Power cycle to get fresh reboot but still hangs

still on serial port: boot usb with miniroot75.img

reboot - needed to reset - 'stty com0 115200; set tty com0', otherwise hangs

Choose upgrade

Success!

This happened on a third apu2 board; thus, two unsucessful and one successful 
sysupgrades on apu2 boards

Any ideas/pointers?  I'd like this not to reoccur.

Re: 7.5 NO hard drive?

[Wolfgang Pfeiffer]

On Sun, Apr 07, 2024 at 05:17:25PM +0200, Wolfgang Pfeiffer wrote:

On Sun, Apr 07, 2024 at 12:03:21AM -0700, lati...@vcn.bc.ca wrote:



So OpenBSD has been correctly installed, thanks so much to maintain it nice!

The problem was with the BIOS, it needs IHCH or something like that to be
recognized!
But it is working now as a xfce Desktop!


Seems to be (not only) a DELL thing: Some time ago I tried an Openbsd
installer on an Alienware computer, ~10 years old, which was sold by
DELL: In UEFI, IIRC, I had to change sata mode from "raid" to "ahci"
to let openbsd detect hard disks on that computer.


Correction: IIRC, at least the USB-thumb with the installer on it was
detected with RAID mode enabled - but that was about it ... ;)
So: no hard disk, no SSD etc. that were detected ...

--
Wolfgang

Re: 7.5 NO hard drive?

[Peter N. M. Hansteen]

On Sun, Apr 07, 2024 at 05:17:25PM +0200, Wolfgang Pfeiffer wrote:
> > 
> > The problem was with the BIOS, it needs IHCH or something like that to be
> > recognized!
> > But it is working now as a xfce Desktop!
> 
> Seems to be (not only) a DELL thing: Some time ago I tried an Openbsd
> installer on an Alienware computer, ~10 years old, which was sold by
> DELL: In UEFI, IIRC, I had to change sata mode from "raid" to "ahci"
> to let openbsd detect hard disks on that computer.
> 
> Seems to an older issue:
> https://daemonforums.org/showthread.php?t=10228
> https://www.mail-archive.com/misc@openbsd.org/msg153583.html

Adding to that list, my experience with an ASUS laptop where it would
be physically impossible to fit more than one storage device, but
the storage controller anyway was set to "Raid" mode by default. Fortunately
it was possible to choose the other options and have the device turn up
as a regular NMVe device: 

https://nxdomain.no/~peter/blog_wild_wild_world_of_windows.html (or with
incrementally nicer formatting at the cost of G's trackers, 
https://bsdly.blogspot.com/2021/07/the-impending-doom-of-your-operating.html)

-- 
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
https://bsdly.blogspot.com/ https://www.bsdly.net/ https://www.nuug.no/
"Remember to set the evil bit on all malicious network traffic"
delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Re: 7.5 NO hard drive?

[Wolfgang Pfeiffer]

On Sun, Apr 07, 2024 at 12:03:21AM -0700, lati...@vcn.bc.ca wrote:

Hello

i have 1 DELL Latitude E4300 that had OBSD 7.3 working correctly, but i
decided to do a clean installation of 7.5 deleting everything on it with a
live cd linux; then tested 7.5 and it says NO disk.

After that i tested Linux, NetBSD, FreeBSD all them where installed
without a problem; But, OBSD 7.3  7.4 7.5 said NO disk!

[ ... ]






So OpenBSD has been correctly installed, thanks so much to maintain it nice!

The problem was with the BIOS, it needs IHCH or something like that to be
recognized!
But it is working now as a xfce Desktop!


Seems to be (not only) a DELL thing: Some time ago I tried an Openbsd
installer on an Alienware computer, ~10 years old, which was sold by
DELL: In UEFI, IIRC, I had to change sata mode from "raid" to "ahci"
to let openbsd detect hard disks on that computer.

Seems to an older issue:
https://daemonforums.org/showthread.php?t=10228
https://www.mail-archive.com/misc@openbsd.org/msg153583.html

IIRC, Debian, Fedora, Windows installed without problems on that machine:
at least in my case it might be a problem with a BIOS version too old for
an openbsd install - not being sure ..
--
Wolfgang

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Страхиња Радић]

Дана 24/04/07 12:46PM, Страхиња Радић написа:
> Ok. The alternative would be to find a way to make 7.5 efifb work on my 
> laptop. 
> The version of efifb from 7.4 works (that is how I installed 7.4 in the first 
> place), unlike 7.5 efifb.

I'd just like to add that it efifb might not even be the reason for system 
hang. I noticed these lines in the output from 7.5 bsd.upgrade I got when I 
entered `verbose` at the UKC prompt and exited UKC:

uhub0: device problem, disabling port 2
uhub0: device problem, disabling port 3
uhub0: device problem, disabling port 5
uhub0: device problem, disabling port 6

on my working 7.4 system, I have

uhub0 at usb0 configuration 1 interface 0 "AMD xHCI root hub" rev \
3.00/1.00 addr 1

and later 

urtwn0 at uhub0 port 2 configuration 1 interface 0 "Realtek 802.11n \
NIC" rev 2.00/0.00 addr 2
urtwn0: MAC/BB RTL8188EU, RF 6052 1T1R, address 
uhidev0 at uhub0 port 3 configuration 1 interface 0 "SiGmaMicro USB \
Optical Mouse" rev 1.10/1.10 addr 3
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse2 at ums0 mux 0
uvideo0 at uhub0 port 5 configuration 1 interface 0 "Sonix Technology \ 
Co., Ltd. Integrated Camera" rev 2.00/0.28 addr 4
video0 at uvideo0
ugen0 at uhub0 port 6 "Atheros Communications product 0xe300" rev \
2.01/0.01 addr 5

so the devices which have a "problem" are all devices connected to USB ports;
or rather, the USB hub itself?

Are there any regressions in the AMD xHCI hub code?

Re: ipv6 assistance

[Florian Obser]

On 2024-04-07 10:27 UTC, Stuart Henderson  wrote:
> On 2024-04-06, Florian Obser  wrote:
>> Someone with pull at UPC^W ziggo^W vodafone^W liberty global could 
>> potentially get that situation improved.
>
> Often on an OpenBSD box using one of these connections, you want
> one or more /64s rather than a host address, I don't think there's
> an alternative to DHCPv6-PD for that unless the ISP will do static
> addressing.
>

I was alluding that I would implement DHCPv6-PD in (probably) rad(8) if
ziggo stopped to be stupid. Or their cpe stopped to be stupid. I.e. the
way things get done in OpenBSD: Someone has a use case and the means to
do something about it. I have the means but no use case.

I'm currently using it with slaac so I have a single collision
domain, but all my devices get native IPv6 from a single /64.

They are not speaking DHCPv6 at all. I can login to the CPE and switch
from slaac to dhcp. Which is pretty much nonsensical. The effect is that
slaac stops working and the dhcp server tells me: no lease or something
to that effect.

There are some forum posts, all in Dutch, of people who try to argue the
finer points of IPv6 with ziggo support - which unsurprisingly isn't
going anywhere.

> Though it would be nice if they'd also allow getting a single address
> via slaac. With NAT that's enough for use on a router too ;)
>
>> On 6 April 2024 19:04:52 CEST, Peter Hessler  wrote:
>>>OpenBSD natively supports IPv6 addressing via static configuration and
>>>SLAAC.  We do not have a DHCPv6 client in base, so currently you have to
>>>use a package for that.
>
> A simple daemon that just runs as a PD client would be pretty welcome.
> At least dhcpcd is nicely privilege-separated (and uses pledge on OpenBSD)
> though it does have a lot more features than are needed for this common
> use case.
>
>
> -- 
> Please keep replies on the mailing list.
>

-- 
In my defence, I have been left unsupervised.

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Страхиња Радић]

Дана 24/04/07 10:32AM, Stuart Henderson написа:
> None of the ramdisk kernels include these large drivers (no difference
> in that respect between 7.4 and 7.5, or any other release).

Ok. The alternative would be to find a way to make 7.5 efifb work on my laptop. 
The version of efifb from 7.4 works (that is how I installed 7.4 in the first 
place), unlike 7.5 efifb.

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Stuart Henderson]

On 2024-04-07, Страхиња Радић  wrote:
> (manually retyped to avoid large attachments):

thank you!

> 2. The result of entering `boot -c` at the `boot>` prompt with /bsd.upgrade 
> having the execute bit set (so 7.5 /bsd.upgrade would boot by default):
>
>   OpenBSD 7.5 (RAMDISK_CD) #76: Wed Mar 20 15:53:54 MDT 2024
>   dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
>   real mem = 6277246976 (5986MB)
>   avail mem = 6082772992 (5800MB)
>   User Kernel Config
>   UKC> find radeondrm
>   UKC> find efifb
>41 efifb0 at mainbus0 apid -1 flags 0x0
>   UKC> find amdgpu
>   UKC>
>
> So it seems that 7.5 /bsd.upgrade doesn't detect nor configure the radeondrm 
> and amdgpu devices?

None of the ramdisk kernels include these large drivers (no difference
in that respect between 7.4 and 7.5, or any other release).


-- 
Please keep replies on the mailing list.

Re: ipv6 assistance

[Stuart Henderson]

On 2024-04-06, Florian Obser  wrote:
> Someone with pull at UPC^W ziggo^W vodafone^W liberty global could 
> potentially get that situation improved.

Often on an OpenBSD box using one of these connections, you want
one or more /64s rather than a host address, I don't think there's
an alternative to DHCPv6-PD for that unless the ISP will do static
addressing.

Though it would be nice if they'd also allow getting a single address
via slaac. With NAT that's enough for use on a router too ;)

> On 6 April 2024 19:04:52 CEST, Peter Hessler  wrote:
>>OpenBSD natively supports IPv6 addressing via static configuration and
>>SLAAC.  We do not have a DHCPv6 client in base, so currently you have to
>>use a package for that.

A simple daemon that just runs as a PD client would be pretty welcome.
At least dhcpcd is nicely privilege-separated (and uses pledge on OpenBSD)
though it does have a lot more features than are needed for this common
use case.


-- 
Please keep replies on the mailing list.

Re: 7.5 /var/log/messages - vfprintf %s NULL in "%.*s"

[Страхиња Радић]

Дана 24/04/07 10:11AM, Stuart Henderson написа:
> Yes. It's very common though, especially when constructing strings for
> debug logging. I see this a lot with software in the GNOME ecosystem
> using sprintf for this.

Programmers who pass NULL to %s format specifier are writing incorrect code, 
whatever project they are contributing to. Mainstream projects like GNOME are 
a mess, and only getting worse with every iteration anyway.


> That's interesting about the compiler optimisation for printf->puts,
> though I think it won't be used in many cases where the pointer nay be
> null so many uses of this won't trigger crashes for that reason.

Keep in mind the quoted answer states this about gcc. I'm not sure about 
Clang/LLVM. In any case, passing NULL to %s specifier in *printf is incorrect 
code.

Re: Migrate to different FS layout of OpenBSD

[Kirill A . Korinsky]

On Sun, 07 Apr 2024 12:02:05 +0200,
Stuart Henderson wrote:
> 
> softraid doesn't allow creating a 'degraded mirror' i.e. a single drive
> that you can later add another drive to make a RAID1. You would need at
> least one spare drive to do what you want.
> 

Thanks, that is a kind of inside which I've been looking for.

-- 
wbr, Kirill

Re: 7.5 /var/log/messages - vfprintf %s NULL in "%.*s"

[Stuart Henderson]

On 2024-04-06, Страхиња Радић  wrote:
> Дана 24/04/06 06:04PM, Stuart Henderson написа:
>> The fact that these all started hitting this with the same printf string
>> (including tmux, which is in base) makes me wonder if it's coming from a
>> library, the most likely being libcurses which was updated between 7.4
>> and 7.5 (which all of those use).
>> 
>> Try to ascertain what's going on when that message is logged. ktrace
>> might give some clues.
>
> Of course, the package containing the code passing NULL to *printf should be 
> identified first, and the bug report should be sent to that package. It is 
> entirely possible that it is libcurses or another library. In the case of 
> dunst, it was dunst.

libcurses and tmux are not packages, they are in the base OS.

> Passing NULL to *printf is Undefined Behavior in C, and there is a 
> StackOverflow answer detailing the reasons at [1].
>
> [1]: https://stackoverflow.com/a/11589500

Yes. It's very common though, especially when constructing strings for
debug logging. I see this a lot with software in the GNOME ecosystem
using sprintf for this.

That's interesting about the compiler optimisation for printf->puts,
though I think it won't be used in many cases where the pointer nay be
null so many uses of this won't trigger crashes for that reason.


-- 
Please keep replies on the mailing list.

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Страхиња Радић]

Дана 24/04/07 11:57AM, Sebastien Marie написа:
> Here, you are comparing GENERIC.MP kernel with RAMDISK_CD kernel.
> 
> A RAMDISK_CD kernel is a reduced kernel with only what is necessary to
> install openbsd. radeondrm and amdgpu are NOT part of it, and it is
> expected.

Is there any way for me to add the necessary drivers to the bsd.upgrade? The 
7.5 bsd.upgrade kernel doesn't boot on my laptop using only efifb.

Re: Migrate to different FS layout of OpenBSD

[Stuart Henderson]

On 2024-04-06, Kirill A  Korinsky  wrote:
> On Sat, 06 Apr 2024 23:14:39 +0200,
> Peter Hessler wrote:
>> 
>> RAID0 is called that because zero is what you'll recover if you lose a
>> disk.  This is amazingly dangerous, and you're going to have a bad time.
>> 
>> Do a backup, then restore from backup.
>> 
>
> I was totally misslead. I mean that I have RAID1 which is know as mirror.
>
> To be clear: here a two identical servers where I'd like to change FS
> layout, and before I go to reinstall everything, I can try this approach.

softraid doesn't allow creating a 'degraded mirror' i.e. a single drive
that you can later add another drive to make a RAID1. You would need at
least one spare drive to do what you want.



-- 
Please keep replies on the mailing list.

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Sebastien Marie]

Страхиња Радић  writes:

> As a follow-up to my previous post, after rereading [1], I tried to compare 
> the 
> status of radeondrm, efifb and amdgpu in 7.4 /bsd boot and 7.5 /bsd.upgrade 
> boot. I got this (manually retyped to avoid large attachments):
>
> 1. The result of entering `boot -c` at the `boot>` prompt with /bsd.upgrade 
> not 
> having the execute bit (so 7.4 /bsd would boot by default):
>
>   OpenBSD 7.4 (GENERIC.MP) #1397: Tue Oct 10 09:02:37 MDT 2023
>   dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>   real mem = 6277246976 (5986MB)
>   avail mem = 6067273728 (5786MB)
>   User Kernel Config
>   UKC> find radeondrm
>   242 radeondrm* at pci* dev -1 function -1 flags 0x0
>   UKC> find efifb
>59 efifb0 at mainbus0 apid -1 flags 0x0
>   UKC> find amdgpu
>   243 amdgpu* at pci* dev -1 function -1 flags 0x0
>   UKC>
>
> 2. The result of entering `boot -c` at the `boot>` prompt with /bsd.upgrade 
> having the execute bit set (so 7.5 /bsd.upgrade would boot by default):
>
>   OpenBSD 7.5 (RAMDISK_CD) #76: Wed Mar 20 15:53:54 MDT 2024
>   dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
>   real mem = 6277246976 (5986MB)
>   avail mem = 6082772992 (5800MB)
>   User Kernel Config
>   UKC> find radeondrm
>   UKC> find efifb
>41 efifb0 at mainbus0 apid -1 flags 0x0
>   UKC> find amdgpu
>   UKC>
>
> So it seems that 7.5 /bsd.upgrade doesn't detect nor configure the radeondrm 
> and amdgpu devices?

Here, you are comparing GENERIC.MP kernel with RAMDISK_CD kernel.

A RAMDISK_CD kernel is a reduced kernel with only what is necessary to
install openbsd. radeondrm and amdgpu are NOT part of it, and it is
expected.

-- 
Sebastien Marie

Re: OpenBSD 7.5 bsd.upgrade hangs after sysupgrade

[Страхиња Радић]

As a follow-up to my previous post, after rereading [1], I tried to compare the 
status of radeondrm, efifb and amdgpu in 7.4 /bsd boot and 7.5 /bsd.upgrade 
boot. I got this (manually retyped to avoid large attachments):

1. The result of entering `boot -c` at the `boot>` prompt with /bsd.upgrade not 
having the execute bit (so 7.4 /bsd would boot by default):

OpenBSD 7.4 (GENERIC.MP) #1397: Tue Oct 10 09:02:37 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 6277246976 (5986MB)
avail mem = 6067273728 (5786MB)
User Kernel Config
UKC> find radeondrm
242 radeondrm* at pci* dev -1 function -1 flags 0x0
UKC> find efifb
 59 efifb0 at mainbus0 apid -1 flags 0x0
UKC> find amdgpu
243 amdgpu* at pci* dev -1 function -1 flags 0x0
UKC>

2. The result of entering `boot -c` at the `boot>` prompt with /bsd.upgrade 
having the execute bit set (so 7.5 /bsd.upgrade would boot by default):

OpenBSD 7.5 (RAMDISK_CD) #76: Wed Mar 20 15:53:54 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 6277246976 (5986MB)
avail mem = 6082772992 (5800MB)
User Kernel Config
UKC> find radeondrm
UKC> find efifb
 41 efifb0 at mainbus0 apid -1 flags 0x0
UKC> find amdgpu
UKC>

So it seems that 7.5 /bsd.upgrade doesn't detect nor configure the radeondrm 
and amdgpu devices?


[1]: https://unix.stackexchange.com/q/761130/367454

Re: 7.5 NO hard drive?

[latinfo]

> Hello
>
> i have 1 DELL Latitude E4300 that had OBSD 7.3 working correctly, but i
> decided to do a clean installation of 7.5 deleting everything on it with a
> live cd linux; then tested 7.5 and it says NO disk.
>
> After that i tested Linux, NetBSD, FreeBSD all them where installed
> without a problem; But, OBSD 7.3  7.4 7.5 said NO disk!
>
> It is something related to OBSD?
> What could happened?
> How to install OBSD 7.5
>
> PS:
> Thanks for the new version 7.5 i run 2 laptops and 1 server with it!
>
> Thanks
>

So OpenBSD has been correctly installed, thanks so much to maintain it nice!

The problem was with the BIOS, it needs IHCH or something like that to be
recognized!
But it is working now as a xfce Desktop!