Public bug reported:

We are running x11vnc 0.9.16-8 on a FIPS enabled Ubuntu 22.04 with
libvncserver1 0.9.13+dfsg-3build2 and libssl3 3.0.2-0ubuntu1.12+Fips1.

With the fips=1 kernel parameter enabling FIPS, it seems that the
password hashing is broken and only a clear text password is written:

$ cat /proc/sys/crypto/fips_enabled 
1
$ x11vnc -storepasswd Abc /tmp/.testpw && cat /tmp/.testpw
stored passwd in file: /tmp/.testpw
Abc

Any connection attempt fails with a 'password check failed!' error.

Running x11vnc with

sudo /usr/bin/x11vnc -auth guess -forever -localhost -loop -noxdamage
-repeat -rfbauth /root/.vncpasswd -rfbport 5900 -shared

logs the following:

Got connection from client 127.0.0.1
  0 other clients
Normal socket connection
check_access: client 127.0.0.1 matches host 127.0.0.1
incr accepted_client=1 for 127.0.0.1:54968  sock=10
Client Protocol Version 3.8
Protocol version sent 3.8, using 3.8
rfbProcessClientSecurityType: executing handler for type 2
Couldn't read password file: /root/.vncpasswd
rfbAuthProcessClientMessage: password check failed
rfbClientSendString("password check failed!")
client_count: 0
Client 127.0.0.1 gone

By turning off FIPS with fips=0 in the kernel, it works as expected:

$ cat /proc/sys/crypto/fips_enabled 
0
$ x11vnc -storepasswd Abc /tmp/.testpw && cat /tmp/.testpw
stored passwd in file: /tmp/.testpw
�97l܊

** Affects: x11vnc (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2058354

Title:
  Enabling FIPS breaks password hashing

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/x11vnc/+bug/2058354/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to