Dear everyone,
please find a (fixed) security issue for albatross-console, discovered
and reported by Edwin Török.
Have a nice weekend,
Hannes
Background
==========
Albatross-console reads the console output from multiple unikernel
tenders (solo5-hvt). This console output can be retrieved using
albatross-client.
The console protocol is fairly simple: the unikernel invokes a PUTS
hypercall, which sends arbitrary bytes of given length to the unikernel
tender (host, typically solo5-hvt), which writes them to a file descriptor.
albatross_console reads this output, and assumes it will be newline
delimited, and keeps at most 1024 lines.
Problem description
===================
This helps guard against unlimited memory usage from runaway/long
running unikernels, however it doesn't guard against malicious (or
buggy) unikernels.
The problem is this line in albatross_console:
```
let rec loop () =
Lwt_io.read_line channel
```
Unfortunately Lwt_io.read_line doesn't take a parameter to limit the
size of the line that is read, so it is very easy for a unikernel to
exhaust the memory of albatross_console: all it needs to do is to write
a lot of bytes without ever writing a newline.
Tested with the Debian packages, but confirmed the above code to be
present in latest master too:
```
$ /usr/libexec/albatross/albatross-console --version
version v2.3.0-9-g5b14787 protocol version 5
```
Impact
======
A typical attack will look like this in albatross_console's logs:
```
Jun 07 16:41:18 ubuntu22 albatross-console[13721]: albatross-console:
[WARNING] disconnected
Jun 07 16:41:42 ubuntu22 albatross-console[13721]: albatross-console:
[ERROR] exception Unix.Unix_error(Unix.EBADF, "check_descriptor", "")
while writing
Jun 07 16:42:09 ubuntu22 albatross-console[13721]: albatross-console:
[WARNING] disconnected
Jun 07 16:44:07 ubuntu22 albatross-console[13721]: albatross-console:
[ERROR] :mine error while reading Out of memory
Jun 07 16:55:43 ubuntu22 albatross-console[13721]: albatross-console:
[ERROR] exception Unix.Unix_error(Unix.EBADF, "check_descriptor", "")
while writing
Jun 07 16:57:20 ubuntu22 albatross-console[13721]: albatross-console:
[ERROR] exception Unix.Unix_error(Unix.EPIPE, "send", "") while writing
Jun 07 16:57:20 ubuntu22 albatross-console[13721]: albatross-console:
[ERROR] exception Unix.Unix_error(Unix.EPIPE, "send", "") while writing
```
While the attack is happening albatross_console will also be very slow
to react to the console of other unikernels, and will use increasing
amounts of memory until an out of memory exception is raised.
Albatross_console stays running in my tests (obviously the bad unikernel
will no longer have a functioning console, but that is to be expected).
However using so much memory can have an effect on other services
running on the host.
Workaround
==========
Set MemoryMax=1G in the service file of albatross_console to limit the
amount of memory a runaway console can use, so at least it fails sooner.
However in practice this just keeps albatross_console using 100% CPU
without raising an Out of Memory exception, although it stays within 1G
of memory.
There is no known workaround for the DoS attack, in theory Lwt should be
switching promises when another one becomes runnable, but it doesn't
provide fairness mechanisms.
Solution
========
Use albatross in version 2.5.0 or above. Another solution is to apply
the patch
dhttps://github.com/robur-coop/albatross/commit/d01805796ec710691a701c01ed3f0e4cd284a161
manually.
Binary builds of version 2.5.0 are available from https://builds.robur.coop
Timeline
========
2025-06-07: issue discovered by Edwin Török as part of the HACKSAT25
challenge
2025-06-07: initial email sent to hack...@parsimoni.co
2025-06-11: initial email sent to albatross maintainer
2025-08-15: albatross 2.5.0 released with the patch
