I'd get something like a 1U ATOM server ($120 eBay) with small SSD ($18). Runup your favorite FOSS OS, and conserver. For more than the single real serialport, you can most likely fit a USB hub inside the case still, and hang a number of USB serial dongles off.
We use Raspberry Pi 2s with single- and 8-port USB serial dongles. Works like a charm, especially with tmux installed.
--lyndon
