On 4/2/26 10:16, Justin Sherrill wrote:
On Thu, Apr 2, 2026 at 9:15 AM George Rosamond
<[email protected] <mailto:[email protected]>> wrote:
I want to initiate a thread on the "BSDs and AI today."
I'm looking to do a presentation on this in the summer for NYC*BUG.
Two quantifiable measures, though they will change by the time you are
doing a summer presentation:
- What models and software run on BSDs? There's all sorts of tooling
for accessing LLMs, but how much have made it to BSD?
its not the model per-se but drivers and library support between freebsd
and most likely the expensive Nvidia GPU you can't afford to perform the
math against the model. i would not recommend trying to do this with
any bsd at this point, while it may be possible you'll be swimming
upstream best case scenario.
regarding the tooling and ecosystem in general, there is plenty of
opportunity there. most people use python to interact with models - i
use freebsd and hugging face as well as other popular libraries to
interact with LLM api's. they work mostly ok but require more effort to
maintain then linux, this is mostly due to developers integrating rust
into python so you'll need to do lots of compiling.
then there are things like vector databases that may be well suited to
freebsd (these tend to be opensearch based things, but can also be
hosted in postgresql for example too).
at the end of the day its just computers once you remove the hype...and
lots of immature code lol.
-pete
--
Pete Wright
[email protected]
