On 25/05/25 17:47, Christoph Cullmann wrote:
[...]
given I played already a bit longer with local running AI stuff via
ollama, I tried to use the REST API
https://github.com/ollama/ollama/blob/main/docs/api.md
for a small Kate plugin skeleton:
https://invent.kde.org/utilities/kate/-/merge_requests/1785
that ATM just allows to send the current line as prompt and inserts
the result below.
The local port and model are hardcoded, as that is just a draft.
There is already some code for handling this communication with ollama
in Alpaka's repository which seems to, at least partially, stem from KDAB:
https://invent.kde.org/utilities/alpaka/-/tree/master/src/core?ref_type=heads
If interest in getting such a thing up and running is there, help is
welcome.
Code completion is feasible with such a local setup, too. Just a
matter of implementing the right prompting and filling
a completion model with the results.
With ollama no data will leave your machine.
Naturally a question is which models should be promoted as default for
licensing/...
That plugin would not just work in Kate but in other KTextEditor
plugin compatible applications like KDevelop or RKWard.
In LabPlot and in Cantor we start using more of KTextEditor now and
these applications will also benefit from this development but I think
it has a bigger scope and potential and the communication with the local
ollama enriched by domain/application specific embeddings can go way
beyond KTextEditor. Think about a built-in documentation in the
application that is helping the user to get faster to the results. As an
example, feed in something like "how to import data into LabPlot and to
create a histogram for it" into Perplexity, etc. - these results, which
are already very good actually for this specific promt, can be
significantly improved with local embeddings based on the pdf of the
documentation, examples scripts and projects for your application, etc.
Or think about "generative AI" workflows where this kind of tooling can
help to reduce the amount of trivial work like in
https://github.com/jupyterlab/jupyter-ai or in similar "AI assistants"
for R and other languages and frameworks.
--
Alexander
