Hi all, With PicoLisp 26.5.19 there are two new files in the distribution:
1. `lib/ai.l` – a small OpenAI client 2. `lib/vip/ai.rc.l` – a Vip plugin implementing a chat interface Below is a short overview and some comments on the more interesting parts of the code. ## 1. Library `lib/ai.l` This is a minimal JSON/OpenAI client. At the moment it only supports the `chat/completions` endpoint, but is structured so more endpoints can be added easily. # 13may26 Software Lab. Alexander Burger (symbols 'ai 'pico) # https://developers.openai.com/api/reference/overview # https://platform.deutschlandgpt.de/docs (local) (*ApiUrl *ApiKey *Max *Result curl result res api models completions) Everything runs in the `ai` namespace, with a few globals: - `*ApiUrl` – base host, e.g. `api.openai.com/v1/` - `*ApiKey` – bearer token - `*Max` – optional `max_tokens` limit (used by the Vip plugin) - `*Result` – last parsed JSON response as a tree of PicoLisp pairs ### The `curl` wrapper (de curl (Res . @) (pass list "curl" "-s" (pack "https://"; *ApiUrl Res) "-H" (pack "Authorization: Bearer " *ApiKey) ) ) — `curl` composes an argument list for an external `curl` call. — `Res` is appended to the base URL (`*ApiUrl`), so the same helper can be used for different endpoints (`"models"`, `"chat/completions"`, etc.). — Extra arguments (like `"--json" "@-"`) are passed via `. @`. ### Reading and post-processing the JSON (de result @ (when (setq *Result (readJson)) (pass res) ) ) (de res @ (let (Rest *Result Lst (rest)) (recur (Lst Rest) (loop (NIL Lst Rest) (setq Rest (get Rest (++ Lst))) (T (and (pair Rest) (pair (caar @))) (mapcar '((Rest) (recurse Lst Rest)) Rest ) ) ) ) ) ) Here are two interesting aspects: 1. `result`: — Reads the entire JSON reply from stdin via `readJson`. — Stores it globally in `*Result`. — Then calls `res` with any symbol path given (e.g. `'choices 'message 'content`). 2. `res`: — `*Result` is a tree of JSON data (lists and conses). — `res` interprets its arguments as a *path* into this tree: — `(res 'choices 'message 'content)` walks recursively and collects all matching fields. — It uses `(get Rest (++ Lst))` and a `recur` to traverse arbitrarily nested objects/arrays, and returns a flattened list of all matches. — This gives a very compact “query language” over the JSON tree. ### API configuration # (api 'file) (de api (File) (in File (setq *ApiUrl (line T) *ApiKey (line T)) (line T) ) ) — Reads the first two lines of `File`: the API URL and key. — A third line is read and returned. It may hold a default model, I use it in some scripts. — The Vip plugin expects this to live in `~/.ai`. ### Models listing # (models) (de models () (in (curl "models") (result 'data 'id) ) ) — Calls `GET /models`. — Extracts all `id` values below `data` using the `result/res` mechanism. — Return value is a list of strings (model IDs). ### Chat completions # (completions 'model 'role 'text 'temp ['role 'text 'temp ..] (de completions (Model . @) (pipe (out (curl "chat/completions" "--json" "@-") (printJson (list (cons 'model Model) (make (link 'messages T) (while (args) (link (make (link (cons 'role (next)) (cons 'content (next)) ) (and (next) (link (cons 'temperature @))) (and *Max (link (cons 'max_tokens @))) ) ) ) ) ) ) ) (result 'choices 'message 'content) ) ) Key points: — The function interface is compact: `(completions "gpt-4.1" "user" "Hello" 2 "system" "You are ..." NIL ...)` — Arguments are consumed in triples: role, content, temperature. — `printJson` builds a JSON object like: { "model": "gpt-4.1", "messages": [ { "role": "user", "content": "Hello", "temperature": 2, "max_tokens": 200 }, ... ] } (`max_tokens` is injected from the global `*Max` when present.) — `pipe`: — Sends the JSON to `curl` on stdin (`"--json" "@-"`). — Then reads the JSON response from `curl`’s stdout and returns all `choices.message.content` strings. So you get a pure PicoLisp function to call the OpenAI chat completions endpoint, with usable defaults and a simple path mechanism into the reply. ## 2. Vip plugin `lib/vip/ai.rc.l` The second file integrates the above into the Vip editor as a mini chat client bound to a buffer. # 16may26 Software Lab. Alexander Burger (symbols '(pico) (load "@lib/json.l" "@lib/ai.l") ) (symbols '(ai vip pico)) — Loads the JSON and AI libraries. — Switches to a combined namespace so `ai`, `vip` and core `pico` symbols are all visible. The plugin defines three commands: `:models`, `:ai`, and `:ai?`. ### `:models` – list model IDs (cmd "models" (L Lst Cnt) (symbols '(ai vip pico) (api "~/.ai") (prCmd (sort (extract '((S) (chop (pre? L S))) (models) ) ) ) ) ) — Calls `(api "~/.ai")` to set URL and key. — `(models)` returns all available models. — `L` is the optional prefix typed after `:models`. — `pre?` filters only those model IDs starting with that prefix. — They are sorted and printed with `prCmd` in the Vip command area. ### Conversation file parsing in `:ai` (cmd "ai" (L Lst Cnt) (symbols '(ai vip pico) (api "~/.ai") (let (Lst (split (make (for L (: buffer text) (when (or (= '("+" "+" "+") L) (and (head '("+" "+" "+" " ") L) (format (cadr (split L " "))) ) (= '("=" "=" "=") L) ) (link 0) ) (link L) ) 0 ) L (split (caar Lst) " ") ) (setq *Model (pack (car L)) *Max (format (cadr L)) ) ... Interesting details: — `(: buffer text)` holds all lines of the current buffer. — The code inserts the marker `0` into the stream whenever it finds: — a line `+++` (start of user prompt), — a line `+++ <temp>` (prompt with explicit temperature, e.g. `+++ 1`), — or a line `===` (start of assistant response). — Then `(split ... 0)` splits the whole buffer into chunks at these zero markers, giving a list of sections, each corresponding to one “block” (system messages, prompts, responses). — `caar Lst` is the first block, which contains the first line: `model [max_tokens]`. — `L` is that line split by spaces. — `*Model` is set from the model name. — `*Max` is optionally set from the second item (token limit). So the *file format* is: <model> [max_tokens] [optional system text...] +++ <prompt 1> === <response 1> +++ <prompt 2> Temperature is given on the `+++` line, optionally, like: +++ 2 This prompt uses temperature 2 #### Building messages for the API call Continuing inside `:ai`: (let? R (or (mapcan '((S) (split (chop S) "\n")) (apply completions (mapcan '((L) (let? S (glue "\n" (cdr L)) (cond ((head '("+" "+" "+") (setq L (car L))) (list "user" S (format (cadr (split L " ")))) ) ((head '("=" "=" "=") L) (list "assistant" S NIL) ) (T (list "system" S NIL)) ) ) ) Lst ) *Model ) ) (mapcar '((X) (chop (cdr X))) (fish '((X) (and (pair X) (memq (car X) '(error message)) (str? (cdr X)) ) ) *Result ) ) ) — Each section in `Lst` is a list where: — `car L` is the marker line (`+++`, `+++ 1`, or `===`). — `cdr L` is the actual multi-line text in that section. — For each such block: — `S` is glued together with `"\n"` to preserve multi-line prompts/responses. — The role is chosen from the marker: — `+++`* → `"user"` and temperature parsed from that line, if present. — `===` → `"assistant"`. — otherwise → `"system"` (the initial text after the first line). — The resulting triples `(role content temp)` are flattened with `mapcan` and passed to `completions` along with `*Model`. If the API call returns successfully, `R` becomes a list of reply strings. These are split into lines again so we can paste them nicely into the buffer. If there is an error, the second branch: (mapcar '((X) (chop (cdr X))) (fish '((X) (and (pair X) (memq (car X) '(error message)) (str? (cdr X)) ) ) *Result ) ) — Searches `*Result` for any `(error . "...")` or `(message . "...")` string pairs. — Returns them as lines instead, so the error text is visible in the buffer. #### Pasting the response into the buffer (move 'goAbs 1 T) (paste (make (link T (chop (if (res 'error) "???" "==="))) (chain R) (unless (res 'error) (and (last R) (link NIL)) (link (chop "+++") NIL) (evCmd (cons (res 'usage 'total_tokens) (res 'choices 'finish_reason) ) ) ) ) 1 ) (=: buffer fmt *Columns) ) ) ) ) — The cursor is moved to the end (`goAbs 1 T`). — `paste` inserts: — A marker line: — `"???"` if there was an error, — or `"==="` otherwise (start of assistant’s response). — All response lines (`R`). — When there is no error, appends: — An empty line (if needed), — A `+++` marker for the next user prompt. — Additionally, it calls `evCmd` with `(total_tokens . finish_reason)` extracted from the response to show token usage info in the Vip REPL. — Finally, buffer formatting is recalculated (`=: buffer fmt *Columns`). Thus the buffer remains a self-contained, incremental chat log that can be re-sent in full at any time. ### `:ai?` – show raw JSON (cmd "ai?" (L Lst Cnt) (symbols '(ai vip pico) (evCmd (pretty *Result)) ) ) — Dumps `*Result` (the last JSON response) as a nicely formatted s-expression using `pretty`. — Useful for debugging, to see all returned fields from OpenAI. ## 3. Conversation file structure (recap) Per-buffer chat format: 1. First line: model, optionally followed by max token limit gpt-4.1-nano 1024 2. Optional system description lines (the “role” / behavior of the model). 3. `+++` starts a user prompt. — Optionally `+++ N` with integer `N` as temperature. 4. `===` starts an assistant response. 5. `:ai`: — Sends the entire conversation so far (system, all prompts and responses). — Appends the new assistant response plus another `+++` for the next prompt. 6. `:models [prefix]`: — Shows available models, optionally filtered by prefix. 7. `:ai?`: — Shows raw JSON response. Example session (shortened): gpt-4.1-nano +++ What is the capital of Bavaria? === ; after :ai The capital of Bavaria is Munich. +++ How many inhabitants does it have? === ; after another :ai As of 2023, Munich has a population of approximately 1.5 million inhabitants. +++ Have fun! Next endpoint will probably be "responses". ☺/ A!ex P.S. I produced this mail by editing the attached file "Chat" with Vip in the way described above. I only changed formatting a little and fixed two or three minor errors.
gpt-5.1 You are a PicoLisp programmer announcing a new feature to the PicoLisp mailing list. Try not to be too verbose. For the two source files, please explain some of the more interesting details. +++ The subject is "OpenAI API and an AI Chat plugin for the Vip editor". With PicoLisp version 26.5.19 there are two new files in the PicoLisp distribution. 1. A new library "lib/ai.l". Currently it supports only the OpenAI API "completions" endpoint. The library will be extended later for further API endpoints. Here is the code: ``` # 13may26 Software Lab. Alexander Burger (symbols 'ai 'pico) # https://developers.openai.com/api/reference/overview # https://platform.deutschlandgpt.de/docs (local) (*ApiUrl *ApiKey *Max *Result curl result res api models completions) (de curl (Res . @) (pass list "curl" "-s" (pack "https://"; *ApiUrl Res) "-H" (pack "Authorization: Bearer " *ApiKey) ) ) (de result @ (when (setq *Result (readJson)) (pass res) ) ) (de res @ (let (Rest *Result Lst (rest)) (recur (Lst Rest) (loop (NIL Lst Rest) (setq Rest (get Rest (++ Lst))) (T (and (pair Rest) (pair (caar @))) (mapcar '((Rest) (recurse Lst Rest)) Rest ) ) ) ) ) ) # (api 'file) (de api (File) (in File (setq *ApiUrl (line T) *ApiKey (line T)) (line T) ) ) # (models) (de models () (in (curl "models") (result 'data 'id) ) ) # (completions 'model 'role 'text 'temp ['role 'text 'temp ..] (de completions (Model . @) (pipe (out (curl "chat/completions" "--json" "@-") (printJson (list (cons 'model Model) (make (link 'messages T) (while (args) (link (make (link (cons 'role (next)) (cons 'content (next)) ) (and (next) (link (cons 'temperature @))) (and *Max (link (cons 'max_tokens @))) ) ) ) ) ) ) ) (result 'choices 'message 'content) ) ) ``` 'api' reads the URL and the API-Key from subsequent lines in the supplied file 'models' returns a list of supported models. 'completions' takes a model and an arbitrary number of role, text and temperature arguments. The completions API is stateless, therefore the whole conversation must be passed to the server each time. This could be done for example in an application using a database for the conversations. But it is much easier To use a Vip buffer with some markup (see below). 2. A Vip resource file "lib/vip/ai.rc.l". It is a plugin for Vip which can be loaded in "~/.pil/viprc" or any local ".viprc" file (or even on the fly in the Vip REPL). Again, here is the code, explain some of the more interesting details: ``` # 16may26 Software Lab. Alexander Burger (symbols '(pico) (load "@lib/json.l" "@lib/ai.l") ) (symbols '(ai vip pico)) (cmd "models" (L Lst Cnt) (symbols '(ai vip pico) (api "~/.ai") (prCmd (sort (extract '((S) (chop (pre? L S))) (models) ) ) ) ) ) (cmd "ai" (L Lst Cnt) (symbols '(ai vip pico) (api "~/.ai") (let (Lst (split (make (for L (: buffer text) (when (or (= '("+" "+" "+") L) (and (head '("+" "+" "+" " ") L) (format (cadr (split L " "))) ) (= '("=" "=" "=") L) ) (link 0) ) (link L) ) ) 0 ) L (split (caar Lst) " ") ) (setq *Model (pack (car L)) *Max (format (cadr L)) ) (let? R (or (mapcan '((S) (split (chop S) "\n")) (apply completions (mapcan '((L) (let? S (glue "\n" (cdr L)) (cond ((head '("+" "+" "+") (setq L (car L))) (list "user" S (format (cadr (split L " ")))) ) ((head '("=" "=" "=") L) (list "assistant" S NIL) ) (T (list "system" S NIL)) ) ) ) Lst ) *Model ) ) (mapcar '((X) (chop (cdr X))) (fish '((X) (and (pair X) (memq (car X) '(error message)) (str? (cdr X)) ) ) *Result ) ) ) (move 'goAbs 1 T) (paste (make (link T (chop (if (res 'error) "???" "==="))) (chain R) (unless (res 'error) (and (last R) (link NIL)) (link (chop "+++") NIL) (evCmd (cons (res 'usage 'total_tokens) (res 'choices 'finish_reason) ) ) ) ) 1 ) (=: buffer fmt *Columns) ) ) ) ) (cmd "ai?" (L Lst Cnt) (symbols '(ai vip pico) (evCmd (pretty *Result)) ) ) ``` It adds three new commands to Vip, ":models", ":ai" and ":ai?". ":models" lists all supported models in the Vip command window (increase the window size if necessary). Optionally you can supply a string prefix to filter the results. For example, ":models gpt" might list gpt-4.1 gpt-5 gpt-5.1 gpt-5.2 gpt-5.4 ":ai" sends the whole conversation so far to the server, and pastes the response (i.e. appends it) to the buffer. ":ai?" displays the whole Json structure of the response as an s-expression for debugging purposes. 3. Structure of the conversation file So there is one dedicated file for each conversation thread. To separate metadata, prompts and responses in the file, it must have the following structure: The first line specifies the model, optionally followed by a token limit. Then an optional text may specify the model's role and behavior. A line with three plus characters indicates the start of a user prompt. It may optionally have a number 0, 1 or 2 (only integers are supported) for the temperature. A line with three equal characters indicates the start of a response. Example conversation: ``` gpt-4.1-nano +++ ``` With first prompt: ``` gpt-4.1-nano +++ What is the capital of Bavaria? ``` Typing ":ai<enter>" gives: ``` gpt-4.1-nano +++ What is the capital of Bavaria? === The capital of Bavaria is Munich. +++ ``` Follow-up prompt and ":ai<enter>" gives: ``` gpt-4.1-nano +++ What is the capital of Bavaria? === The capital of Bavaria is Munich. +++ How many inhabitants does it have? === As of 2023, Munich has a population of approximately 1.5 million inhabitants. +++ ```
