branch: externals/llm
commit 8a63863b11410a9695f07bdbf8a974792d85910e
Author: Andrew Hyatt <[email protected]>
Commit: GitHub <[email protected]>
Almost always raise new llm specific signals, add llm specific error
handling (#238)
This adds a hierarchy of error types, with the root type `llm-error`.
Authors can use this to distinguish errors from the `llm` package with
other errors. We also detect tool calling errors specifically, and raise
them as their own types of errors. This includes the LLM calling an
unavailable tool, or calling with the wrong argument.
This fixes https://github.com/ahyatt/llm/issues/237
---
NEWS.org | 2 ++
README.org | 51 ++++++++++++++++++++++++++++++++
llm-claude.el | 11 +++----
llm-fake.el | 12 ++++++--
llm-openai.el | 8 +++--
llm-prompt-test.el | 12 ++++----
llm-prompt.el | 52 +++++++++++++++++----------------
llm-provider-utils-test.el | 73 ++++++++++++++++++++++++++++++++++++++++++++++
llm-provider-utils.el | 41 +++++++++++++++++++-------
llm-request-plz.el | 42 ++++++++++++++++++--------
llm-vertex.el | 10 +++++--
llm.el | 64 +++++++++++++++++++++++++++++++++-------
12 files changed, 303 insertions(+), 75 deletions(-)
diff --git a/NEWS.org b/NEWS.org
index e2e13b8281..9659cfdade 100644
--- a/NEWS.org
+++ b/NEWS.org
@@ -1,3 +1,5 @@
+* Version 0.29.0
+- Check for tool use mismatches and define new errors for them
* Version 0.28.5
- Improved the tool calling docs
- Fix for running tools in the original buffer with streaming
diff --git a/README.org b/README.org
index c52c9f4156..0a16f46e3a 100644
--- a/README.org
+++ b/README.org
@@ -227,6 +227,57 @@ Conversations can take place by repeatedly calling
~llm-chat~ and its variants.
#+end_src
** Caution about ~llm-chat-prompt-interactions~
The interactions in a prompt may be modified by conversation or by the
conversion of the context and examples to what the LLM understands. Different
providers require different things from the interactions. Some can handle
system prompts, some cannot. Some require alternating user and assistant chat
interactions, others can handle anything. It's important that clients keep to
behaviors that work on all providers. Do not attempt to read or manipulate
~llm-chat-prompt-interactions~ af [...]
+** Error handling
+The =llm= package defines several error symbols that can be signaled during
operations. These errors follow a hierarchy, allowing you to catch errors at
different levels of specificity.
+
+*** Error hierarchy
+All LLM-related errors inherit from ~llm-error~:
+- ~llm-error~: The base error for all LLM operations.
+ - ~llm-invalid-argument~: Signaled when an invalid argument is provided to
an LLM function.
+ - ~llm-not-supported~: Signaled when a requested operation or feature is not
supported by the provider or model.
+ - ~llm-provider-error~: The base error for provider-related issues.
+ - ~llm-provider-unconfigured~: Signaled when the provider is not
configured correctly (e.g., missing API key).
+ - ~llm-request-error~: The base error for request failures.
+ - ~llm-request-timeout~: Signaled when a request times out.
+ - ~llm-request-authentication-error~: Signaled when authentication fails
(e.g., invalid API key).
+ - ~llm-request-bad-request~: Signaled when the request was invalid (e.g.,
bad format).
+ - ~llm-tool-call-error~: The base error for all tool calling errors.
+ - ~llm-tool-unknown-tool~: Signaled when an LLM attempts to call a tool
that was not provided in the prompt's tools list.
+ - ~llm-tool-unknown-argument~: Signaled when an LLM calls a tool with an
argument that is not defined in the tool's argument specification.
+ - ~llm-tool-missing-argument~: Signaled when an LLM calls a tool but omits
a required (non-optional) argument.
+
+*** Error data
+Most errors return a list with a string as their data (the same as the
standard error).
+
+Some errors have structured data instead. These are those errors;
+When these errors are signaled, they include a data plist with additional
information:
+
+- ~llm-tool-unknown-tool~: ~(:tool TOOL-NAME)~
+ - ~TOOL-NAME~: The name of the tool the LLM attempted to call.
+
+- ~llm-tool-unknown-argument~: ~(:tool TOOL-NAME :arg ARG-KEY)~
+ - ~TOOL-NAME~: The name of the tool being called.
+ - ~ARG-KEY~: The argument key that was not recognized.
+
+- ~llm-tool-missing-argument~: ~(:tool TOOL-NAME :arg ARG-SPEC)~
+ - ~TOOL-NAME~: The name of the tool being called.
+ - ~ARG-SPEC~: The full argument specification plist for the missing required
argument.
+
+*** Example
+#+begin_src emacs-lisp
+(condition-case err
+ (llm-chat my-provider my-prompt)
+ (llm-tool-unknown-tool
+ (message "Unknown tool requested: %s" (plist-get (cdr err) :tool)))
+ (llm-tool-call-error
+ (message "Tool call error: %s" err))
+ (llm-invalid-argument
+ (message "Invalid argument: %s" (error-message-string err)))
+ (llm-provider-error
+ (message "Provider error: %s" (error-message-string err)))
+ (llm-error
+ (message "LLM error: %s" err)))
+#+end_src
** Tool use
Tool use is a way to give the LLM a list of functions it can call, and have it
call the functions for you. The standard interaction has the following steps:
1. The client sends the LLM a prompt with tools it can use.
diff --git a/llm-claude.el b/llm-claude.el
index 4becc74670..dd7eae0171 100644
--- a/llm-claude.el
+++ b/llm-claude.el
@@ -42,7 +42,7 @@
(cl-defmethod llm-provider-prelude ((provider llm-claude))
(unless (llm-claude-key provider)
- (error "No API key provided for Claude")))
+ (signal 'llm-provider-unconfigured '("No API key provided for Claude"))))
(defun llm-claude--tool-call (tool)
"A Claude version of a function spec for TOOL."
@@ -117,8 +117,9 @@
('any "any")
('none "none")
((pred stringp) "tool")
- (_ (error "Unknown tool choice
option: %s"
-
(llm-tool-options-tool-choice options)))))
+ (_ (signal 'llm-not-supported
+ (list (format
"Unknown tool choice option: %s"
+
(llm-tool-options-tool-choice options)))))))
(when (stringp (llm-tool-options-tool-choice
options))
(list :name (llm-tool-options-tool-choice
options)))))))
(when (llm-chat-prompt-reasoning prompt)
@@ -153,9 +154,9 @@
"image")
:source ,source)))
(t
- (error "Unsupported multipart content: %s" part))))
+ (signal 'llm-invalid-argument
+ (list (format "Unsupported multipart content: %s"
part))))))
(llm-multipart-parts content))))
-
(cl-defmethod llm-provider-extract-tool-uses ((_ llm-claude) response)
(let ((content (append (assoc-default 'content response) nil)))
(cl-loop for item in content
diff --git a/llm-fake.el b/llm-fake.el
index b20845f3ce..ce53d42131 100644
--- a/llm-fake.el
+++ b/llm-fake.el
@@ -66,7 +66,10 @@ message cons. If nil, the response will be a simple vector."
(pcase (type-of result)
('string result)
('cons (signal (car result) (cdr result)))
- (_ (error "Incorrect type found in `chat-action-func': %s"
(type-of result)))))
+ (_ (signal
+ 'llm-invalid-argument
+ (list (format "Incorrect type found in
`chat-action-func': %s"
+ (type-of result)))))))
"Sample response from `llm-chat-async'")))
(setf (llm-chat-prompt-interactions prompt)
(append (llm-chat-prompt-interactions prompt)
@@ -87,7 +90,9 @@ message cons. If nil, the response will be a simple vector."
(pcase (type-of result)
('string (setq text result))
('cons (signal (car result) (cdr result)))
- (_ (error "Incorrect type found in `chat-action-func': %s" (type-of
result))))))
+ (_ (signal 'llm-invalid-argument
+ (list (format "Incorrect type found in
`chat-action-func': %s"
+ (type-of result))))))))
(let ((accum ""))
(mapc (lambda (word)
(setq accum (concat accum word " "))
@@ -110,7 +115,8 @@ message cons. If nil, the response will be a simple
vector."
(pcase (type-of result)
('vector result)
('cons (signal (car result) (cdr result)))
- (_ (error "Incorrect type found in `chat-embedding-func': %s"
(type-of result)))))
+ (_ (signal 'llm-invalid
+ (list (format "Incorrect type found in
`chat-embedding-func': %s" (type-of result)))))))
[0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9]))
(cl-defmethod llm-embedding-async ((provider llm-fake) string vector-callback
error-callback)
diff --git a/llm-openai.el b/llm-openai.el
index 2b67ae5a10..95e257fa19 100644
--- a/llm-openai.el
+++ b/llm-openai.el
@@ -99,7 +99,8 @@ PROVIDER is the Open AI provider struct."
(cl-defmethod llm-openai--check-key ((provider llm-openai))
(unless (llm-openai-key provider)
- (error "To call Open AI API, add a key to the `llm-openai' provider")))
+ (signal 'llm-provider-unconfigured
+ '("To call Open AI API, add a key to the `llm-openai' provider"))))
(cl-defmethod llm-openai--check-key ((_ llm-openai-compatible))
;; It isn't always the case that a key is needed for Open AI compatible APIs.
@@ -212,7 +213,9 @@ PROVIDER is the Open AI provider struct."
(list
:function (list :name (llm-tool-options-tool-choice options))
:type "function"))
- (_ (error "Unknown tool choice option: %s"
(llm-tool-options-tool-choice options)))))))
+ (_ (signal 'llm-not-supported
+ (list (format "Unknown tool choice option: %s"
+ (llm-tool-options-tool-choice
options)))))))))
(defun llm-openai--build-tool-interaction (interaction)
"Build the tool interaction for INTERACTION."
@@ -232,6 +235,7 @@ PROVIDER is the Open AI provider struct."
(defun llm-openai--build-tool-uses (fcs)
"Convert back from the generic representation to the Open AI.
+
FCS is a list of `llm-provider-utils-tool-use' structs."
(vconcat
(mapcar (lambda (fc)
diff --git a/llm-prompt-test.el b/llm-prompt-test.el
index b8f00c6110..543f248237 100644
--- a/llm-prompt-test.el
+++ b/llm-prompt-test.el
@@ -61,21 +61,21 @@ to converge."
:name 'var-1 :tickets 10
:generator (funcall
(iter-lambda ()
- (while t (iter-yield 'a)))))
+ (while t (iter-yield 'a)))))
(make-llm-prompt-variable-full
:name 'var-2 :tickets 20
:generator (funcall
(iter-lambda ()
- (while t (iter-yield 'b)))))
+ (while t (iter-yield 'b)))))
(make-llm-prompt-variable-full
:name 'var-3 :tickets 30
:generator (funcall
(iter-lambda ()
- (while t (iter-yield 'c)))))
+ (while t (iter-yield 'c)))))
(make-llm-prompt-variable-full
:name 'var-4 :tickets 30
:generator (funcall (iter-lambda ()
- (iter-yield 'd))))))
+ (iter-yield 'd))))))
(selector (llm-prompt--select-tickets vars))
(iters 20000))
(dotimes (_ iters)
@@ -98,7 +98,7 @@ to converge."
:name 'var-1 :tickets 10
:generator (funcall
(iter-lambda ()
- (while t (iter-yield 'a)))))))
+ (while t (iter-yield 'a)))))))
(selector (llm-prompt--select-tickets vars)))
(push (cdr (iter-next selector)) result)
(push (cdr (iter-next selector)) result)
@@ -110,7 +110,7 @@ to converge."
:name 'var :tickets 10
:generator (funcall
(iter-lambda ()
- (iter-yield 'a))))))
+ (iter-yield 'a))))))
(selector (llm-prompt--select-tickets vars)))
(should (equal (cdr (iter-next selector)) 'a))
(condition-case nil
diff --git a/llm-prompt.el b/llm-prompt.el
index f6f10ec623..8116719f05 100644
--- a/llm-prompt.el
+++ b/llm-prompt.el
@@ -147,18 +147,18 @@ counting the tickets not specified, which should equal
this number."
(while using-vars
(let ((r (random total)))
(cl-loop for v in using-vars
- with count = 0
- do
- (cl-incf count (llm-prompt-variable-tickets v))
- until (> count r)
- finally
- (condition-case nil
- (iter-yield (cons (llm-prompt-variable-name v)
- (iter-next
(llm-prompt-variable-full-generator v))))
- (iter-end-of-sequence
- (progn
- (setq using-vars (remove v using-vars)
- total (- total (llm-prompt-variable-tickets
v)))))))))))
+ with count = 0
+ do
+ (cl-incf count (llm-prompt-variable-tickets v))
+ until (> count r)
+ finally
+ (condition-case nil
+ (iter-yield (cons (llm-prompt-variable-name v)
+ (iter-next
(llm-prompt-variable-full-generator v))))
+ (iter-end-of-sequence
+ (progn
+ (setq using-vars (remove v using-vars)
+ total (- total (llm-prompt-variable-tickets
v)))))))))))
(defun llm-prompt--ensure-iterator (var)
"Return an iterator for VAR, if it's not already one.
@@ -251,7 +251,8 @@ from the variable."
(add-location (if (consp (cdr val-cons))
(cddr val-cons) 'front)))
(unless (member add-location '(front back))
- (error "Add location specification must be one of 'front or
'back"))
+ (signal 'llm-invalid-argument
+ '("Add location specification must be one of 'front
or 'back")))
;; Only add if there is space, otherwise we ignore this value.
(when (<= (+ total-tokens (llm-count-tokens provider sval))
(* (/ llm-prompt-default-max-pct 100.0)
@@ -267,17 +268,17 @@ from the variable."
(push (cons var (list sval)) final-vals)))))
(iter-end-of-sequence nil)))
(cl-loop for (var-name . val) in final-vals
- do
- (goto-char
- (llm-prompt-variable-marker
- (seq-find (lambda (e) (eq (llm-prompt-variable-name e)
- var-name))
- vars)))
- (insert (format "%s" (if (listp val)
- (mapconcat (lambda (e)
- (format "%s" e))
- (reverse val) " ")
- val)))))
+ do
+ (goto-char
+ (llm-prompt-variable-marker
+ (seq-find (lambda (e) (eq (llm-prompt-variable-name e)
+ var-name))
+ vars)))
+ (insert (format "%s" (if (listp val)
+ (mapconcat (lambda (e)
+ (format "%s" e))
+ (reverse val) " ")
+ val)))))
(buffer-substring-no-properties (point-min) (point-max))))
(defun llm-prompt-get (name)
@@ -295,7 +296,8 @@ generator."
(with-temp-buffer
(let ((prompt-text (gethash name llm-prompt-prompts)))
(unless prompt-text
- (error "Could not find prompt with name %s" name))
+ (signal 'llm-invalid-argument
+ (list (format "Could not find prompt with name %s" name))))
(apply #'llm-prompt-fill-text prompt-text provider keys))))
(provide 'llm-prompt)
diff --git a/llm-provider-utils-test.el b/llm-provider-utils-test.el
index faae8228cb..7b679cb58e 100644
--- a/llm-provider-utils-test.el
+++ b/llm-provider-utils-test.el
@@ -23,6 +23,7 @@
(require 'cl-macs)
(require 'llm-provider-utils)
+(require 'llm)
(ert-deftest llm-provider-utils-openai-arguments ()
(let* ((args
@@ -160,5 +161,77 @@
(should (equal '(:a 1 :b nil)
(llm-provider-utils--normalize-args '(:a 1 :b :json-false)))))
+(cl-defstruct llm-testing-provider (llm-standard-chat-provider) ())
+
+(cl-defmethod llm-provider-populate-tool-uses ((provider llm-testing-provider)
+ prompt tool-uses))
+
+(ert-deftest llm-provider-utils-execute-tool-uses--missing-tool ()
+ (should-error
+ (llm-provider-utils-execute-tool-uses
+ (make-llm-testing-provider)
+ (llm-make-chat-prompt
+ ""
+ :tools (list
+ (llm-make-tool
+ :name "tool-a"
+ :description "Tool A"
+ :function (lambda (&rest args) "Result A")
+ :args '())))
+ (list
+ (make-llm-provider-utils-tool-use
+ :id "1"
+ :name "tool-b"
+ :args '()))
+ nil
+ nil
+ #'identity)
+ :type '(llm-tool-unknown-tool)))
+
+(ert-deftest llm-provider-utils-execute-tool-uses--unknown-arg ()
+ (should-error
+ (llm-provider-utils-execute-tool-uses
+ (make-llm-testing-provider)
+ (llm-make-chat-prompt
+ ""
+ :tools (list
+ (llm-make-tool
+ :name "tool-a"
+ :description "Tool A"
+ :function (lambda (&rest args) "Result A")
+ :args '((:name "arg1" :type string :description "Argument 1")))))
+ (list
+ (make-llm-provider-utils-tool-use
+ :id "1"
+ :name "tool-a"
+ :args '((arg1 . "value1")
+ (arg2 . "value2"))))
+ nil
+ nil
+ #'identity)
+ :type '(llm-tool-unknown-argument)))
+
+(ert-deftest llm-provider-utils-execute-tool-uses--missing-arg ()
+ (should-error
+ (llm-provider-utils-execute-tool-uses
+ (make-llm-testing-provider)
+ (llm-make-chat-prompt
+ ""
+ :tools (list
+ (llm-make-tool
+ :name "tool-a"
+ :description "Tool A"
+ :function (lambda (&rest args) "Result A")
+ :args '((:name "arg1" :type string :description "Argument 1")))))
+ (list
+ (make-llm-provider-utils-tool-use
+ :id "1"
+ :name "tool-a"
+ :args '()))
+ nil
+ nil
+ #'identity)
+ :type '(llm-tool-missing-argument)))
+
(provide 'llm-provider-utils-test)
;;; llm-provider-utils-test.el ends here
diff --git a/llm-provider-utils.el b/llm-provider-utils.el
index da3f349560..55afba5532 100644
--- a/llm-provider-utils.el
+++ b/llm-provider-utils.el
@@ -397,8 +397,11 @@ Any strings will be concatenated, integers will be added,
etc."
(if new
(progn
(unless (eq (type-of current) (type-of new))
- (error "Cannot accumulate different types of streaming results:
%s and %s"
- current new))
+ (signal
+ 'llm-invalid-argument
+ (list (format
+ "Cannot accumulate different types of streaming results:
%s and %s"
+ current new))))
(pcase (type-of current)
('string (concat current new))
('integer (+ current new))
@@ -820,7 +823,8 @@ This will convert all :json-false and :false values to nil."
((member args '(:json-false :false)) nil)
(t args)))
-(defun llm-provider-utils-execute-tool-uses (provider prompt tool-uses
multi-output partial-result success-callback)
+(defun llm-provider-utils-execute-tool-uses (provider prompt tool-uses
multi-output
+ partial-result
success-callback)
"Execute TOOL-USES, a list of `llm-provider-utils-tool-use'.
A response suitable for returning to the client will be returned.
@@ -845,14 +849,22 @@ have returned results."
for tool-use in tool-uses do
(let* ((name (llm-provider-utils-tool-use-name tool-use))
(arguments (llm-provider-utils-tool-use-args tool-use))
- (tool (seq-find
- (lambda (f) (equal name (llm-tool-name f)))
- (llm-chat-prompt-tools prompt)))
+ (tool (or
+ (seq-find
+ (lambda (f) (equal name (llm-tool-name f)))
+ (llm-chat-prompt-tools prompt))
+ (signal 'llm-tool-unknown-tool `(:tool ,name))))
(call-args (cl-loop for arg in (llm-tool-args tool)
- collect (cdr (seq-find (lambda (a)
- (eq (intern
(plist-get arg :name))
- (car a)))
- arguments))))
+ collect (cdr (or
+ (seq-find (lambda (a)
+ (eq (intern
(plist-get arg :name))
+ (car a)))
+ arguments)
+ ;; Arg wasn't found, if it wasn't
+ ;; optional, signal an error.
+ (unless (plist-get arg :optional)
+ (signal
'llm-tool-missing-argument
+ `((:tool ,name :arg
,arg))))))))
(end-func (lambda (result)
(llm--log
'api-funcall
@@ -871,6 +883,15 @@ have returned results."
(append partial-result
`(:tool-results
,tool-use-and-results)))
tool-use-and-results))))))
+ ;; Check to see that there were no unknown args.
+ (dolist (arg-key (map-keys arguments))
+ (unless (seq-find
+ (lambda (a) (eq (intern (plist-get a :name))
+ arg-key))
+ (llm-tool-args tool))
+ (signal 'llm-tool-unknown-argument
+ `((:tool ,name :arg ,arg-key)))))
+
(if (llm-tool-async tool)
(apply (llm-tool-function tool)
(append (list end-func) call-args))
diff --git a/llm-request-plz.el b/llm-request-plz.el
index 57fb0f98fa..342f32f2c1 100644
--- a/llm-request-plz.el
+++ b/llm-request-plz.el
@@ -26,6 +26,7 @@
(require 'plz-media-type)
(require 'rx)
(require 'url-http)
+(require 'llm)
(defcustom llm-request-plz-timeout nil
"The number of seconds to wait for a response from a HTTP server.
@@ -74,31 +75,48 @@ TIMEOUT is the number of seconds to wait for a response."
(seq-let [error-sym message data] error
(cond
((eq 'plz-http-error error-sym)
- (let ((response (plz-error-response data)))
- (error "LLM request failed with code %d: %s (additional
information: %s)"
- (plz-response-status response)
- (nth 2 (assq (plz-response-status response) url-http-codes))
- (plz-response-body response))))
+ (let* ((response (plz-error-response data))
+ (status (plz-response-status response))
+ (body (plz-response-body response)))
+ (cond
+ ((or (eq status 401) (eq status 403))
+ (signal 'llm-request-authentication-error (list body)))
+ ((eq status 400)
+ (signal 'llm-request-bad-request (list body)))
+ (t
+ (signal 'llm-request-error
+ (list (format "LLM request failed with code %d: %s
(additional information: %s)"
+ status
+ (nth 2 (assq status url-http-codes))
+ body)))))))
((and (eq 'plz-curl-error error-sym)
(eq 28 (car (plz-error-curl-error data))))
- (error "LLM request timed out"))
+ (signal 'llm-request-timeout nil))
(t (signal error-sym (list message data))))))))
(defun llm-request-plz--handle-error (error on-error)
"Handle the ERROR with the ON-ERROR callback."
(cond ((plz-error-curl-error error)
(let ((curl-error (plz-error-curl-error error)))
- (funcall on-error 'error
- (format "curl error code %d: %s"
- (car curl-error)
- (cdr curl-error)))))
+ (if (eq 28 (car curl-error))
+ (funcall on-error 'llm-request-timeout (cdr curl-error))
+ (funcall on-error 'llm-request-error
+ (format "curl error code %d: %s"
+ (car curl-error)
+ (cdr curl-error))))))
((plz-error-response error)
(when-let ((response (plz-error-response error))
(status (plz-response-status response))
(body (plz-response-body response)))
- (funcall on-error 'error body)))
+ (cond
+ ((or (eq status 401) (eq status 403))
+ (funcall on-error 'llm-request-authentication-error body))
+ ((eq status 400)
+ (funcall on-error 'llm-request-bad-request body))
+ (t
+ (funcall on-error 'llm-request-error body)))))
((plz-error-message error)
- (funcall on-error 'error (plz-error-message error)))
+ (funcall on-error 'llm-request-error (plz-error-message error)))
(t (user-error "Unexpected error: %s" error))))
(cl-defun llm-request-plz-async (url &key headers data on-success media-type
diff --git a/llm-vertex.el b/llm-vertex.el
index 1186c3e530..da583b98bf 100644
--- a/llm-vertex.el
+++ b/llm-vertex.el
@@ -104,7 +104,11 @@ information than standard tool use."
(float-time (time-subtract (current-time) (or
(llm-vertex-key-gentime provider) 0)))))
(let ((result (string-trim (shell-command-to-string (concat
llm-vertex-gcloud-binary " auth print-access-token")))))
(when (string-match-p "ERROR" result)
- (error "Could not refresh gcloud access token, received the following
error: %s" result))
+ (signal
+ 'llm-provider-error
+ (list (format
+ "Could not refresh gcloud access token, received the following
error: %s"
+ result))))
;; We need to make this unibyte, or else it doesn't causes problems when
;; the user is using multibyte strings.
(setf (llm-vertex-key provider) (encode-coding-string result 'utf-8)))
@@ -269,7 +273,9 @@ information than standard tool use."
('none "NONE")
('any "ANY")
((pred stringp) "ANY")
- (_ (error "Unknown tool choice option: %s"
(llm-tool-options-tool-choice options)))))
+ (_ (signal
+ 'llm-not-supported
+ (list (format "Unknown tool choice option: %s"
(llm-tool-options-tool-choice options)))))))
(when (stringp (llm-tool-options-tool-choice options))
`(:allowedFunctionNames [,(llm-tool-options-tool-choice
options)]))))))))
(llm-vertex--chat-parameters prompt model)))
diff --git a/llm.el b/llm.el
index f8c71519d2..b93e3a5e7c 100644
--- a/llm.el
+++ b/llm.el
@@ -338,9 +338,9 @@ different providers, because it is likely to cause a
request error. The
cars of the alist are strings and the cdrs can be strings, numbers or
vectors (if a list). This is optional."
(unless content
- (error "CONTENT is required"))
+ (signal 'llm-invalid-argument '("CONTENT is required")))
(when (and (listp content) (zerop (mod (length content) 2)))
- (error "CONTENT, as a list, must have an odd number of elements"))
+ (signal 'llm-invalid-argument '("CONTENT, as a list, must have an odd
number of elements")))
(make-llm-chat-prompt
:context context
:examples examples
@@ -396,7 +396,8 @@ the output) `tool-uses' (a list of plists with tool `:name'
and
(cl-defmethod llm-chat ((_ (eql nil)) _ &optional _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-chat :before (provider _ &optional _)
"Issue a warning if the LLM is non-free."
@@ -513,7 +514,8 @@ be passed to `llm-cancel-request'."
(cl-defmethod llm-chat-streaming ((_ (eql nil)) _ _ _ _ &optional _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-chat-streaming :before (provider _ _ _ _ &optional _)
"Issue a warning if the LLM is non-free."
@@ -581,11 +583,12 @@ be passed to `llm-cancel-request'."
(lambda (text) (insert-text text))
(lambda (text) (insert-text text)
(funcall finish-callback))
- (lambda (_ msg) (error "Error calling the LLM:
%s" msg))))))))
+ (lambda (type msg) (signal type (list
msg)))))))))
(cl-defmethod llm-chat-async ((_ (eql nil)) _ _ _ &optional _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-chat-async :before (provider _ _ _ &optional _)
"Issue a warning if the LLM is non-free."
@@ -646,7 +649,8 @@ call."
(cl-defmethod llm-embedding ((_ (eql nil)) _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-embedding :before (provider _)
"Issue a warning if the LLM is non-free."
@@ -666,7 +670,8 @@ be passed to `llm-cancel-request'."
(cl-defmethod llm-embedding-async ((_ (eql nil)) _ _ _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-embedding-async :before (provider _ _ _)
"Issue a warning if the LLM is non-free."
@@ -685,7 +690,8 @@ PROVIDER is the provider struct that will be used for an
LLM call."
(cl-defmethod llm-batch-embeddings ((_ (eql nil)) _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-batch-embeddings :before (provider _)
"Issue a warning if the LLM is non-free."
@@ -704,7 +710,8 @@ and a string message."
(cl-defmethod llm-batch-embeddings-async ((_ (eql nil)) _ _ _)
"Catch trivial configuration mistake."
- (error "LLM provider was nil. Please set the provider in the application
you are using"))
+ (signal 'llm-provider-unconfigured
+ '("LLM provider was nil. Please set the provider in the application
you are using")))
(cl-defmethod llm-batch-embeddings-async :before (provider _ _ _)
"Issue a warning if the LLM is non-free."
@@ -807,5 +814,42 @@ This should only be used for logging or debugging."
(when (llm-chat-prompt-max-tokens prompt)
(format "Max tokens: %s\n" (llm-chat-prompt-max-tokens prompt)))))
+;; Error conditions
+(define-error
+ 'llm-error "An error occurred in LLM operations")
+
+(define-error
+ 'llm-invalid-argument "Invalid argument for LLM operation" 'llm-error)
+(define-error
+ 'llm-not-supported "Operation or feature not supported" 'llm-error)
+
+(define-error
+ 'llm-provider-error "LLM provider error" 'llm-error)
+(define-error
+ 'llm-provider-unconfigured "LLM provider is not configured correctly"
'llm-provider-error)
+
+(define-error
+ 'llm-request-error "LLM request failed" 'llm-error)
+(define-error
+ 'llm-request-timeout "LLM request timed out" 'llm-request-error)
+(define-error
+ 'llm-request-authentication-error "LLM request authentication failed"
'llm-request-error)
+(define-error
+ 'llm-request-bad-request "LLM request was invalid" 'llm-request-error)
+
+(define-error
+ 'llm-tool-call-error "An error occurred when calling an LLM tool" 'llm-error)
+(define-error
+ 'llm-tool-unknown-tool "An LLM tool was called, but not found in the
available tools"
+ 'llm-tool-call-error)
+(define-error
+ 'llm-tool-unknown-argument
+ "An LLM tool was called with an unknown argument"
+ 'llm-tool-call-error)
+(define-error
+ 'llm-tool-missing-argument
+ "A tool was called missing a required argument"
+ 'llm-tool-call-error)
+
(provide 'llm)
;;; llm.el ends here
