This is an automated email from the ASF dual-hosted git repository.
mattcasters pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 32907c2cb0 Issue #8629 : Document max tokens and context size in AI
Provider precedence (#8853)
32907c2cb0 is described below
commit 32907c2cb0523615c443558442d2334a87cbd223
Author: Bart Maertens <[email protected]>
AuthorDate: Sun Oct 11 22:06:19 2026 +0200
Issue #8629 : Document max tokens and context size in AI Provider
precedence (#8853)
The pgvector "Skip if empty" filter option and the AI Provider precedence
docs were done in #8630. Since then, #8784 made the provider also override Max
New Tokens, and the Ollama context size, which defaults to 16384 when the
provider leaves it empty. Add both to the precedence table and the AI Provider
page, and stop saying max tokens always come from the transform.
---
.../modules/ROOT/pages/metadata-types/ai-provider.adoc | 3 ++-
.../ROOT/pages/pipeline/transforms/languagemodelchat.adoc | 10 ++++++++--
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc
b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc
index a94b19b538..a4f5e0ee99 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/metadata-types/ai-provider.adoc
@@ -40,7 +40,8 @@ The same pattern applies to every other secret in the
project: never hard-code i
See
xref:hop-gui/perspective-ai-advisor.adoc#secrets-and-personal-information[Secrets
and personal information].
The xref:pipeline/transforms/languagemodelchat.adoc[Language model chat]
transform can optionally select a named AI Provider.
-The provider's type, base URL, API key, chat model, timeout and temperature
then take precedence over the transform's inline values; an inline value is
only used when the provider leaves that field empty.
+The provider's type, base URL, API key, chat model, timeout, temperature, max
output tokens and thinking then take precedence over the transform's inline
values; an inline value is only used when the provider leaves that field empty.
+With an Ollama provider, the context size always comes from the provider
(16384 when it is empty), and an Anthropic provider without *Max output tokens*
sends 4096 rather than the transform's value.
In particular, a temperature set on the provider overrides the temperature on
every transform that uses it, so leave *Temperature* empty here if transforms
need their own.
Input/output mapping, mock mode, proxy and retries stay on the transform.
See xref:pipeline/transforms/languagemodelchat.adoc#ai-provider-precedence[AI
Provider and inline settings] for the details.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc
index dd32da2381..e8bc9c1e04 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/languagemodelchat.adoc
@@ -98,15 +98,21 @@ When *AI Provider* is set, the transform starts from its
own settings and then c
| Timeout
| The provider's *Timeout (seconds)* if it resolves to a whole number,
otherwise the inline timeout.
+| Max New Tokens
+| The provider's *Max output tokens* if it resolves to a positive whole
number, otherwise the inline *Max New Tokens*. An Anthropic provider without
one sends 4096, not the inline value.
+
+| Context Window Size (Ollama)
+| The provider's *Context size (tokens)* if it resolves to a positive whole
number, otherwise 16384. The inline *Context Window Size* is not used.
+
| Thinking (Ollama)
| The provider's *Thinking* if it is *Off* or *On*, otherwise the inline
*Thinking*.
|===
-Variables in provider fields are resolved first, so apart from *Temperature*,
a field holding a variable that resolves to an empty value counts as empty.
Empty provider fields fall back to the inline values, not to the defaults
listed for each provider type on the xref:metadata-types/ai-provider.adoc[AI
Provider] page.
+Variables in provider fields are resolved first, so apart from *Temperature*,
a field holding a variable that resolves to an empty value counts as empty.
Empty provider fields fall back to the inline values, not to the defaults
listed for each provider type on the xref:metadata-types/ai-provider.adoc[AI
Provider] page. The exceptions are *Max New Tokens* for Anthropic and *Context
Window Size* for Ollama, described above.
The inline fallbacks come from the provider's model type, not from the *Model
API* selected on the transform. With an Anthropic provider, for example, an
empty provider *Model name* falls back to the inline Anthropic model name, even
when the transform itself was set up for OpenAI.
-All other options always come from the transform: mock mode, parallelism and
the input and output fields, and the model-specific options such as Top P, Top
K, maximum tokens, response format, proxy, retries and logging. The
model-specific ones are read from the transform's fields for the provider's
model type.
+All other options always come from the transform: mock mode, parallelism and
the input and output fields, and the model-specific options such as Top P, Top
K, response format, proxy, retries and logging. The model-specific ones are
read from the transform's fields for the provider's model type.
To tune the temperature per transform while sharing one provider, leave
*Temperature* empty on the provider and set it on each transform.
