bamaer commented on issue #8441:
URL: https://github.com/apache/hop/issues/8441#issuecomment-5718369270
Before I build the transform, a proposal for `AiProvider`, because it
changes the shape of this PR.
### The problem
`AiProvider` carries one `modelName`, because it was built for chat. An
embedding transform needs a second model from the same endpoint and
credentials, and a reranker after a vector search would need a third.
langchain4j already separates these: `ChatModel`, `EmbeddingModel`,
`ScoringModel`, plus `ImageModel`, `ModerationModel` and the two audio ones.
Without a change, every transform has to carry its own model name, so it
gets repeated in every pipeline, or you create one provider object per role and
duplicate the base URL and API key in each.
### Proposal: model roles on the provider
```java
// unchanged
private String modelName = ""; // still the chat model
private String temperature = "0.3";
// added
@HopMetadataProperty(key = "models")
private List<AiProviderModel> models = new ArrayList<>();
public class AiProviderModel {
private AiModelRole role; // CHAT, EMBEDDING, SCORING, IMAGE,
MODERATION
private String modelName;
}
```
One Ollama provider then covers:
```
CHAT llama3.2
EMBEDDING nomic-embed-text
SCORING bge-reranker-v2-m3
```
A list rather than `embeddingModelName` plus `rerankModelName` plus the next
one, so adding a role is a new enum constant instead of a new field, a new
editor row and a new migration each time. In the editor it is one `TableView`
with a role dropdown and a model name, the same shape as the pgvector column
mappings.
### How a transform picks its model
The role is fixed by what the transform is, so the user never chooses one.
`Embed text` can only use an `EmbeddingModel`, so it resolves `EMBEDDING`.
Language Model Chat resolves `CHAT`. A later rerank transform resolves
`SCORING`. Point all three at the same provider object and each gets the right
model, with no role dropdown anywhere in a transform dialog.
This means **role is unique within a provider**, one row per role, which is
what keeps the lookup unambiguous. A transform needing a different model than
the provider's default for its role sets an optional override on the transform
itself.
Resolution order for embeddings: transform override, then the provider's
`EMBEDDING` row, then `modelName`.
### What changes
- `AiProvider`: the `models` list, a new `AiProviderModel`, an `AiModelRole`
enum, and a `resolveModelName(role)` helper.
- `AiProviderEditor`: one `TableView`.
- `AiChatFactory.applyConnection`: one line, `provider.getModelName()`
becomes `provider.resolveModelName(CHAT)`, so a `CHAT` row is honoured rather
than ignored.
- **Language Model Chat: nothing.** It never reads `AiProvider` fields, it
only calls `LanguageModelChatAiProviderSupport.resolve(...)`, which reflects
into the factory.
- `AiEmbeddingFactory`: new, alongside `AiChatFactory`, resolving
`EMBEDDING`.
### Backward compatibility
Additive, and fully compatible rather than mostly:
- `modelName` and `temperature` keep their current meaning and are not
deprecated.
- An existing serialized provider has no `models` key, so it deserializes to
an empty list, `CHAT` falls back to `modelName`, and it behaves exactly as it
does today.
- A provider edited in a newer Hop and opened in an older one keeps working:
the unknown `models` key is ignored and `modelName` is still there.
- There is no serialized `ai-provider` JSON anywhere in the repo today, not
even in the integration tests, so there is no stored data to migrate.
- The editor shows an empty table for existing objects, so nothing has to be
filled in.
The one case that is not automatic is a user who wants chat and embeddings
from one object: they add an `EMBEDDING` row. That is new configuration for new
behaviour, not a migration.
### One related gap
`AiModelCatalog.listModelNames` keeps only `CHAT` and `OTHER` and skips the
rest, so embedding models are filtered out of the dropdown today. Listing them
needs either a type filter on `listModelNames` or a sibling method. To keep
this PR small, the model name can be free text with sensible defaults and the
dropdown can follow later.
Comments welcome before I build it, since it touches the `AiProvider`
metadata type and its editor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
