Croway commented on code in PR #23614: URL: https://github.com/apache/camel/pull/23614#discussion_r3323018354
########## components/camel-ai/camel-openai/src/main/docs/openai-component.adoc: ########## @@ -492,6 +492,113 @@ String-valued fields are set directly. Non-string fields (numbers, booleans, obj NOTE: This maps fields from the response message's additional properties (fields not part of the standard schema). Standard response fields like `content`, `role`, and `tool_calls` are not accessible through this option. +== OpenAI-Compatible Providers + +Because the component speaks the OpenAI API, you do not need a separate component to use third-party +gateways or local model servers that expose an OpenAI-compatible API. Point `baseUrl` at the provider, +set `apiKey` if it requires one, and use the same operations and options as with OpenAI. + +[cols="1,2,2",options="header"] +|=== +| Provider | `baseUrl` | Notes + +| OpenAI | `https://api.openai.com/v1` | Default; `baseUrl` can be omitted +| OpenRouter | `https://openrouter.ai/api/v1` | Multi-model gateway with provider routing and fallbacks +| Ollama | `http://localhost:11434/v1` | Local LLM server +| LM Studio | `http://localhost:1234/v1` | Local model runner +| vLLM | `http://localhost:8000/v1` | High-throughput, self-hosted serving engine +|=== + +=== Ollama (local) + +Ollama runs models locally and does not require an API key. Review Comment: Can you add the ollama command to install and run `llama3.2`, as in the example route? and a link to the ollama website? ########## components/camel-ai/camel-openai/src/main/docs/openai-component.adoc: ########## @@ -492,6 +492,113 @@ String-valued fields are set directly. Non-string fields (numbers, booleans, obj NOTE: This maps fields from the response message's additional properties (fields not part of the standard schema). Standard response fields like `content`, `role`, and `tool_calls` are not accessible through this option. +== OpenAI-Compatible Providers + +Because the component speaks the OpenAI API, you do not need a separate component to use third-party +gateways or local model servers that expose an OpenAI-compatible API. Point `baseUrl` at the provider, +set `apiKey` if it requires one, and use the same operations and options as with OpenAI. + +[cols="1,2,2",options="header"] +|=== +| Provider | `baseUrl` | Notes + +| OpenAI | `https://api.openai.com/v1` | Default; `baseUrl` can be omitted +| OpenRouter | `https://openrouter.ai/api/v1` | Multi-model gateway with provider routing and fallbacks +| Ollama | `http://localhost:11434/v1` | Local LLM server +| LM Studio | `http://localhost:1234/v1` | Local model runner +| vLLM | `http://localhost:8000/v1` | High-throughput, self-hosted serving engine +|=== + +=== Ollama (local) + +Ollama runs models locally and does not require an API key. + +[tabs] +==== +Java:: ++ +[source,java] +---- +from("direct:chat") + .setBody(constant("What is Apache Camel?")) + .to("openai:chat-completion?baseUrl=http://localhost:11434/v1&model=llama3.2";); +---- + +YAML:: ++ +[source,yaml] +---- +- to: + uri: openai:chat-completion + parameters: + baseUrl: http://localhost:11434/v1 + model: llama3.2 + userMessage: What is Apache Camel? +---- +==== + +For local embeddings, use an embedding model such as `nomic-embed-text` (see the *Embedding Models by +Provider* table below). + +=== LM Studio (local) + +LM Studio serves the model currently loaded in the app. Set `model` to the identifier shown in its UI. + +[source,java] +---- +from("direct:chat") + .to("openai:chat-completion?baseUrl=http://localhost:1234/v1&model=local-model";); +---- + +=== vLLM (self-hosted) + +[source,java] +---- +from("direct:chat") + .to("openai:chat-completion?baseUrl=http://localhost:8000/v1"; + + "&model=meta-llama/Llama-3.1-8B-Instruct"); +---- + +If vLLM was started with `--api-key`, pass the same value via the `apiKey` option. + +=== OpenRouter Review Comment: Did you manage to actually test it with openrouter? To be honest, when I opened the jira issue, I never used openrouter, I just supposed it would work :D if you have openrouter, and can do a quick test would be nice, otherwise, no worries, maybe @orpiske can test and review the docs (if needed) in the future ########## components/camel-ai/camel-openai/src/main/docs/openai-component.adoc: ########## @@ -492,6 +492,113 @@ String-valued fields are set directly. Non-string fields (numbers, booleans, obj NOTE: This maps fields from the response message's additional properties (fields not part of the standard schema). Standard response fields like `content`, `role`, and `tool_calls` are not accessible through this option. +== OpenAI-Compatible Providers + +Because the component speaks the OpenAI API, you do not need a separate component to use third-party +gateways or local model servers that expose an OpenAI-compatible API. Point `baseUrl` at the provider, +set `apiKey` if it requires one, and use the same operations and options as with OpenAI. + +[cols="1,2,2",options="header"] +|=== +| Provider | `baseUrl` | Notes + +| OpenAI | `https://api.openai.com/v1` | Default; `baseUrl` can be omitted +| OpenRouter | `https://openrouter.ai/api/v1` | Multi-model gateway with provider routing and fallbacks +| Ollama | `http://localhost:11434/v1` | Local LLM server +| LM Studio | `http://localhost:1234/v1` | Local model runner +| vLLM | `http://localhost:8000/v1` | High-throughput, self-hosted serving engine +|=== + +=== Ollama (local) + +Ollama runs models locally and does not require an API key. + +[tabs] +==== +Java:: ++ +[source,java] +---- +from("direct:chat") + .setBody(constant("What is Apache Camel?")) + .to("openai:chat-completion?baseUrl=http://localhost:11434/v1&model=llama3.2";); +---- + +YAML:: ++ +[source,yaml] +---- +- to: + uri: openai:chat-completion + parameters: + baseUrl: http://localhost:11434/v1 + model: llama3.2 + userMessage: What is Apache Camel? +---- +==== + +For local embeddings, use an embedding model such as `nomic-embed-text` (see the *Embedding Models by +Provider* table below). + +=== LM Studio (local) + +LM Studio serves the model currently loaded in the app. Set `model` to the identifier shown in its UI. + +[source,java] +---- +from("direct:chat") + .to("openai:chat-completion?baseUrl=http://localhost:1234/v1&model=local-model";); +---- + +=== vLLM (self-hosted) Review Comment: Can you add the vLLM command to install and run `meta-llama/Llama-3.1-8B-Instruct`, as in the example route? and a link to the vLLM website? For example, locally I am using vllm-mlx a lot on mac os, can you mention this as well? For example, this is how I run it on my mac `vllm-mlx serve mlx-community/Qwen3.6-35B-A3B-4bit --port 8000` ########## components/camel-ai/camel-openai/src/main/docs/openai-component.adoc: ########## @@ -492,6 +492,113 @@ String-valued fields are set directly. Non-string fields (numbers, booleans, obj NOTE: This maps fields from the response message's additional properties (fields not part of the standard schema). Standard response fields like `content`, `role`, and `tool_calls` are not accessible through this option. +== OpenAI-Compatible Providers + +Because the component speaks the OpenAI API, you do not need a separate component to use third-party +gateways or local model servers that expose an OpenAI-compatible API. Point `baseUrl` at the provider, +set `apiKey` if it requires one, and use the same operations and options as with OpenAI. + +[cols="1,2,2",options="header"] +|=== +| Provider | `baseUrl` | Notes + +| OpenAI | `https://api.openai.com/v1` | Default; `baseUrl` can be omitted +| OpenRouter | `https://openrouter.ai/api/v1` | Multi-model gateway with provider routing and fallbacks +| Ollama | `http://localhost:11434/v1` | Local LLM server +| LM Studio | `http://localhost:1234/v1` | Local model runner +| vLLM | `http://localhost:8000/v1` | High-throughput, self-hosted serving engine +|=== + +=== Ollama (local) + +Ollama runs models locally and does not require an API key. + +[tabs] +==== +Java:: ++ +[source,java] +---- +from("direct:chat") + .setBody(constant("What is Apache Camel?")) + .to("openai:chat-completion?baseUrl=http://localhost:11434/v1&model=llama3.2";); +---- + +YAML:: ++ +[source,yaml] +---- +- to: + uri: openai:chat-completion + parameters: + baseUrl: http://localhost:11434/v1 + model: llama3.2 + userMessage: What is Apache Camel? +---- +==== + +For local embeddings, use an embedding model such as `nomic-embed-text` (see the *Embedding Models by +Provider* table below). + +=== LM Studio (local) + +LM Studio serves the model currently loaded in the app. Set `model` to the identifier shown in its UI. Review Comment: Please add a link to LM Studio website/documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
