Copilot commented on code in PR #12897: URL: https://github.com/apache/apisix/pull/12897#discussion_r2697431771
########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +104,156 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. + Now send a request: ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "level of detail", - "topic": "psychology", - "level": "brief" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + "created": 1723194057, + "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x", + "model": "gpt-4o-2024-05-13", + "object": "chat.completion", + "system_fingerprint": "fp_abc28019ad", + "usage": { + "completion_tokens": 234, + "prompt_tokens": 18, + "total_tokens": 252 + } +} +``` + +### Configure Multiple Templates + +The following example demonstrates how you can configure multiple templates on the same Route. When requesting the Route, users will be able to pass custom inputs to different templates by specifying the template name. + +The example continues with the [last example](#configure-a-template-for-open-questions-in-custom-complexity). Update the Plugin with another template: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "uri": "/v1/chat/completions", + "plugins": { + "ai-prompt-template": { + "templates": [ + { + "name": "QnA with complexity", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "Answer in {{complexity}}." + }, + { + "role": "user", + "content": "Explain {{prompt}}." + } + ] + } + }, + { + "name": "echo", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "You are an echo bot. You must repeat exactly what the user says without any changes or additional text." + }, + { + "role": "user", + "content": "Echo {{prompt}}." + } + ] + } + } + ] + } + } + }' +``` + +You should now be able to use both templates through the same Route. + +Send a POST request to the Route and use the first template: + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ Review Comment: Missing line continuation backslash at the end of this line. The curl command spans multiple lines, so this line should end with a backslash to continue to the next line. ########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +104,156 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. + Now send a request: ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "level of detail", - "topic": "psychology", - "level": "brief" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + "created": 1723194057, + "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x", + "model": "gpt-4o-2024-05-13", + "object": "chat.completion", + "system_fingerprint": "fp_abc28019ad", + "usage": { + "completion_tokens": 234, + "prompt_tokens": 18, + "total_tokens": 252 + } +} +``` + +### Configure Multiple Templates + +The following example demonstrates how you can configure multiple templates on the same Route. When requesting the Route, users will be able to pass custom inputs to different templates by specifying the template name. + +The example continues with the [last example](#configure-a-template-for-open-questions-in-custom-complexity). Update the Plugin with another template: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "uri": "/v1/chat/completions", + "plugins": { + "ai-prompt-template": { + "templates": [ + { + "name": "QnA with complexity", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "Answer in {{complexity}}." + }, + { + "role": "user", + "content": "Explain {{prompt}}." + } + ] + } + }, + { + "name": "echo", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "You are an echo bot. You must repeat exactly what the user says without any changes or additional text." + }, + { + "role": "user", + "content": "Echo {{prompt}}." + } + ] + } + } + ] + } + } + }' +``` + +You should now be able to use both templates through the same Route. + +Send a POST request to the Route and use the first template: + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + ... +} +``` + +Send a POST request to the Route and use the second template: + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ Review Comment: Missing line continuation backslash at the end of this line. The curl command spans multiple lines, so this line should end with a backslash to continue to the next line. ########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +104,156 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. + Now send a request: ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "level of detail", - "topic": "psychology", - "level": "brief" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" Review Comment: Missing line continuation backslash at the end of this line. The curl command spans multiple lines, so this line should end with a backslash to continue to the next line. ```suggestion -H "Content-Type: application/json" \ ``` ########## docs/zh/latest/plugins/ai-prompt-template.md: ########## @@ -27,48 +27,166 @@ description: 本文档包含有关 Apache APISIX ai-prompt-template 插件的信 # --> +<head> + <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-template"; /> +</head> + ## 描述 -`ai-prompt-template` 插件通过使用模板预定义请求格式，简化了对 LLM 提供商（如 OpenAI 和 Anthropic）及其模型的访问，只允许用户将自定义值传递到模板变量中。 +`ai-prompt-template` 插件简化了对 OpenAI、Anthropic 等大语言模型提供商及其模型的访问。它预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用“填空”的方式。 ## 插件属性 -| **字段** | **必选项** | **类型** | **描述** | -| ------------------------------------- | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------- | -| `templates` | 是 | Array | 模板对象数组 | -| `templates.name` | 是 | String | 模板的名称。 | -| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。有关更多可用模型，请参阅您的 LLM 提供商 API 文档。 | -| `templates.template.messages.role` | 是 | String | 消息的角色（`system`、`user`、`assistant`） | -| `templates.template.messages.content` | 是 | String | 消息的内容。 | +| **字段** | **是否必填** | **类型** | **描述** | +| :--- | :--- | :--- | :--- | +| `templates` | 是 | Array | 模板对象数组。 | +| `templates.name` | 是 | String | 模板的名称。在请求路由时，请求中应包含与所配置模板相对应的模板名称。 | +| `templates.template` | 是 | Object | 模板规范。 | +| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。更多可用模型请参阅 LLM 提供商的 API 文档。 | +| `templates.template.messages` | 是 | Array | 模板消息规范。 | +| `templates.template.messages.role` | 是 | String | 消息的角色，例如 `system`、`user` 或 `assistant`。 | +| `templates.template.messages.content` | 是 | String | 消息（提示词）的内容。 | ## 使用示例 -创建一个带有 `ai-prompt-template` 插件的路由，如下所示： +以下示例将使用 OpenAI 作为上游服务提供商。开始之前，请先创建一个 OpenAI 账户 和一个 API 密钥。你可以选择将密钥保存到环境变量中，如下所示： Review Comment: There is an extra space between "账户" and "和一个". The text should read "OpenAI 账户和一个 API 密钥" without the extra space. ```suggestion 以下示例将使用 OpenAI 作为上游服务提供商。开始之前，请先创建一个 OpenAI 账户和一个 API 密钥。你可以选择将密钥保存到环境变量中，如下所示： ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
