Re: [PR] feat(ai-rag): support multiple embedding providers, add Cohere rerank, and standardize chat interface [apisix]
ChuanFF commented on PR #12941: URL: https://github.com/apache/apisix/pull/12941#issuecomment-4095311214 @moonming Thanks for the suggestion. 1. I will split the pull request into three parts: one that follows a standardized interface, supports multiple embedding providers, and includes Cohere. 2. The embedding implementation needs to implement the `get_embeddings` interface (this is consistent with the original plugin logic). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(ai-rag): support multiple embedding providers, add Cohere rerank, and standardize chat interface [apisix]
ChuanFF commented on PR #12941: URL: https://github.com/apache/apisix/pull/12941#issuecomment-3977373845 @Baoyuantop I've addressed the feedback from Copilot in this PR. Could you please take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(ai-rag): support multiple embedding providers, add Cohere rerank, and standardize chat interface [apisix]
Copilot commented on code in PR #12941:
URL: https://github.com/apache/apisix/pull/12941#discussion_r2802885516
##
apisix/plugins/ai-rag.lua:
##
@@ -14,65 +14,113 @@
-- See the License for the specific language governing permissions and
-- limitations under the License.
--
-local next= next
-local require = require
-local ngx_req = ngx.req
+local next = next
+local require = require
+local ngx_req = ngx.req
+local table= table
+local ipairs = ipairs
+local pcall= pcall
+local tostring = tostring
-local http = require("resty.http")
local core = require("apisix.core")
-local azure_openai_embeddings =
require("apisix.plugins.ai-rag.embeddings.azure_openai").schema
-local azure_ai_search_schema =
require("apisix.plugins.ai-rag.vector-search.azure_ai_search").schema
+local openai_base_embeddings_schema =
require("apisix.plugins.ai-rag.embeddings.openai-base").schema
+local azure_ai_search_schema =
require("apisix.plugins.ai-rag.vector-search.azure-ai-search").schema
+local cohere_rerank_schema =
require("apisix.plugins.ai-rag.rerank.cohere").schema
local HTTP_INTERNAL_SERVER_ERROR = ngx.HTTP_INTERNAL_SERVER_ERROR
local HTTP_BAD_REQUEST = ngx.HTTP_BAD_REQUEST
+local embeddings_drivers = {}
+local vector_search_drivers = {}
+local rerank_drivers = {}
+
+local plugin_name = "ai-rag"
+
+local input_strategy_enum = {
+last = "last",
+all = "all"
+}
+
local schema = {
type = "object",
properties = {
-type = "object",
embeddings_provider = {
type = "object",
-properties = {
-azure_openai = azure_openai_embeddings
+oneOf = {
+{
+properties = {
+openai = openai_base_embeddings_schema
+},
+required = { "openai" },
+additionalProperties = false
+},
+{
+properties = {
+["azure-openai"] = openai_base_embeddings_schema
+},
+required = { "azure-openai" },
+additionalProperties = false
+},
+{
+properties = {
+["openai-compatible"] = openai_base_embeddings_schema
+},
+required = { "openai-compatible" },
+additionalProperties = false
+}
},
--- ensure only one provider can be configured while implementing
support for
--- other providers
-required = { "azure_openai" },
-maxProperties = 1,
+description = "Configuration for the embeddings provider."
},
vector_search_provider = {
type = "object",
-properties = {
-azure_ai_search = azure_ai_search_schema
+oneOf = {
+{
+properties = {
+["azure-ai-search"] = azure_ai_search_schema
+},
+required = { "azure-ai-search" },
+additionalProperties = false
+}
},
--- ensure only one provider can be configured while implementing
support for
--- other providers
-required = { "azure_ai_search" },
-maxProperties = 1
+description = "Configuration for the vector search provider."
},
-},
-required = { "embeddings_provider", "vector_search_provider" }
-}
-
-local request_schema = {
-type = "object",
-properties = {
-ai_rag = {
+rerank_provider = {
+type = "object",
+oneOf = {
+{
+properties = {
+cohere = cohere_rerank_schema
+},
+required = { "cohere" },
+additionalProperties = false
+}
+},
+description = "Configuration for the rerank provider."
+},
+rag_config = {
type = "object",
properties = {
-vector_search = {},
-embeddings = {},
+input_strategy = {
+type = "string",
+enum = { input_strategy_enum.last, input_strategy_enum.all
},
+default = input_strategy_enum.last,
+description = "Strategy for extracting input text from
messages."
+.. "'last' uses the last user message"
Review Comment:
The description for `input_strategy` is missing spaces between sentences. It
should have spaces after the periods to improve readability.
```suggestion
description = "Strategy for extracting input text from
messages. "
.. "'last' uses th
Re: [PR] feat(ai-rag): support multiple embedding providers, add Cohere rerank, and standardize chat interface [apisix]
ChuanFF commented on PR #12941: URL: https://github.com/apache/apisix/pull/12941#issuecomment-3832744063 > Hi @ChuanFF, could you explain these breaking changes? Is it necessary to introduce these changes? @Baoyuantop 1. **Request format**：The previous plugin required the `ai_rag` field to be included in the request, which is not standard practice. Ideally, the document retrieval should be performed on the user's question (usually the last question). This approach is directly compatible with the `openai-api`'s completions interface. We can refer to other AI proxy projects like Higress and Literm for this. 2. **Azure OpenAI key**： `azure_openai` was changed to `azure-openai` to maintain consistency with the `ai-proxy` plugin's fields. Please let me know if prefer to keep it as `azure_openai`. 3. **Context position**：The `ai-rag` information should be inserted before the user's question to keep the LLM focused on the user's question. Otherwise, in some scenarios, the LLM might treat inserted documents as user questions, reducing the quality of the LLM's response. We can refer to other AI proxy projects for this as well. 4. **Vector search output**：The previous plugin, when using `azure-ai-search` for document retrieval, neither filtered the fields nor parsed the response body to obtain the document content. This resulted in the `rag` results passed to the LLM containing a large amount of information unrelated to the retrieved documents. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat(ai-rag): support multiple embedding providers, add Cohere rerank, and standardize chat interface [apisix]
Baoyuantop commented on PR #12941: URL: https://github.com/apache/apisix/pull/12941#issuecomment-3832549177 Hi @ChuanFF, could you explain these breaking changes? Is it necessary to introduce these changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
