shreemaan-abhishek commented on code in PR #14139:
URL: https://github.com/apache/apisix/pull/14139#discussion_r4465255356
##########
apisix/plugins/ai-prompt-template.lua:
##########
@@ -136,17 +140,13 @@ function _M.rewrite(conf, ctx)
return 400, { message = "template: " .. template_name .. " not
configured." }
end
- local template_json = templates_json_lrucache(template, template,
core.json.encode, template)
- core.log.info("sending template to body_transformer, name: ",
template_name)
- return body_transformer.rewrite(
- {
- request = {
- template = template_json,
- input_format = "json"
- }
- },
- ctx
- )
+ local rendered, render_err = render_template(template, body_tab)
+ if not rendered then
+ return 503, {message = render_err}
+ end
+
+ body.replace(body_tab, rendered)
Review Comment:
Confirmed that the protocol-shape problem predates this PR: the base
ai-prompt-template implementation also renders and replaces the request with
the configured Chat Completions-shaped template regardless of the endpoint
protocol. I documented the expected protocol contract and acceptance criteria
in #14155. We will address that separately; this PR should stop claiming that
ai-prompt-template is protocol-aligned and should focus on not introducing
additional regressions.
##########
apisix/plugins/ai-prompt-template.lua:
##########
@@ -88,43 +72,63 @@ local templates_lrucache = core.lrucache.new({
ttl = 300, count = 256
})
-local templates_json_lrucache = core.lrucache.new({
- ttl = 300, count = 256
-})
-
function _M.check_schema(conf)
return core.schema.check(schema, conf)
end
-local function get_request_body_table(max_size)
- local body, err = core.request.get_body(max_size)
- if not body then
- return nil, { message = "could not get body: " .. (err or "request
body is empty") }
+local function find_template(conf, template_name)
+ for _, template in ipairs(conf.templates) do
+ if template.name == template_name then
+ return template.template
+ end
end
+ return nil
+end
- local body_tab, err = core.json.decode(body)
- if not body_tab then
- return nil, { message = "could not parse JSON request body: " .. (err
or "invalid JSON") }
+
+local function render_string(value, input)
+ local raw_template = string_gsub(value, "{{(.-)}}", "{*%1*}")
+ local ok, render = pcall(template.compile, raw_template)
+ if not ok then
+ return nil, "template compile: " .. render
end
- return body_tab
+ local output
+ ok, output = pcall(render, input)
Review Comment:
Confirmed as pre-existing. The base body-transformer path also evaluates the
decoded request table as the resty.template context, so request keys can shadow
runtime globals and false or missing values have surprising lookup behavior.
This needs an explicit compatibility and variable-resolution design rather than
an incidental change in this PR. Tracked for later work in #14155.
##########
apisix/plugins/ai-prompt-template.lua:
##########
@@ -88,43 +72,63 @@ local templates_lrucache = core.lrucache.new({
ttl = 300, count = 256
})
-local templates_json_lrucache = core.lrucache.new({
- ttl = 300, count = 256
-})
-
function _M.check_schema(conf)
return core.schema.check(schema, conf)
end
-local function get_request_body_table(max_size)
- local body, err = core.request.get_body(max_size)
- if not body then
- return nil, { message = "could not get body: " .. (err or "request
body is empty") }
+local function find_template(conf, template_name)
+ for _, template in ipairs(conf.templates) do
+ if template.name == template_name then
+ return template.template
+ end
end
+ return nil
+end
- local body_tab, err = core.json.decode(body)
- if not body_tab then
- return nil, { message = "could not parse JSON request body: " .. (err
or "invalid JSON") }
+
+local function render_string(value, input)
+ local raw_template = string_gsub(value, "{{(.-)}}", "{*%1*}")
+ local ok, render = pcall(template.compile, raw_template)
+ if not ok then
+ return nil, "template compile: " .. render
end
- return body_tab
+ local output
+ ok, output = pcall(render, input)
+ if not ok then
+ return nil, "template rendering: " .. output
+ end
+ return output
end
-local function find_template(conf, template_name)
- for _, template in ipairs(conf.templates) do
- if template.name == template_name then
- return template.template
+local function render_template(configured, input)
+ local rendered = core.table.deepcopy(configured)
+ local err
+
+ if rendered.model then
+ rendered.model, err = render_string(rendered.model, input)
+ if not rendered.model then
+ return nil, err
end
end
- return nil
+
+ for _, message in ipairs(rendered.messages or {}) do
+ message.content, err = render_string(message.content, input)
+ if not message.content then
+ return nil, err
+ end
+ end
+
+ return rendered
end
+
function _M.rewrite(conf, ctx)
- local body_tab, err = get_request_body_table(conf.max_req_body_size)
+ local body_tab, status, err = body.read(conf.max_req_body_size)
if not body_tab then
- return 400, err
+ return status, err
end
local template_name = body_tab.template_name
Review Comment:
Confirmed that these malformed-shape failures predate this PR. On the base
branch, ai-prompt-template indexes non-object decoded JSON and concatenates a
non-string template_name, while the Anthropic decorator path can iterate a
non-table messages value. I documented the validation requirements and
sanitized 4xx acceptance criteria in #14155. We will fix those separately; the
newly introduced cache and rendering regressions remain in scope for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
