Sxnan opened a new issue, #781: URL: https://github.com/apache/flink-agents/issues/781
### Search before asking - [x] I searched in the [issues](https://github.com/apache/flink-agents/issues) and found nothing similar. ### Description Found during YAML doc verification (#742) while running an end-to-end agent with a real LLM. `SafeFormatter` in `python/flink_agents/api/prompts/utils.py:22-41` uses the regex `\{([^{}]+)\}` to substitute `{key}` placeholders, and offers no escape syntax for emitting a literal `{` or `}`: - `{name}` → substituted with the value of `name` - `{{`, `}}`, `{{name}}` → all preserved as-is in the output Python f-strings and `str.format` use `{{` and `}}` for literal `{` and `}`. A user porting prompt content from f-string-shaped examples to a Flink Agents prompt template will reasonably double the braces in JSON examples to "escape" them — and find that the LLM sees literal `{{` and `}}` in the prompt, then sometimes mimics the doubled-brace form in its response, breaking downstream JSON parsing. This was hit during end-to-end YAML verification: a system prompt with double-brace JSON examples occasionally produced LLM responses with `{{ ... }}` instead of `{ ... }`, which then failed `json.loads` in the downstream action and silently dropped the output (related: #780). The framework's behavior is reasonable from an implementation perspective, but it is undocumented and diverges from a common convention. ### Suggested fix Pick one: 1. **Document only** — add a short note to `docs/content/docs/development/prompts.md` (and a cross-reference from `yaml.md`'s prompt section) stating: - `{name}` is the only substitution syntax - `{` and `}` characters that are not placeholders pass through verbatim - there is no `{{` / `}}` escape — if you need a literal `{` or `}`, write it as-is - reminder that LLM examples inside the prompt should use single braces 2. **Document + code** — extend `SafeFormatter` (and the Java equivalent in `Prompt.formatMessages`) to recognize `{{` → `{` and `}}` → `}`, matching Python f-string / str.format conventions, then update the doc accordingly. This is a behavior change; consider a deprecation cycle if anyone might depend on `{{` being literal today. ### How to reproduce In a YAML prompt, write a JSON example with doubled braces (a natural copy from Python f-string examples): ```yaml prompts: - name: p messages: - role: system content: | Respond as JSON like: {{ "result": "ok" }} - role: user content: "{input}" ``` The rendered prompt the LLM sees contains literal `{{` and `}}`. The LLM may then mimic the doubled-brace form in its response. ### Version and environment Flink Agents 0.3.0 (`main`). Python `SafeFormatter`; the Java equivalent `Prompt.formatMessages` should be checked for parity. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
