Assertive-xxh opened a new issue, #1387:
URL: https://github.com/apache/flink-agents/issues/1387

   ### Search before asking
   
   - [x] I searched in the 
[issues](https://github.com/apache/flink-agents/issues) and found nothing 
similar.
   
   ### Description
   
   #1366 made Java `loadYaml` check registration conflicts before it changes 
the environment. Python `load_yaml` already has a pre-check, but it only looks 
for a duplicate name within the same resource type. The rule that a chat model 
and a model router cannot share a name is checked later, inside 
`env.add_resource`, while the file is being committed. When that check fails, 
the resources committed before it stay registered.
   
   I hit two cases:
   
   1. The environment already has a chat model `assistant`. Loading a file that 
declares a prompt `fresh` and a model router `assistant` fails with `Resource 
'assistant' cannot name both a chat model and a model router.` The prompt 
`fresh` is registered anyway. After renaming the router, loading the same file 
again fails with `Duplicate shared prompt 'fresh'`.
   2. A single file declares a shared chat model and a shared model router with 
the same name. Loading fails with the same message, and the chat model and the 
prompts from that file stay registered.
   
   Java covers case 1 since #1366 
(`sharedChatModelNameClashWithRouterLeavesFailedFileUnregistered`).
   
   Expected behavior: a file that fails on this clash registers nothing, the 
same as in Java.
   
   ### How to reproduce
   
   Case 1, `candidate.yaml`:
   
   ```yaml
   prompts:
     - name: fresh
       text: hello
   model_routers:
     - name: assistant
       clazz: router
       candidates: [small]
       strategy_type: rule_based
   ```
   
   ```python
   env.add_resource(
       "assistant", ResourceType.CHAT_MODEL, 
ResourceDescriptor(clazz="some.Setup")
   )
   
   load_yaml(env, "candidate.yaml")
   # ValueError: Resource 'assistant' cannot name both a chat model and a model 
router.
   
   env.resources[ResourceType.PROMPT]
   # {'fresh': ...}  <- registered despite the failure
   
   # rename the router to "picker" in candidate.yaml, then:
   load_yaml(env, "candidate.yaml")
   # ValueError: Duplicate shared prompt 'fresh' (loading candidate.yaml)
   ```
   
   Case 2, loaded into an empty environment:
   
   ```yaml
   prompts:
     - name: fresh
       text: hello
   chat_model_setups:
     - name: same
       clazz: ollama
   model_routers:
     - name: same
       clazz: router
       candidates: [small]
       strategy_type: rule_based
   ```
   
   Loading fails with the same `ValueError`, and afterwards `env.resources` 
holds the chat model `same` and the prompt `fresh`.
   
   I reproduced both with the mocked environment that `test_loader.py` uses 
(`_registration_env`), without submitting a job.
   
   A possible fix is to call `check_model_name_collision` in the pre-check loop 
of `load_yaml`, once against `env.resources` and once against the file's own 
shared resources. I tried that locally and the existing YAML tests still pass.
   
   ### Version and environment
   
   Flink Agents: main at 656b1c8 (1.0.dev0)
   PyFlink: 2.3.0
   Python: 3.12.3
   OS: Ubuntu 24.04 (WSL2)
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to