ai-yang opened a new issue, #733:
URL: https://github.com/apache/rocketmq-dashboard/issues/733

   ## Affected branch
   
   `rocketmq-studio` at `bbf1b7e0cf25a5065ba049b5450cc8155569f710`.
   
   ## Problem
   
   The LLM settings page does not track ownership of asynchronous model-list 
requests. A response started for the previously selected provider can overwrite 
the fallback models for a provider that the user selected later. The model 
loading indicator also stops before the model request finishes.
   
   ## Deterministic reproduction
   
   1. Configure the initial LLM provider as OpenAI and keep the `/llm/models` 
response pending.
   2. Open the LLM settings page and wait until that model request has started.
   3. Select DeepSeek. The page correctly displays its built-in `deepseek-chat` 
/ `deepseek-reasoner` options.
   4. Complete the older OpenAI model request with a model named 
`openai-only-late-model`.
   5. Open the model selector.
   
   The late OpenAI response replaces the DeepSeek options even though DeepSeek 
remains selected.
   
   A second controlled-Promise regression keeps `/llm/models` pending and 
observes that the model selector's loading indicator has already disappeared.
   
   The regression tests are:
   
   ```text
   web/src/pages/studio/__tests__/LlmSettingsAsyncState.test.tsx
   
   LlmSettingsPage async request ownership > does not replace a newly selected 
provider model list with an older response
   LlmSettingsPage async request ownership > keeps the model selector loading 
while the provider model request is pending
   ```
   
   On the unmodified branch, both business assertions failed in each of 5/5 
isolated Vitest processes under Node 20.19.5. The tests use controlled Promises 
only: no timers, network access, fixed sleeps, or random scheduling.
   
   ## Root cause
   
   `fetchModels` starts `getLlmModels()` inside the `getLlmConfig().then(...)` 
callback but does not return or await that inner Promise. Therefore, the outer 
`finally` executes as soon as the configuration request finishes and sets 
`modelsLoading` to `false` while the model request is still pending.
   
   The eventual model response also updates `modelOptions` unconditionally. 
There is no request generation or selected-provider check, and changing the 
provider does not invalidate an older request.
   
   ## Impact
   
   - The model dropdown can show models belonging to a different provider than 
the selected provider.
   - A user can save or test a mismatched provider/model combination.
   - The loading indicator reports completion before the provider model request 
has completed.
   - A response arriving after the page is unmounted can still attempt to 
update component state.
   
   ## Expected behavior
   
   - Only the latest model request that still belongs to the active provider 
may update model options or loading state.
   - Changing providers must invalidate any older model request before applying 
the new provider's fallback options.
   - The loading indicator must remain active until the owned model request 
resolves or rejects.
   - Unmounting the page must invalidate outstanding work.
   
   ## Proposed atomic scope
   
   - Move initial loading out of the render path into an effect with cleanup.
   - Await/return the complete configuration-plus-model Promise chain.
   - Use a request generation (or equivalent ownership token) so stale 
responses and stale `finally` callbacks cannot update state.
   - Add controlled-Promise regression coverage for provider switching and 
loading lifetime.
   
   This does not change the LLM API contract or provider configuration format.
   
   ## Related work
   
   - The expected production change is limited to 
`web/src/pages/studio/LlmSettings.tsx`, with regression coverage in its page 
tests.
   - Merged PR #483 introduced the settings page, and merged PR #651 introduced 
provider model loading and fallback options. PR #651 does not invalidate stale 
requests or await the inner model request.
   - Merged PR #536 concerns API-key masking across provider changes, not 
asynchronous model-request ownership.
   - Closed issue #545 tracks the backend LLM gateway and is not a frontend 
request-ordering report.
   - Searches across open and closed issues, plus open/closed/merged pull 
requests, covered `LlmSettings`, `LlmSettings.tsx`, `getLlmModels`, provider 
switching, model-list loading, fallback models, out-of-order responses, and 
stale model responses. No equivalent report, fix, or claimed work was found.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to