devmadhuu commented on code in PR #10808:
URL: https://github.com/apache/ozone/pull/10808#discussion_r3654854338
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/chatbot/agent/ChatbotAgent.java:
##########
@@ -251,23 +285,31 @@ public String processQuery(String userQuery, String
model, String provider)
/**
* "Step 1" Helper: Talks to the LLM and asks for a JSON object telling us
which API to call.
*/
- private ToolSelection chooseToolsForQuery(String userQuery, String model,
- String provider) throws LLMClient.LLMException,
IOException {
+ private ToolSelection chooseToolsForQuery(String userQuery, String model,
String provider,
+ String historyContext)
+ throws LLMClient.LLMException, IOException {
// --- 1. BUILD THE PROMPT ---
// The system prompt teaches the LLM the Recon API schema and the rules
for picking a tool.
- // The user prompt is just the raw question the user typed.
+ // The user prompt is the current question, optionally preceded by a
fenced,
+ // trimmed conversation-history block (built once by the caller) so the
model
+ // can resolve references.
String systemPrompt = buildToolSelectionPrompt();
- String userPrompt = "User Query: " + userQuery;
+ String userPrompt;
+ if (historyContext.isEmpty()) {
+ userPrompt = "User Query: " + userQuery;
+ } else {
+ userPrompt = historyContext
+ + "\n## CURRENT QUESTION (answer THIS):\n" + userQuery;
+ }
List<ChatMessage> messages = new ArrayList<>();
messages.add(new ChatMessage("system", systemPrompt));
messages.add(new ChatMessage("user", userPrompt));
// --- 2. CONFIGURE GENERATION SETTINGS ---
// Temperature 0.1: very low creativity — we want strict, deterministic
tool selection.
- // max_tokens 8192: allow a large enough reply to fit all tool
descriptions.
- GenParams params = new GenParams(0.1, 8192);
+ GenParams params = new GenParams(0.1, MAX_TOKENS);
Review Comment:
This params applied when model is built right ? So `MAX_TOKENS` is sent to
provider. Default value now increased from `8192` to `MAX_TOKENS`. Is this
configurable ? Because Claude Sonnet-class models default to 8192 max output
tokens. going above requires an output-extending beta header. The configured
anthropic.beta.header is context-1m-2025-08-07 — that extends input context,
not output. So max_tokens=16384 can be rejected by the Anthropic API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
