adityamparikh commented on PR #196: URL: https://github.com/apache/solr-mcp/pull/196#issuecomment-5685922807
Reduced to one commit. All payload-splitting and format guidance is reverted — the instructions, the tool descriptions and the `index-data` prompt are untouched by this PR. What remains is the commit-mode change, verified against the condition that documents stay searchable. **Why the guidance went away.** It did not work, and the MCP maintainers consider the shape of it an anti-pattern. Tested live: Sonnet 5 in Claude Desktop, an 86 KB markdown paste. With the guidance in place the model did batch correctly — 15 calls of 10 documents, no truncation — and still took over 15 minutes, because splitting does not reduce total output tokens and each extra call re-reads the whole conversation. Meanwhile [discussion #1197](https://github.com/modelcontextprotocol/modelcontextprotocol/discussions/1197) records the consensus as *keep the data plane out of JSON-RPC arguments entirely*, naming "passing entire file content through LLM context" as an anti-pattern, and the [File Uploads WG charter](https://modelcontextprotocol.io/community/working-groups/file-uploads) names "prose instructions asking for base64 strings or local paths" as the problem it exists to remove. Tuning prose was treating a symptom the spec intends to delete; the real answer is the handle -based path ([SEP-2356](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2356), [SEP-2631](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1306)), which is bigger than this PR. **Why the commit change stays.** Both paths hard-committed — `indexDocuments` via `solrClient.commit(collection)`, and #205's CSV/XML `forward()` via `setAction(COMMIT, waitFlush=true, waitSearcher=true)` — so every tool call waited on an fsync. Solr's own `_default` configset does the opposite: `autoCommit` at 15 s with `openSearcher=false` for tlog truncation, `autoSoftCommit` at 3 s for visibility. We were fighting the configset. Measured, 20 interleaved reps, same endpoint and document, only the commit parameter varying: | mode | median | p90 | |---|---|---| | no commit | 4.05 ms | 4.64 ms | | soft commit | 8.61 ms | 10.65 ms | | hard commit | 18.94 ms | 41.32 ms | 2.2x faster, and far tighter — the hard commit's p90 is four times its median, which is fsync variance. Over the MCP tools, 61 single-document CSV calls go from 1853 ms to 585 ms. **Searchability is preserved**, which was the condition: `waitSearcher=true` means documents are visible when the call returns, verified 30/30 with zero delay on both the JSON and CSV paths. **Durability is unchanged** — the transaction log is written on the `add`, before any commit, so documents survive a crash in every commit mode; a hard commit governs how much tlog is replayed on recovery, not whether data is lost. One caveat for the docs: an operator running a custom configset with `autoCommit` disabled should enable it. Still depends on #205 and should merge after it. Build green: 390 tests, 0 failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
