andygrove opened a new pull request, #6343: URL: https://github.com/apache/datafusion-comet/pull/6343
## Which issue does this PR close? No issue; this is documentation only. ## Rationale for this change The memory tuning page had two answers to how much memory Comet needs. The older one, "Determining How Much Memory to Allocate", was added in March 2025, before Comet 0.8.0 was released. Its advice rests on TPC-H timings from a single executor at that time, and many memory bugs have been fixed since. The newer one, "Sizing the Overhead from the Memory Usage Log", uses the memory usage log added in 1.1.0 to measure what Comet actually uses on the user's own workload. The page also explained untracked memory twice: once under Configuring Comet Memory, and again at the start of Configuring Executor Memory Overhead. ## What changes are included in this PR? - Removes the "Determining How Much Memory to Allocate" section and the intro paragraph that pointed to it. The intro now points to the memory usage log section instead. - Keeps the old section's one general point as a short paragraph under Configuring Comet Memory: a larger pool lets operators that can spill keep more in memory, and an operator that cannot spill fails the task when it cannot reserve the memory it needs. Nothing else on the page said why the pool's size matters. - Condenses the three opening paragraphs of Configuring Executor Memory Overhead into one, since they repeated the untracked-memory list from the section above. The paragraph keeps the two points specific to the overhead: the container already has room for `spark.memory.offHeap.size`, and the JVM's own non-heap memory draws on the same overhead. The headings that other pages link to are unchanged. The migration guide links to three of them, and the contributor guide links to the memory usage log section. Nothing linked to the removed section. ## How are these changes tested? Documentation only, no code paths touched. I did not build the site. A script checked that every reference link and heading anchor on the page resolves, and a search of the repository found no remaining links to the removed section. `npx prettier@latest --check` passes on the page. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
