LuciferYang opened a new pull request, #56897:
URL: https://github.com/apache/spark/pull/56897

   ### What changes were proposed in this pull request?
   
   This PR fixes the class-level Javadoc of 
`org.apache.spark.util.sketch.CountMinSketch`, which assigned the two table 
dimensions to the wrong symbols. The doc previously read:
   
   ```
   d = ceil(2 / eps)
   w = ceil(-log(1 - confidence) / log(2))
   ```
   
   but the implementation (`CountMinSketchImpl`, the `(eps, confidence, seed)` 
constructor) actually sets:
   
   ```java
   this.width = (int) Math.ceil(2 / eps);
   this.depth = (int) Math.ceil(-Math.log1p(-confidence) / Math.log(2));
   ```
   
   i.e. `width` is derived from `eps` and `depth` from `confidence` -- the 
opposite of what the doc stated. The fix swaps the two lines so the doc matches 
the code:
   
   ```
   w = ceil(2 / eps)
   d = ceil(-log(1 - confidence) / log(2))
   ```
   
   ### Why are the changes needed?
   
   The documented formulas were swapped relative to the implementation, and 
also relative to the standard Count-Min sketch definition and the stream-lib 
implementation this class is based on (depth = number of hash rows, driven by 
`confidence`; width = buckets per row, driven by `eps`). For example, with `eps 
= 0.01, confidence = 0.9` the code produces `width = 200, depth = 4`, which the 
old doc described backwards. This is a doc-only correctness fix to avoid 
misleading readers.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. Documentation-only change; no behavior, serialization, or API impact.
   
   ### How was this patch tested?
   
   No tests needed -- this is a Javadoc-only change. Verified by reading the 
formulas against `CountMinSketchImpl`'s constructors.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to