cboumalh commented on code in PR #52551:
URL: https://github.com/apache/spark/pull/52551#discussion_r2430855561
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ThetaSketchUtils.scala:
##########
@@ -36,6 +38,18 @@ object ThetaSketchUtils {
final val MAX_LG_NOM_LONGS = 26
final val DEFAULT_LG_NOM_LONGS = 12
+ /*
+ * QUICKSELECT is optimized for speed and is the default choice for most use
cases,
+ * providing faster updates and queries with slightly higher error rates.
ALPHA offers
+ * better accuracy with slightly higher resource consumption, making it
suitable when
+ * precision is more important than performance. The choice primarily
affects the speed
+ * vs accuracy trade-off.
Review Comment:
nit, but this is not entirely true. consider this:
```
/*
* ALPHA is optimized for speed and offers slightly better initial accuracy
* (lower error) for simple updates in a streaming environment. Its
estimation
* precision reverts to the standard level if merged with other sketches.
* QUICKSELECT is the default and more flexible choice, providing the
standard
* level of accuracy and full support for all set operations (Union,
Intersection, etc.).
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
