cboumalh commented on code in PR #54338: URL: https://github.com/apache/spark/pull/54338#discussion_r2828310445
########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/tupleDifference.scala: ########## @@ -97,6 +97,78 @@ case class TupleDifferenceInteger(left: Expression, right: Expression) } } +// scalastyle:off line.size.limit +@ExpressionDescription( + usage = """ + _FUNC_(tupleSketch, thetaSketch) - Subtracts the binary representation of a + Datasketches ThetaSketch from a TupleSketch with double summary data type using a TupleSketch + AnotB object. Returns elements in the TupleSketch that are not in the ThetaSketch. """, + examples = """ + Examples: + > SELECT tuple_sketch_estimate_double(_FUNC_(tuple_sketch_agg_double(col1, val1), theta_sketch_agg(col2))) FROM VALUES (5, 5.0D, 4), (1, 1.0D, 4), (2, 2.0D, 5), (3, 3.0D, 1) tab(col1, val1, col2); + 2.0 + """, + group = "sketch_funcs", + since = "4.2.0") +// scalastyle:on line.size.limit +case class TupleDifferenceThetaDouble(left: Expression, right: Expression) Review Comment: The second byte of every sketch represents the type of sketch it is. I added a check in both the theta and tuple sketches deserialization (heapify, wrap) functions with their corresponding errors. This way this error will come up in every function call where the wrong sketch type is passed in, not just the ones defined in this PR. Let me know what you think! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
