manuzhang commented on code in PR #4775:
URL: https://github.com/apache/datafusion-comet/pull/4775#discussion_r3900863955
##########
native/proto/src/proto/expr.proto:
##########
@@ -263,6 +264,36 @@ message Correlation {
DataType datatype = 4;
}
+// Simple linear regression aggregates (regr_slope, regr_intercept, regr_r2,
+// regr_sxx, regr_syy, regr_sxy). child1 is the dependent variable (y) and
+// child2 is the independent variable (x).
+message Regr {
+ enum RegrType {
+ SLOPE = 0;
+ INTERCEPT = 1;
+ R2 = 2;
+ SXX = 3;
+ SYY = 4;
+ SXY = 5;
+ }
+ Expr child1 = 1;
+ Expr child2 = 2;
+ RegrType regr_type = 3;
+ DataType datatype = 4;
+ // Only consulted for SLOPE and INTERCEPT. When true (Spark 3.5+),
VariancePop(x)
+ // is computed only over rows where both y and x are non-null. When false
+ // (Spark 3.4), VariancePop(x) includes every row where x is non-null even
if y
+ // is null, matching the pre-fix Spark 3.4 semantics.
+ bool filter_var_by_pair_nulls = 5;
+ // Only consulted for R2. Spark swapped the degenerate-case handling of
+ // regr_r2. When true, a constant dependent variable (m2(y) = 0) returns 1.0
+ // and a constant independent variable (m2(x) = 0) returns null. When false
+ // those two cases are reversed. The swap is present in the Spark versions
+ // Comet builds against for 3.5 and later (3.5.9, 4.0.3+, 4.1, 4.2) but not
Review Comment:
The list can become style. This comment can be simplified to
```
// The swap is present in the Spark versions Comet builds against for 3.5+,
but not in 3.4.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
