Github user tanejagagan commented on a diff in the pull request:
https://github.com/apache/spark/pull/16497#discussion_r95307232
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala
---
@@ -51,16 +51,31 @@ import org.apache.spark.util.collection.OpenHashMap
_FUNC_(col, array(percentage1 [, percentage2]...)) - Returns the
exact percentile value array
of numeric column `col` at the given percentage(s). Each value of
the percentage array must
be between 0.0 and 1.0.
- """)
+
+ _FUNC_(col, frequency, percentage) - Returns the exact percentile
value of numeric
+ column `col` with frequency column `frequency` at the given
percentage. The value of
+ percentage must be between 0.0 and 1.0.
+
+ _FUNC_(col, frequency, array(percentage1 [, percentage2]...)) -
Returns the exact percentile
+ value array of numeric column `col` with frequency column
`frequency` at the given
+ percentage(s).Each value of the percentage array must be between 0.0
and 1.0.
+
+ """)
case class Percentile(
child: Expression,
+ frequency : Expression,
percentageExpression: Expression,
+ withFrqExpr : Boolean,
--- End diff --
Please see my comment below why we need to make a distinction either using
flag or Option
I am inclined towards using a flag because switching to option would change
the code in update
from
val frqValue = frequency.eval(input)
to
val frqValue = frequency.getOrElse( unit).eval(input)
But i think Option[Expression] would be better logically
Once we have an agreement if we need to have a distinction or not i will
make the changes accordingly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
