This is an automated email from the ASF dual-hosted git repository.
LuciferYang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9bef20cc4207 [MINOR][SQL] Fix incorrect scaladoc in functions.scala
9bef20cc4207 is described below
commit 9bef20cc42072ae8661210daf71fce7857e28838
Author: YangJie <[email protected]>
AuthorDate: Mon Jul 20 13:27:39 2026 +0800
[MINOR][SQL] Fix incorrect scaladoc in functions.scala
### What changes were proposed in this pull request?
Corrects copy-paste scaladoc defects in `sql/api`'s `functions.scala` where
the doc text contradicts the function signature or the function's actual
semantics:
- `regr_avgy`: "average of the independent variable" -> "dependent
variable" (it averages `y`, which the same sentence defines as the dependent
variable; the summary was pasted from `regr_avgx`).
- `any_value(e, ignoreNulls)`: the description referenced `isIgnoreNull`,
which is not a parameter -> `ignoreNulls`.
- `ceil(columnName)`: referenced `` `e` `` (the parameter of the `ceil(e:
Column)` overload) -> `columnName`; matches the corrected `floor(columnName)`
sibling.
- `bitmap_bit_position` / `bitmap_bucket_number`: the two summaries were
transposed; each is now aligned with its catalyst expression
(`BitmapBitPosition` / `BitmapBucketNumber`).
- `datediff` / `date_diff`: the examples called `dateddiff(...)` (a
nonexistent function) -> the actual function name.
- `to_timestamp_ltz` (both overloads): "to a timestamp without time zone"
-> "with local time zone". The builder injects `TimestampType` (TIMESTAMP WITH
LOCAL TIME ZONE); the wording was pasted from the adjacent `to_timestamp_ntz`
docs.
- `reduce` (both overloads): the examples called `aggregate(...)` ->
`reduce(...)`.
### Why are the changes needed?
The scaladoc misdescribes these functions. Most are harmless wrong names in
examples, but `to_timestamp_ltz` documenting "without time zone" actively
misleads about the return type (it produces a timestamp with local time zone),
and the transposed `bitmap_*` summaries point users to the wrong function.
### Does this PR introduce _any_ user-facing change?
No. Documentation-only change to scaladoc comments.
### How was this patch tested?
No tests; scaladoc-only change. Each correction was verified against the
function signature and, where semantics were involved (`to_timestamp_ltz`,
`bitmap_*`), against the corresponding catalyst expression builder/definition.
Checked the changed lines stay within 100 chars and introduce no non-ASCII
characters.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #57334 from LuciferYang/SPARK-functions-scaladoc.
Authored-by: YangJie <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.../scala/org/apache/spark/sql/functions.scala | 25 +++++++++++-----------
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
b/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
index f602c64a6248..ba400a2c8201 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
@@ -2318,7 +2318,7 @@ object functions {
def regr_avgx(y: Column, x: Column): Column = Column.fn("regr_avgx", y, x)
/**
- * Aggregate function: returns the average of the independent variable for
non-null pairs in a
+ * Aggregate function: returns the average of the dependent variable for
non-null pairs in a
* group, where `y` is the dependent variable and `x` is the independent
variable.
*
* @group agg_funcs
@@ -2399,7 +2399,7 @@ object functions {
def any_value(e: Column): Column = Column.fn("any_value", e)
/**
- * Aggregate function: returns some value of `e` for a group of rows. If
`isIgnoreNull` is true,
+ * Aggregate function: returns some value of `e` for a group of rows. If
`ignoreNulls` is true,
* returns only non-null values.
*
* @group agg_funcs
@@ -3597,7 +3597,7 @@ object functions {
def ceil(e: Column): Column = Column.fn("ceil", e)
/**
- * Computes the ceiling of the given value of `e` to 0 decimal places.
+ * Computes the ceiling of the given value of `columnName` to 0 decimal
places.
*
* @group math_funcs
* @since 1.4.0
@@ -4958,7 +4958,7 @@ object functions {
def random(): Column = random(lit(SparkClassUtils.random.nextLong))
/**
- * Returns the bucket number for the given input column.
+ * Returns the bit position for the given input column.
*
* @group misc_funcs
* @since 3.5.0
@@ -4967,7 +4967,7 @@ object functions {
Column.fn("bitmap_bit_position", col)
/**
- * Returns the bit position for the given input column.
+ * Returns the bucket number for the given input column.
*
* @group misc_funcs
* @since 3.5.0
@@ -7684,7 +7684,7 @@ object functions {
*
* Only considers the date part of the input. For example:
* {{{
- * dateddiff("2018-01-10 00:00:00", "2018-01-09 23:59:59")
+ * datediff("2018-01-10 00:00:00", "2018-01-09 23:59:59")
* // returns 1
* }}}
*
@@ -7707,7 +7707,7 @@ object functions {
*
* Only considers the date part of the input. For example:
* {{{
- * dateddiff("2018-01-10 00:00:00", "2018-01-09 23:59:59")
+ * date_diff("2018-01-10 00:00:00", "2018-01-09 23:59:59")
* // returns 1
* }}}
*
@@ -8796,7 +8796,7 @@ object functions {
def time_to_micros(e: Column): Column = Column.fn("time_to_micros", e)
/**
- * Parses the `timestamp` expression with the `format` expression to a
timestamp without time
+ * Parses the `timestamp` expression with the `format` expression to a
timestamp with local time
* zone. Returns null with invalid input.
*
* @group datetime_funcs
@@ -8806,8 +8806,9 @@ object functions {
Column.fn("to_timestamp_ltz", timestamp, format)
/**
- * Parses the `timestamp` expression with the default format to a timestamp
without time zone.
- * The default format follows casting rules to a timestamp. Returns null
with invalid input.
+ * Parses the `timestamp` expression with the default format to a timestamp
with local time
+ * zone. The default format follows casting rules to a timestamp. Returns
null with invalid
+ * input.
*
* @group datetime_funcs
* @since 3.5.0
@@ -9291,7 +9292,7 @@ object functions {
* to a single state. The final state is converted into the final result by
applying a finish
* function.
* {{{
- * df.select(aggregate(col("i"), lit(0), (acc, x) => acc + x, _ * 10))
+ * df.select(reduce(col("i"), lit(0), (acc, x) => acc + x, _ * 10))
* }}}
*
* @param expr
@@ -9319,7 +9320,7 @@ object functions {
* Applies a binary operator to an initial state and all elements in the
array, and reduces this
* to a single state.
* {{{
- * df.select(aggregate(col("i"), lit(0), (acc, x) => acc + x))
+ * df.select(reduce(col("i"), lit(0), (acc, x) => acc + x))
* }}}
*
* @param expr
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
