This is an automated email from the ASF dual-hosted git repository.
maxgekk pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new db121c34dbe [SPARK-39184][SQL][FOLLOWUP] Make interpreted and codegen
paths for date/timestamp sequences the same
db121c34dbe is described below
commit db121c34dbe22e809c42ebd72618c1597932ac29
Author: Bruce Robbins <bersprock...@gmail.com>
AuthorDate: Mon Aug 22 19:17:26 2022 +0300
[SPARK-39184][SQL][FOLLOWUP] Make interpreted and codegen paths for
date/timestamp sequences the same
### What changes were proposed in this pull request?
Change how the length of the new result array is calculated in
`InternalSequenceBase.eval` to match how the same is calculated in the
generated code.
### Why are the changes needed?
This change brings the interpreted mode code in line with the generated
code. Although I am not aware of any case where the current interpreted mode
code fails, the generated code is more correct (it handles the case where the
result array must grow more than once, whereas the current interpreted mode
code does not).
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing unit tests.
Closes #37542 from bersprockets/date_sequence_array_size_issue_follow_up.
Authored-by: Bruce Robbins <bersprock...@gmail.com>
Signed-off-by: Max Gekk <max.g...@gmail.com>
(cherry picked from commit d718867a16754c62cb8c30a750485f4856481efc)
Signed-off-by: Max Gekk <max.g...@gmail.com>
---
.../apache/spark/sql/catalyst/expressions/collectionOperations.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
index 3786c1a33bc..8186d006296 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
@@ -3070,7 +3070,7 @@ object Sequence {
// "spring forward" without a corresponding "fall back", make a copy
// that's larger by 1
if (i == arr.length) {
- arr = arr.padTo(estimatedArrayLength + 1, fromLong(0L))
+ arr = arr.padTo(i + 1, fromLong(0L))
}
arr(i) = fromLong(result)
i += 1
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
