Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/16818#discussion_r100049000
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/window/BoundOrdering.scala
---
@@ -25,18 +25,22 @@ import
org.apache.spark.sql.catalyst.expressions.Projection
* Function for comparing boundary values.
*/
private[window] abstract class BoundOrdering {
- def compare(inputRow: InternalRow, inputIndex: Int, outputRow:
InternalRow, outputIndex: Int): Int
+ def compare(
+ inputRow: InternalRow,
+ inputIndex: Long,
+ outputRow: InternalRow,
+ outputIndex: Long): Long
}
/**
* Compare the input index to the bound of the output index.
*/
-private[window] final case class RowBoundOrdering(offset: Int) extends
BoundOrdering {
+private[window] final case class RowBoundOrdering(offset: Long) extends
BoundOrdering {
--- End diff --
@hvanhovell `rowBuffer` is used to buffer all the rows of one partition. As
you said, we only support 32-bit values (less than `(1 << 31) - 1`). But the
row frame or range frame is just to restrict the range to fetch proper value
from `rowBuffer`. This PR changes the type of offset from `Int` to `Long`. It
will make the `rowBuffer` overflows. Besides, how to support a 64-bit buffer is
another topic. Let me know if I understand the wrong. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
