Julien Champ created SPARK-19451: ------------------------------------ Summary: Long values in Window function Key: SPARK-19451 URL: https://issues.apache.org/jira/browse/SPARK-19451 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 2.0.2, 1.6.1 Reporter: Julien Champ
Hi there, there seems to be a major limitation in spark window functions and rangeBetween method. If I have the following code : ``` val tw = Window.orderBy("date") .partitionBy("id") .rangeBetween( from , 0) ``` Everything seems ok, while "from" value is not too large... Even if the rangeBetween() method supports Long parameters. But.... If i set "-2160000000L" value to "from" it does not work ! It is probably related to this part of code in the between() method, of the WindowSpec class, called by rangeBetween() ``` val boundaryStart = start match { case 0 => CurrentRow case Long.MinValue => UnboundedPreceding case x if x < 0 => ValuePreceding(-start.toInt) case x if x > 0 => ValueFollowing(start.toInt) } ``` ( look at this " .toInt " ) Does anybody know it there's a way to solve / patch this behavior ? Any help will be appreciated Thx -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org