GraphFrames 0.5.0 - critical bug fix + other improvements

Hi Spark community, I'd like to announce a new release of GraphFrames, a Spark Package for DataFrame-based graphs! *We strongly encourage all users to use this latest release for the bug fix described below.* *Critical bug fix* This release fixes a bug in indexing vertices. This may have

Re: [build system] jenkins got itself wedged...

last update of the week: things are looking great... we're GCing happily and staying well within our memory limits. i'm going to do one more restart after the two pull request builds finish to re-enable backups, and call it a weekend. :) shane On Fri, May 19, 2017 at 8:29 AM, shane knapp

Re: [VOTE] Apache Spark 2.2.0 (RC2)

All the outstanding ML QA doc and user guide items are done for 2.2 so from that side we should be good to cut another RC :) On Thu, 18 May 2017 at 00:18 Russell Spitzer wrote: > Seeing an issue with the DataScanExec and some of our integration tests > for the SCC.

Re: [Spark SQL] ceil and floor functions on doubles

Hi Dongjoon, yeah, it seems to be the same. So, was it done on purpose to match the behavior of Hive? Best regards, Anton 2017-05-19 16:39 GMT+02:00 Dong Joon Hyun : > Hi, Anton. > > > > It’s the same result with Hive, isn’t it? > > > > hive> select 9.223372036854786E20,

Re: [Spark SQL] ceil and floor functions on doubles

Hi, Anton. It’s the same result with Hive, isn’t it? hive> select 9.223372036854786E20, ceil(9.223372036854786E20); OK _c0 _c1 9.223372036854786E20 9223372036854775807 Time taken: 2.041 seconds, Fetched: 1 row(s) Bests, Dongjoon. From: Anton Okolnychyi

[Spark SQL] ceil and floor functions on doubles

Hi all, I am wondering why the results of ceil and floor functions on doubles are internally casted to longs. This causes loss of precision since doubles can hold bigger numbers. Consider the following example: // 9.223372036854786E20 is greater than Long.MaxValue val df =