Re: Spark data frames integration merged

Hello, Denis. > Nikolay, could you document the feature before the release [1]? Yes, I can. I will document these feature in a next few days. В Пт, 29/12/2017 в 13:37 -0800, Denis Magda пишет: > Great news, > > Thanks Nikolay and Val! > > Nikolay, could you document the feature before the

Re: Spark data frames integration merged

s, daily. > > We will keep in touch and thanks again for all the great work and help > everyone. > > Revin > > From: Denis Magda <dma...@apache.org> > Date: Thursday, January 4, 2018 at 12:34 PM > To: Revin Chalil <rcha...@expedia.com> > Cc: &qu

Re: Spark data frames integration merged

Hello, guys. Currently `getPreferredLocations` implemented in `IgniteRDD -> IgniteAbstractRDD`. But DataFrame implementation uses `IgniteSQLDataFrameRDD -> IgniteSqlRDD -> IgniteAbstractRDD` Where `->` is extension. So, for now, getPreferredLocation doesn't implemented for a IgniteDataFrame.

Re: Spark data frames integration merged

. Revin From: Denis Magda <dma...@apache.org> Date: Thursday, January 4, 2018 at 12:34 PM To: Revin Chalil <rcha...@expedia.com> Cc: "dev@ignite.apache.org" <dev@ignite.apache.org> Subject: Re: Spark data frames integration merged Revin, As as side note, do you hav

Re: Spark data frames integration merged

Revin, As as side note, do you have a public article published or any other relevant material that explains how Ignite is used at Expedia? You would help the community out a lot if such information is referenced from this page: https://ignite.apache.org/provenusecases.html

Re: Spark data frames integration merged

Revin, I doubt IgniteRDD#getPrefferredLocations has any affect on data frames, but this is an interesting point. Nikolay, as a developer of this functionality, can you please comment on this? -Val On Wed, Jan 3, 2018 at 1:22 PM, Revin Chalil wrote: > Thanks Val for the

Re: Spark data frames integration merged

Thank you and this is great news. We currently use the Ignite cache as a Reference dataset RDD in Spark, convert it into a spark DataFrame and then join this DF with the incoming-data DF. I hope we can change this 3 step process to a single step with the Spark DF integration. If so, would

Re: Spark data frames integration merged

Thank you, guys. Val, thanks for all reviews, advices and patience. Anton, thanks for ignite wisdom you share with me. Looking forward for next issues :) P.S Happy New Year for all Ignite community! В Пт, 29/12/2017 в 13:22 -0800, Valentin Kulichenko пишет: > Igniters, > > Great news! We

Re: Spark data frames integration merged

Great news, Thanks Nikolay and Val! Nikolay, could you document the feature before the release [1]? I’ve granted you required permission. More on the doc process can be found here [2]. [1] https://issues.apache.org/jira/browse/IGNITE-7345