Re: Spark data frames

That's a great idea! Nikolay, let me know if you need any help with the presentation, I will be happy to help. -Val On Fri, Feb 16, 2018 at 12:19 AM, Nikolay Izhikov wrote: > Ok, Igniters. > > I will do it in a few weeks. > I need time to prepare to the talk. > > В Пт, 16/02/2018 в 07:16 +,

Re: Spark data frames

Ok, Igniters. I will do it in a few weeks. I need time to prepare to the talk. В Пт, 16/02/2018 в 07:16 +, Dmitry Pavlov пишет: > +1 for starting new topic from Nikolay when ' Community Meeting to > Introduce Ignite Spark Data Frames' is ready to be announced. > > пт, 16 февр. 2018 г. в 9:2

Re: Spark data frames

+1 for starting new topic from Nikolay when ' Community Meeting to Introduce Ignite Spark Data Frames' is ready to be announced. пт, 16 февр. 2018 г. в 9:27, Denis Magda : > I'm in :) > > Nikolay, we can use my GoToMeeting account to host the webinar. To draw > more attention I would suggest sta

Re: Spark data frames

I'm in :) Nikolay, we can use my GoToMeeting account to host the webinar. To draw more attention I would suggest starting a more specific thread titled like "[RSVP] Community Meeting to Introduce Ignite Spark Data Frames". This discussion sounds too generic, folks could simply pass by. Negotiated

Re: Spark data frames

Dmitry, it's a great idea! Nikolay, I also have the interest to get familiar with Spark Data Frames integration. I'd prefer webinar of the format similar to "Ignite Persistent Store Walkthrough" by Denis Magda, which has been presented some times ago. On Wed, Feb 14, 2018 at 5:03 PM, Dmitriy Set

Re: Spark data frames

I am definitely interested. Great idea! On Wed, Feb 14, 2018 at 4:32 AM, Nikolay Izhikov wrote: > Hello, Dmitry. > > If other community member are also iterested in that kind of information I > can try to do the talk. > > В Ср, 14/02/2018 в 10:49 +, Dmitry Pavlov пишет: > > Hi Nikolay, > > >

Re: Spark data frames

Hello, Dmitry. If other community member are also iterested in that kind of information I can try to do the talk. В Ср, 14/02/2018 в 10:49 +, Dmitry Pavlov пишет: > Hi Nikolay, > > I've notices there are a number of very lively discussions on dev list about > SparkDataFrames. But I, for ex

Re: Spark data frames integration merged

Hello, Denis. > Nikolay, could you document the feature before the release [1]? Yes, I can. I will document these feature in a next few days. В Пт, 29/12/2017 в 13:37 -0800, Denis Magda пишет: > Great news, > > Thanks Nikolay and Val! > > Nikolay, could you document the feature before the re

Re: Spark data frames integration merged

will keep in touch and thanks again for all the great work and help > everyone. > > Revin > > From: Denis Magda > Date: Thursday, January 4, 2018 at 12:34 PM > To: Revin Chalil > Cc: "dev@ignite.apache.org" > Subject: Re: Spark data frames integration merg

Re: Spark data frames integration merged

Hello, guys. Currently `getPreferredLocations` implemented in `IgniteRDD -> IgniteAbstractRDD`. But DataFrame implementation uses `IgniteSQLDataFrameRDD -> IgniteSqlRDD -> IgniteAbstractRDD` Where `->` is extension. So, for now, getPreferredLocation doesn't implemented for a IgniteDataFrame.

Re: Spark data frames integration merged

. Revin From: Denis Magda Date: Thursday, January 4, 2018 at 12:34 PM To: Revin Chalil Cc: "dev@ignite.apache.org" Subject: Re: Spark data frames integration merged Revin, As as side note, do you have a public article published or any other relevant material that explains how Ignite

Re: Spark data frames integration merged

Revin, As as side note, do you have a public article published or any other relevant material that explains how Ignite is used at Expedia? You would help the community out a lot if such information is referenced from this page: https://ignite.apache.org/provenusecases.html

Re: Spark data frames integration merged

Revin, I doubt IgniteRDD#getPrefferredLocations has any affect on data frames, but this is an interesting point. Nikolay, as a developer of this functionality, can you please comment on this? -Val On Wed, Jan 3, 2018 at 1:22 PM, Revin Chalil wrote: > Thanks Val for the info on indexes with DF.

Re: Spark data frames integration merged

Thank you and this is great news. We currently use the Ignite cache as a Reference dataset RDD in Spark, convert it into a spark DataFrame and then join this DF with the incoming-data DF. I hope we can change this 3 step process to a single step with the Spark DF integration. If so, would inde

Re: Spark data frames integration merged

Thank you, guys. Val, thanks for all reviews, advices and patience. Anton, thanks for ignite wisdom you share with me. Looking forward for next issues :) P.S Happy New Year for all Ignite community! В Пт, 29/12/2017 в 13:22 -0800, Valentin Kulichenko пишет: > Igniters, > > Great news! We comp

Re: Spark data frames integration merged

Great news, Thanks Nikolay and Val! Nikolay, could you document the feature before the release [1]? I’ve granted you required permission. More on the doc process can be found here [2]. [1] https://issues.apache.org/jira/browse/IGNITE-7345 [2