Good point, now they are not in the Best state, we have a few ideas how to fix, will discuss it later here in this thread.
If you have any objection write it, but my position - creation special Spark Visa on TC, like for ML, maybe two for different Spark versions, and all commits to Spark will require this Visa. ср, 23 окт. 2019 г., 16:25 Ivan Pavlukhin <[email protected]>: > Actually, I meant running tests on TeamCity. > > ср, 23 окт. 2019 г. в 15:36, Alexey Zinoviev <[email protected]>: > > > > What kind of testing do you mean? JUnit, integration via example or > > something else? > > > > ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin <[email protected]>: > > > > > Correction. > > > > > > How are you going to TEST integrations with different Spark versions? > > > > > > ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin <[email protected]>: > > > > > > > > Alexey, > > > > > > > > Thank you for sharing the summary! > > > > > > > > Small question. How are you going to integrations with different > Spark > > > versions? > > > > > > > > чт, 17 окт. 2019 г. в 22:30, Denis Magda <[email protected]>: > > > > > > > > > > > > > > > > > 1. We are going to release Apache Ignite 2.8 with limited > support of > > > > > > Spark 2.4.4 (known issues are listed here > > > > > > https://issues.apache.org/jira/browse/IGNITE-12054) > > > > > > > > > > > > > > > What does limited support mean? Is it about regressions (something > that > > > > > worked before but will fail) or capabilities unsupported even for > the > > > > > presently released version of the integration? > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev < > > > [email protected]> > > > > > wrote: > > > > > > > > > > > The meeting notes and results of discussion are next: > > > > > > > > > > > > 1. We are going to release Apache Ignite 2.8 with limited > support > > > of > > > > > > Spark 2.4.4 (known issues are listed here > > > > > > https://issues.apache.org/jira/browse/IGNITE-12054) > > > > > > 2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4 > with > > > known > > > > > > fixed bugs > > > > > > 3. We will create the developer-profiles spark_2_3 and > spark_2_4 > > > > > > separately from current scala profile > > > > > > 4. We are ready to move the Ignite-Spark integration to the > > > separate > > > > > > module after release 2.8 with support of most actual Spark > > > versions and > > > > > > with independent release cycle > > > > > > 5. We will work together on the next release roadmap with > Nikolay > > > and > > > > > > all interested igniters to improve the current solution (add > new > > > > > > features > > > > > > as thin client support, working with caches via binary object > and > > > so on) > > > > > > 6. Nikolay is ready to help with the review of tickets as an > > > author of > > > > > > this component > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev < > [email protected] > > > >: > > > > > > > > > > > > > Yes, let's use the Russian for short discussion, the meeting > notes > > > will > > > > > > be > > > > > > > presented in English > > > > > > > > > > > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov < > [email protected] > > > >: > > > > > > > > > > > > > >> Hello, Alexey. > > > > > > >> > > > > > > >> I'm ready to take part in the meeting. > > > > > > >> It will be in Russian, isn't it? > > > > > > >> > > > > > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет: > > > > > > >> > Hi, Igniters, and especially Nikolay Izhikov > > > > > > >> > > > > > > > >> > I suggest to arrange the meeting via ASF-slack in > > > > > > >> #ignite-spark-integration > > > > > > >> > channel at 17 October, in 16 MSK Time. > > > > > > >> > The language of the meeting: Russian > > > > > > >> > > > > > > > >> > *The topics to discuss:* > > > > > > >> > > > > > > > >> > - Review of next tickets > > > > > > >> > > > > > > > >> > > > > > > > >> > 1. [Spark][Bug] IgniteSpark integration forget to close > the > > > > > > >> > IgniteContext and stops the client node in case if > error > > > during > > > > > > >> > PairFunction logic > > > > > > >> > <https://issues.apache.org/jira/browse/IGNITE-11724> > > > > > > >> > 2. [Spark] Add initial support of Spark 2.4 > > > > > > >> > <https://issues.apache.org/jira/browse/IGNITE-12247> > > > > > > >> > 3. [Spark] Add joins to Spark Dataframe examples > > > > > > >> > <https://issues.apache.org/jira/browse/IGNITE-9732> > > > > > > >> > > > > > > > >> > > > > > > > >> > - Known issues of migration to 2.4 version > > > > > > >> > - Organization of multiple version support for Apache > Spark > > > > > > >> > > > > > > > >> > P.S. meeting notes will be shared with the community > > > > > > >> > P.P.S Nikolay, if you have any objections about the time or > > > topics, > > > > > > >> please > > > > > > >> > let me know > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > -- > Best regards, > Ivan Pavlukhin >
