Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project
Hello Dmitriy, Denis Thank you for help in creating the repo. I can see it has synced in github as well. Can you please add a sample readme.md file as the repo is empty I am unable to fork it? or let me check if I can directly commit a sample readme file from local then I should be able to fork this repo. I will start working on the migration for Flink, Camel, MQTT, etc. Emmanouil, please take a look and feel free to raise PR in this new repo. Regards, Saikat On Thu, Oct 31, 2019 at 3:14 PM Denis Magda wrote: > Dmitriy, thanks a lot for a quick turnaround! Saikat, please let us know if > anything else is needed. > > - > Denis > > > On Thu, Oct 31, 2019 at 1:12 PM Dmitriy Pavlov wrote: > > > Hi, I've created > > https://gitbox.apache.org/repos/asf/ignite-extensions.git > > > > It should be synced in a few minutes. > > > > Sincerely, > > Dmitriy Pavlov > > > > чт, 31 окт. 2019 г. в 22:58, Denis Magda : > > > > > Ignite PMCs, can anybody create the repo for Saikat? I'm on the road > and > > > can't do that. > > > > > > - > > > Denis > > > > > > > > > On Wed, Oct 30, 2019 at 8:40 PM Saikat Maitra > > > > wrote: > > > > > > > Hi, > > > > > > > > I would need help from Ignite PMC member to create the new > > > > repository ignite-extensions. > > > > > > > > Can you please create the repository here > > https://selfserve.apache.org/ > > > > > > > > Regards, > > > > Saikat > > > > > > > > On Wed, Oct 30, 2019 at 10:28 PM Saikat Maitra < > > saikat.mai...@gmail.com> > > > > wrote: > > > > > > > > > Hello Denis, > > > > > > > > > > Thank you for your response. > > > > > > > > > > I tried to request for new repository but the repository name is > > > showing > > > > > up as YourPMCs-ignite-extensions.git. I have reached out to Apache > > > Infra > > > > > user email list to understand the process to create a new > repository. > > > > > > > > > > I will definitely help to migrate the other ignite extensions to > the > > > new > > > > > repository once the repository is created. > > > > > > > > > > Regards, > > > > > Saikat > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 11:34 AM Denis Magda > > > wrote: > > > > > > > > > >> "ignite-extensions" might be a better name in the long term if we > > > decide > > > > >> to > > > > >> place any extension to the repo that doesn't integrate with any > 3rd > > > > party > > > > >> technology but rather enhances Ignite with extra API. > > > > >> > > > > >> After thinking about it for the last few days, I think that the > best > > > > >> approach to start with should be as follows: > > > > >> > > > > >>1. Let's create an "ignite-extensions" repo and let Emmanouil > > > > >> contribute > > > > >>the Pub/Sub Streamer integration there. > > > > >>2. Let's move after or in parallel our streaming integrations > > there > > > > >> like > > > > >>Flink, Camel, MQTT, etc.: > > > > >> > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > > > > >>3. After checking the approach with the first integrations we > can > > > > carry > > > > >>on with the rest of modules: Spark and Hibernate require their > > own > > > > >> repos, > > > > >>our TensorFlow integration might be moved to > "ignite-extensions" > > as > > > > >> well > > > > >>unless Alexey Zinoviev suggests a dedicated extensions repo for > > ML > > > > >>integrations. > > > > >> > > > > >> Thoughts? Saikat, are you willing to help with #1 and lead #2? > > > > >> > > > > >> - > > > > >> Denis > > > > >> > > > > >> > > > > >> On Tue, Oct 29, 2019 at 1:25 AM Ivan Pavlukhin < > vololo...@gmail.com > > > > > > > >> wrote: > > > > >> > > > > >> > Folks, > > > > >> > > > > > >> > Just another one name candidate "ignite-integrations" > > > > >> > > > > > >> > Or are there not only integrations? > > > > >> > > > > > >> > вт, 29 окт. 2019 г. в 03:31, Saikat Maitra < > > saikat.mai...@gmail.com > > > >: > > > > >> > > > > > > >> > > Hi Denis, > > > > >> > > > > > > >> > > I meant we can create dedicated repository for individual > > > extensions > > > > >> and > > > > >> > we > > > > >> > > can create submodules inside each specific repository for > > > individual > > > > >> > > extensions versions. > > > > >> > > > > > > >> > > Regards, > > > > >> > > Saikat > > > > >> > > > > > > >> > > > > > > >> > > On Mon, Oct 28, 2019 at 7:14 PM Saikat Maitra < > > > > >> saikat.mai...@gmail.com> > > > > >> > > wrote: > > > > >> > > > > > > >> > > > Hi Denis, > > > > >> > > > > > > > >> > > > Yes, dedicated repository is also a good idea and we can > > create > > > > >> > submodules > > > > >> > > > inside each specific repository for individual extensions. > > > > >> > > > > > > > >> > > > Regards, > > > > >> > > > Saikat > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > On Mon, Oct 28, 2019 at
Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project
Dmitriy, thanks a lot for a quick turnaround! Saikat, please let us know if anything else is needed. - Denis On Thu, Oct 31, 2019 at 1:12 PM Dmitriy Pavlov wrote: > Hi, I've created > https://gitbox.apache.org/repos/asf/ignite-extensions.git > > It should be synced in a few minutes. > > Sincerely, > Dmitriy Pavlov > > чт, 31 окт. 2019 г. в 22:58, Denis Magda : > > > Ignite PMCs, can anybody create the repo for Saikat? I'm on the road and > > can't do that. > > > > - > > Denis > > > > > > On Wed, Oct 30, 2019 at 8:40 PM Saikat Maitra > > wrote: > > > > > Hi, > > > > > > I would need help from Ignite PMC member to create the new > > > repository ignite-extensions. > > > > > > Can you please create the repository here > https://selfserve.apache.org/ > > > > > > Regards, > > > Saikat > > > > > > On Wed, Oct 30, 2019 at 10:28 PM Saikat Maitra < > saikat.mai...@gmail.com> > > > wrote: > > > > > > > Hello Denis, > > > > > > > > Thank you for your response. > > > > > > > > I tried to request for new repository but the repository name is > > showing > > > > up as YourPMCs-ignite-extensions.git. I have reached out to Apache > > Infra > > > > user email list to understand the process to create a new repository. > > > > > > > > I will definitely help to migrate the other ignite extensions to the > > new > > > > repository once the repository is created. > > > > > > > > Regards, > > > > Saikat > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 11:34 AM Denis Magda > > wrote: > > > > > > > >> "ignite-extensions" might be a better name in the long term if we > > decide > > > >> to > > > >> place any extension to the repo that doesn't integrate with any 3rd > > > party > > > >> technology but rather enhances Ignite with extra API. > > > >> > > > >> After thinking about it for the last few days, I think that the best > > > >> approach to start with should be as follows: > > > >> > > > >>1. Let's create an "ignite-extensions" repo and let Emmanouil > > > >> contribute > > > >>the Pub/Sub Streamer integration there. > > > >>2. Let's move after or in parallel our streaming integrations > there > > > >> like > > > >>Flink, Camel, MQTT, etc.: > > > >> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > > > >>3. After checking the approach with the first integrations we can > > > carry > > > >>on with the rest of modules: Spark and Hibernate require their > own > > > >> repos, > > > >>our TensorFlow integration might be moved to "ignite-extensions" > as > > > >> well > > > >>unless Alexey Zinoviev suggests a dedicated extensions repo for > ML > > > >>integrations. > > > >> > > > >> Thoughts? Saikat, are you willing to help with #1 and lead #2? > > > >> > > > >> - > > > >> Denis > > > >> > > > >> > > > >> On Tue, Oct 29, 2019 at 1:25 AM Ivan Pavlukhin > > > > >> wrote: > > > >> > > > >> > Folks, > > > >> > > > > >> > Just another one name candidate "ignite-integrations" > > > >> > > > > >> > Or are there not only integrations? > > > >> > > > > >> > вт, 29 окт. 2019 г. в 03:31, Saikat Maitra < > saikat.mai...@gmail.com > > >: > > > >> > > > > > >> > > Hi Denis, > > > >> > > > > > >> > > I meant we can create dedicated repository for individual > > extensions > > > >> and > > > >> > we > > > >> > > can create submodules inside each specific repository for > > individual > > > >> > > extensions versions. > > > >> > > > > > >> > > Regards, > > > >> > > Saikat > > > >> > > > > > >> > > > > > >> > > On Mon, Oct 28, 2019 at 7:14 PM Saikat Maitra < > > > >> saikat.mai...@gmail.com> > > > >> > > wrote: > > > >> > > > > > >> > > > Hi Denis, > > > >> > > > > > > >> > > > Yes, dedicated repository is also a good idea and we can > create > > > >> > submodules > > > >> > > > inside each specific repository for individual extensions. > > > >> > > > > > > >> > > > Regards, > > > >> > > > Saikat > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > On Mon, Oct 28, 2019 at 2:34 PM Denis Magda < > dma...@apache.org> > > > >> wrote: > > > >> > > > > > > >> > > >> Folks, > > > >> > > >> > > > >> > > >> What do you think about having a dedicated repository for > each > > > >> > extension > > > >> > > >> instead of a single one? It sounds complicated but might be > > > >> > worthwhile for > > > >> > > >> cases when an integration requires to support several > versions. > > > >> Take > > > >> > Spark > > > >> > > >> or Hibernate as an example, we already provide different > > versions > > > >> of > > > >> > > >> Ignite > > > >> > > >> packages for various Spark/Hibernate versions. It will be > > easier > > > to > > > >> > handle > > > >> > > >> if Spark or Hibernate had their own repos. > > > >> > > >> > > > >> > > >> - > > > >> > > >> Denis > > > >> > > >> > > > >> > > >> > > > >> > > >> On Sat, Oct 26, 2019 at 10:06 PM Saikat Maitra < > > >
Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project
Hi, I've created https://gitbox.apache.org/repos/asf/ignite-extensions.git It should be synced in a few minutes. Sincerely, Dmitriy Pavlov чт, 31 окт. 2019 г. в 22:58, Denis Magda : > Ignite PMCs, can anybody create the repo for Saikat? I'm on the road and > can't do that. > > - > Denis > > > On Wed, Oct 30, 2019 at 8:40 PM Saikat Maitra > wrote: > > > Hi, > > > > I would need help from Ignite PMC member to create the new > > repository ignite-extensions. > > > > Can you please create the repository here https://selfserve.apache.org/ > > > > Regards, > > Saikat > > > > On Wed, Oct 30, 2019 at 10:28 PM Saikat Maitra > > wrote: > > > > > Hello Denis, > > > > > > Thank you for your response. > > > > > > I tried to request for new repository but the repository name is > showing > > > up as YourPMCs-ignite-extensions.git. I have reached out to Apache > Infra > > > user email list to understand the process to create a new repository. > > > > > > I will definitely help to migrate the other ignite extensions to the > new > > > repository once the repository is created. > > > > > > Regards, > > > Saikat > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 11:34 AM Denis Magda > wrote: > > > > > >> "ignite-extensions" might be a better name in the long term if we > decide > > >> to > > >> place any extension to the repo that doesn't integrate with any 3rd > > party > > >> technology but rather enhances Ignite with extra API. > > >> > > >> After thinking about it for the last few days, I think that the best > > >> approach to start with should be as follows: > > >> > > >>1. Let's create an "ignite-extensions" repo and let Emmanouil > > >> contribute > > >>the Pub/Sub Streamer integration there. > > >>2. Let's move after or in parallel our streaming integrations there > > >> like > > >>Flink, Camel, MQTT, etc.: > > >> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > > >>3. After checking the approach with the first integrations we can > > carry > > >>on with the rest of modules: Spark and Hibernate require their own > > >> repos, > > >>our TensorFlow integration might be moved to "ignite-extensions" as > > >> well > > >>unless Alexey Zinoviev suggests a dedicated extensions repo for ML > > >>integrations. > > >> > > >> Thoughts? Saikat, are you willing to help with #1 and lead #2? > > >> > > >> - > > >> Denis > > >> > > >> > > >> On Tue, Oct 29, 2019 at 1:25 AM Ivan Pavlukhin > > >> wrote: > > >> > > >> > Folks, > > >> > > > >> > Just another one name candidate "ignite-integrations" > > >> > > > >> > Or are there not only integrations? > > >> > > > >> > вт, 29 окт. 2019 г. в 03:31, Saikat Maitra >: > > >> > > > > >> > > Hi Denis, > > >> > > > > >> > > I meant we can create dedicated repository for individual > extensions > > >> and > > >> > we > > >> > > can create submodules inside each specific repository for > individual > > >> > > extensions versions. > > >> > > > > >> > > Regards, > > >> > > Saikat > > >> > > > > >> > > > > >> > > On Mon, Oct 28, 2019 at 7:14 PM Saikat Maitra < > > >> saikat.mai...@gmail.com> > > >> > > wrote: > > >> > > > > >> > > > Hi Denis, > > >> > > > > > >> > > > Yes, dedicated repository is also a good idea and we can create > > >> > submodules > > >> > > > inside each specific repository for individual extensions. > > >> > > > > > >> > > > Regards, > > >> > > > Saikat > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Mon, Oct 28, 2019 at 2:34 PM Denis Magda > > >> wrote: > > >> > > > > > >> > > >> Folks, > > >> > > >> > > >> > > >> What do you think about having a dedicated repository for each > > >> > extension > > >> > > >> instead of a single one? It sounds complicated but might be > > >> > worthwhile for > > >> > > >> cases when an integration requires to support several versions. > > >> Take > > >> > Spark > > >> > > >> or Hibernate as an example, we already provide different > versions > > >> of > > >> > > >> Ignite > > >> > > >> packages for various Spark/Hibernate versions. It will be > easier > > to > > >> > handle > > >> > > >> if Spark or Hibernate had their own repos. > > >> > > >> > > >> > > >> - > > >> > > >> Denis > > >> > > >> > > >> > > >> > > >> > > >> On Sat, Oct 26, 2019 at 10:06 PM Saikat Maitra < > > >> > saikat.mai...@gmail.com> > > >> > > >> wrote: > > >> > > >> > > >> > > >> > Hello Alexey, > > >> > > >> > > > >> > > >> > Thank you for your email. Yes, I am also aligned for > > >> > ignite-extensions. > > >> > > >> > > > >> > > >> > If others are also ok we can create the new repository with > > name > > >> > > >> > ignite-extensions. > > >> > > >> > > > >> > > >> > Regards, > > >> > > >> > Saikat > > >> > > >> > > > >> > > >> > On Sat, Oct 26, 2019 at 3:11 AM Alexey Zinoviev < > > >> > zaleslaw@gmail.com > > >> > > >> > > > >> > > >> > wrote: > > >> > > >> > > > >> > > >> > >
Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project
Ignite PMCs, can anybody create the repo for Saikat? I'm on the road and can't do that. - Denis On Wed, Oct 30, 2019 at 8:40 PM Saikat Maitra wrote: > Hi, > > I would need help from Ignite PMC member to create the new > repository ignite-extensions. > > Can you please create the repository here https://selfserve.apache.org/ > > Regards, > Saikat > > On Wed, Oct 30, 2019 at 10:28 PM Saikat Maitra > wrote: > > > Hello Denis, > > > > Thank you for your response. > > > > I tried to request for new repository but the repository name is showing > > up as YourPMCs-ignite-extensions.git. I have reached out to Apache Infra > > user email list to understand the process to create a new repository. > > > > I will definitely help to migrate the other ignite extensions to the new > > repository once the repository is created. > > > > Regards, > > Saikat > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 30, 2019 at 11:34 AM Denis Magda wrote: > > > >> "ignite-extensions" might be a better name in the long term if we decide > >> to > >> place any extension to the repo that doesn't integrate with any 3rd > party > >> technology but rather enhances Ignite with extra API. > >> > >> After thinking about it for the last few days, I think that the best > >> approach to start with should be as follows: > >> > >>1. Let's create an "ignite-extensions" repo and let Emmanouil > >> contribute > >>the Pub/Sub Streamer integration there. > >>2. Let's move after or in parallel our streaming integrations there > >> like > >>Flink, Camel, MQTT, etc.: > >> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > >>3. After checking the approach with the first integrations we can > carry > >>on with the rest of modules: Spark and Hibernate require their own > >> repos, > >>our TensorFlow integration might be moved to "ignite-extensions" as > >> well > >>unless Alexey Zinoviev suggests a dedicated extensions repo for ML > >>integrations. > >> > >> Thoughts? Saikat, are you willing to help with #1 and lead #2? > >> > >> - > >> Denis > >> > >> > >> On Tue, Oct 29, 2019 at 1:25 AM Ivan Pavlukhin > >> wrote: > >> > >> > Folks, > >> > > >> > Just another one name candidate "ignite-integrations" > >> > > >> > Or are there not only integrations? > >> > > >> > вт, 29 окт. 2019 г. в 03:31, Saikat Maitra : > >> > > > >> > > Hi Denis, > >> > > > >> > > I meant we can create dedicated repository for individual extensions > >> and > >> > we > >> > > can create submodules inside each specific repository for individual > >> > > extensions versions. > >> > > > >> > > Regards, > >> > > Saikat > >> > > > >> > > > >> > > On Mon, Oct 28, 2019 at 7:14 PM Saikat Maitra < > >> saikat.mai...@gmail.com> > >> > > wrote: > >> > > > >> > > > Hi Denis, > >> > > > > >> > > > Yes, dedicated repository is also a good idea and we can create > >> > submodules > >> > > > inside each specific repository for individual extensions. > >> > > > > >> > > > Regards, > >> > > > Saikat > >> > > > > >> > > > > >> > > > > >> > > > On Mon, Oct 28, 2019 at 2:34 PM Denis Magda > >> wrote: > >> > > > > >> > > >> Folks, > >> > > >> > >> > > >> What do you think about having a dedicated repository for each > >> > extension > >> > > >> instead of a single one? It sounds complicated but might be > >> > worthwhile for > >> > > >> cases when an integration requires to support several versions. > >> Take > >> > Spark > >> > > >> or Hibernate as an example, we already provide different versions > >> of > >> > > >> Ignite > >> > > >> packages for various Spark/Hibernate versions. It will be easier > to > >> > handle > >> > > >> if Spark or Hibernate had their own repos. > >> > > >> > >> > > >> - > >> > > >> Denis > >> > > >> > >> > > >> > >> > > >> On Sat, Oct 26, 2019 at 10:06 PM Saikat Maitra < > >> > saikat.mai...@gmail.com> > >> > > >> wrote: > >> > > >> > >> > > >> > Hello Alexey, > >> > > >> > > >> > > >> > Thank you for your email. Yes, I am also aligned for > >> > ignite-extensions. > >> > > >> > > >> > > >> > If others are also ok we can create the new repository with > name > >> > > >> > ignite-extensions. > >> > > >> > > >> > > >> > Regards, > >> > > >> > Saikat > >> > > >> > > >> > > >> > On Sat, Oct 26, 2019 at 3:11 AM Alexey Zinoviev < > >> > zaleslaw@gmail.com > >> > > >> > > >> > > >> > wrote: > >> > > >> > > >> > > >> > > Vote for ignite-extensions (it's more widely than > >> > ignite-connectors) > >> > > >> > > > >> > > >> > > сб, 26 окт. 2019 г. в 05:52, Saikat Maitra < > >> > saikat.mai...@gmail.com>: > >> > > >> > > > >> > > >> > > > Hello Denis, Dmitriy > >> > > >> > > > > >> > > >> > > > Thank you for your reply. I am thinking when a new repo is > >> > created > >> > > >> that > >> > > >> > > > will not mean that the project will undergo Incubator > >> process , > >> > Is > >> > > >> > > > this correct? > >> > > >> > > > > >> > > >> > > > We
Re: Re[2]: [VOTE] Apache Ignite PMC Chair
+1 Alexey Goncharuk On Thu, Oct 31, 2019 at 12:53 PM Sergey Antonov wrote: > +1 Dmitry Pavlov > > чт, 31 окт. 2019 г. в 13:59, Вячеслав Коптилин : > > > +1 Dmitry Pavlov > > > > Thanks, > > S. > > > > чт, 31 окт. 2019 г. в 12:31, Алексей Платонов : > > > > > +1 for Dmitry Pavlov > > > > > > чт, 31 окт. 2019 г. в 01:23, Valentin Kulichenko < > > > valentin.kuliche...@gmail.com>: > > > > > > > +1 Dmitry Pavlov (binding) > > > > > > > > On Wed, Oct 30, 2019 at 12:55 PM Alex Plehanov < > > plehanov.a...@gmail.com> > > > > wrote: > > > > > > > > > + 1 Dmitry Pavlov > > > > > > > > > > ср, 30 окт. 2019 г. в 20:50, Pavel Kovalenko : > > > > > > > > > > > +1 for Dmitry Pavlov > > > > > > > > > > > > ср, 30 окт. 2019 г. в 18:46, Alexei Scherbakov < > > > > > > alexey.scherbak...@gmail.com > > > > > > >: > > > > > > > > > > > > > +1 for Dmitry Pavlov > > > > > > > > > > > > > > ср, 30 окт. 2019 г. в 18:22, aealexsandrov < > > > aealexsand...@gmail.com > > > > >: > > > > > > > > > > > > > > > +1 Alexey Goncharuk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Sent from: > > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Best regards, > > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > BR, Sergey Antonov >
Re: H2O integration
Yes, you are correct чт, 31 окт. 2019 г., 20:29 Denis Magda : > Basically it means a model trained with H2O can be transformed to Ignite’s > one and executed in our cluster? Sorry if I’m still missing something. > > Denis > > On Thursday, October 31, 2019, Alexey Zinoviev > wrote: > > > Thanks, Denis, no, the initial H2O integration is not the same as > > TensorFlow integration, it perform the prediction phase (than you have > > trained model) nor training as I mentioned "As a result we could predict > > something on Ignite data via the model trained in H2O system." > > > > It's the same as XGBost and Spark ML integration, but I hope that we > could > > extend our integration here and provide something like Ignite data-source > > for training in H2O in the future. > > > > чт, 31 окт. 2019 г. в 16:48, Denis Magda : > > > > > Hi Alexey, > > > > > > That’s great to see that our ML and DL capabilities keep growing via > > > integrations! > > > > > > Is it correct to say that H2O uses Ignite as a data source in a way > > similar > > > to what we did for TensorFlow? > > > > > > Btw, do you have any plans to spread the word about the additions > through > > > blogging? H2O, Spark ML, XGBoost deserve their own articles. > > > > > > Denis > > > > > > On Thursday, October 31, 2019, Alexey Zinoviev > > > > wrote: > > > > > > > Hi, Igniters, > > > >I happy to announce the new part of Ignite ML functionality. > > > > > > > > The integration with the popular distributed library H2O > > > > https://github.com/h2oai was added. > > > > As a result we could predict something on Ignite data via the model > > > trained > > > > in H2O system. > > > > > > > > Many thanks to Michal Kurka (https://github.com/michalkurka), > waiting > > > him > > > > on our dev-list, hope that he will help with the backward integration > > > from > > > > Ignite to H2O system. > > > > > > > > Remind, that now we have integrations with XGBoost, TensorFlow, Spark > > ML > > > > via SparkMlParser and MLeap integration and waiting for the new > > > > integrations with Dl4j or PyTorch for example > > > > > > > > Sincerely yours, > > > > Alexey Zinoviev > > > > > > > > > > > > > -- > > > - > > > Denis > > > > > > > > -- > - > Denis >
Re: H2O integration
Basically it means a model trained with H2O can be transformed to Ignite’s one and executed in our cluster? Sorry if I’m still missing something. Denis On Thursday, October 31, 2019, Alexey Zinoviev wrote: > Thanks, Denis, no, the initial H2O integration is not the same as > TensorFlow integration, it perform the prediction phase (than you have > trained model) nor training as I mentioned "As a result we could predict > something on Ignite data via the model trained in H2O system." > > It's the same as XGBost and Spark ML integration, but I hope that we could > extend our integration here and provide something like Ignite data-source > for training in H2O in the future. > > чт, 31 окт. 2019 г. в 16:48, Denis Magda : > > > Hi Alexey, > > > > That’s great to see that our ML and DL capabilities keep growing via > > integrations! > > > > Is it correct to say that H2O uses Ignite as a data source in a way > similar > > to what we did for TensorFlow? > > > > Btw, do you have any plans to spread the word about the additions through > > blogging? H2O, Spark ML, XGBoost deserve their own articles. > > > > Denis > > > > On Thursday, October 31, 2019, Alexey Zinoviev > > wrote: > > > > > Hi, Igniters, > > >I happy to announce the new part of Ignite ML functionality. > > > > > > The integration with the popular distributed library H2O > > > https://github.com/h2oai was added. > > > As a result we could predict something on Ignite data via the model > > trained > > > in H2O system. > > > > > > Many thanks to Michal Kurka (https://github.com/michalkurka), waiting > > him > > > on our dev-list, hope that he will help with the backward integration > > from > > > Ignite to H2O system. > > > > > > Remind, that now we have integrations with XGBoost, TensorFlow, Spark > ML > > > via SparkMlParser and MLeap integration and waiting for the new > > > integrations with Dl4j or PyTorch for example > > > > > > Sincerely yours, > > > Alexey Zinoviev > > > > > > > > > -- > > - > > Denis > > > -- - Denis
Re: [DISCUSSION] Single point in API for changing cluster state.
Sergey Antonov, > Read-only mode doesn't affects rebalance process. After activation (in read-only mode or not) rebalancing is possible to begin and the grid will not be free of updates until it's finished. So the grid will not be in truly read-only mode even if cache updates are prohibited. Probably it would be enough just wait until rebalancing is finished before releasing future. > How about INACTIVE, ACTIVE, ACTIVE_READ-ONLY states? INACTIVE, READ_WRITE, READ_ONLY seems more appropriate for ClusterState. I do not understand the necessity of handling states comparison on transition. Why not just return current(previous) state ? Could you give more detailed explanation for this ? вт, 29 окт. 2019 г. в 14:25, Sergey Antonov : > He, Igniters! > > I'd like to share some points encountered during work on ticket [1]: > >- I added property clusterStateOnStart with type ClusterState to >IgniteConfiguration. The property will be analogue of activeOnStart. >Default value of the property will be ACTIVE for keeping defalut value >consistency. Also I marked property activeOnStart as deprecated. >- I introduced order on values of ClusterState. It needs for user >friendly behaviour during cluster state transition. If cluster is > changing >state from state_A to state_B and user is requesting current cluster >state without waiting end of transition we must return lesser of two >states: state_A and state_B. I think the order must be: ACTIVE > >READ_ONLY > INACTIVE. Examples (state_A -> state_B = response on the >users cluster state request during transition): > - ACTIVE -> INACTIVE = INACTIVE (Now we have this behavior) > - INACTIVE -> ACTIVE = INACTIVE (Now we have this behavior) > - ACTIVE -> READ_ONLY = READ_ONLY > - READ_ONLY -> ACTIVE = READ_ONLY > - READ_ONLY -> INACTIVE = INACTIVE > - INACTIVE -> READ_ONLY = INACTIVE > > I'd like to know your opinion about my points. What do you think about it? > > [1] https://issues.apache.org/jira/browse/IGNITE-12225 > > > вт, 15 окт. 2019 г. в 14:56, Sergey Antonov : > > > Hi, Alexei! > > > > Thank you for reply! > > > > > The states ACTIVE, INACTIVE, READ-ONLY look confusing. Actually > > read-only cluster is active too. > > How about INACTIVE, ACTIVE, ACTIVE_READ-ONLY states? > > > > > Also it would be useful to allow users wait for re-balance which could > > happen after activation in read-only mode to achieve really idle grid. > > Read-only mode doesn't affects rebalance process. > > > > > I would suggest adding new property to Ignite configuration like > > setActivationOptions(ActivationOption... options) which should be mutable > > in runtime. > > I'm not sure that it's good idea. @Alexey Goncharuk > > I'd like to know your opinion about activation > > option and storing them on PDS. > > > > > This proposal also better regarding backward compatibility. > > Which kind of compatibility did you mean? New cluster mode doesn't > affects > > PDS compatibility. > > > > ср, 25 сент. 2019 г. в 13:26, Alexei Scherbakov < > > alexey.scherbak...@gmail.com>: > > > >> Sergey Antonov, > >> > >> The states ACTIVE, INACTIVE, READ-ONLY look confusing. > >> Actually read-only cluster is active too. > >> > >> I would suggest adding new property to Ignite configuration like > >> setActivationOptions(ActivationOption... options) which should be > mutable > >> in runtime. > >> > >> For control.sh should be the same options for activate command. > >> > >> Also it would be useful to allow users wait for re-balance which could > >> happen after activation in read-only mode to achieve really idle grid. > >> > >> So, available options list in my opinion: READ_ONLY, WAIT_FOR_REBALANCE. > >> > >> This proposal also better regarding backward compatibility. > >> > >> вт, 24 сент. 2019 г. в 20:35, Sergey Antonov >: > >> > >> > Maxim, > >> > > >> > > I think from the user point the INACTIVE -> READ-ONLY transition [1] > >> > should be allowed prior to adding a new `state` command [2] to avoid > >> > unnecessary error messages. > >> > Yes. I plan complete both tickets till the code freeze of 2.8 release. > >> > > >> > > I also think we can avoid the word 'set` in this command. > >> > I don't think so. We already have command control.sh --state command > for > >> > getting current state. I think we shouldn't "overload" commands in > >> > control.sh. > >> > > >> > > What about cluster deactivation confirmation? Will it be used for > all > >> the > >> > cluster state changes? > >> > Yes. I think we should confirm any cluster state transition. > >> > > >> > вт, 24 сент. 2019 г. в 19:07, Maxim Muzafarov : > >> > > >> > > Sergey, > >> > > > >> > > +1, I like your idea. > >> > > > >> > > I think from the user point the INACTIVE -> READ-ONLY transition [1] > >> > > should be allowed prior to adding a new `state` command [2] to avoid > >> > > unnecessary error messages. I also think we can avoid the word 'set` > >> > > in
[jira] [Created] (IGNITE-12348) .NET: Query cursors are not thread-safe
Pavel Tupitsyn created IGNITE-12348:
---
Summary: .NET: Query cursors are not thread-safe
Key: IGNITE-12348
URL: https://issues.apache.org/jira/browse/IGNITE-12348
Project: Ignite
Issue Type: Bug
Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Query cursors (inheritors of {{QueryCursorBase}}) are not thread safe. The code
has an attempt to make them thread-safe, but race condition exists between
Dispose and MoveNext calls - it is possible that GetBatch is called after
Dispose, causing all kinds of issues like calls to non-existent filter.
We should either fix the race, or update the documentation with a thread
unsafety note.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: H2O integration
Thanks, Denis, no, the initial H2O integration is not the same as TensorFlow integration, it perform the prediction phase (than you have trained model) nor training as I mentioned "As a result we could predict something on Ignite data via the model trained in H2O system." It's the same as XGBost and Spark ML integration, but I hope that we could extend our integration here and provide something like Ignite data-source for training in H2O in the future. чт, 31 окт. 2019 г. в 16:48, Denis Magda : > Hi Alexey, > > That’s great to see that our ML and DL capabilities keep growing via > integrations! > > Is it correct to say that H2O uses Ignite as a data source in a way similar > to what we did for TensorFlow? > > Btw, do you have any plans to spread the word about the additions through > blogging? H2O, Spark ML, XGBoost deserve their own articles. > > Denis > > On Thursday, October 31, 2019, Alexey Zinoviev > wrote: > > > Hi, Igniters, > >I happy to announce the new part of Ignite ML functionality. > > > > The integration with the popular distributed library H2O > > https://github.com/h2oai was added. > > As a result we could predict something on Ignite data via the model > trained > > in H2O system. > > > > Many thanks to Michal Kurka (https://github.com/michalkurka), waiting > him > > on our dev-list, hope that he will help with the backward integration > from > > Ignite to H2O system. > > > > Remind, that now we have integrations with XGBoost, TensorFlow, Spark ML > > via SparkMlParser and MLeap integration and waiting for the new > > integrations with Dl4j or PyTorch for example > > > > Sincerely yours, > > Alexey Zinoviev > > > > > -- > - > Denis >
[jira] [Created] (IGNITE-12347) Flaky spark tests
Yury Gerzhedovich created IGNITE-12347: -- Summary: Flaky spark tests Key: IGNITE-12347 URL: https://issues.apache.org/jira/browse/IGNITE-12347 Project: Ignite Issue Type: Task Components: spark Reporter: Yury Gerzhedovich Assignee: Alexey Zinoviev There are few flaky spark tests. Need to investigate the reason and fix it. [TC spark flaky tests |https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5554421907791235109=%3Cdefault%3E=testDetails] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: H2O integration
Hi Alexey, That’s great to see that our ML and DL capabilities keep growing via integrations! Is it correct to say that H2O uses Ignite as a data source in a way similar to what we did for TensorFlow? Btw, do you have any plans to spread the word about the additions through blogging? H2O, Spark ML, XGBoost deserve their own articles. Denis On Thursday, October 31, 2019, Alexey Zinoviev wrote: > Hi, Igniters, >I happy to announce the new part of Ignite ML functionality. > > The integration with the popular distributed library H2O > https://github.com/h2oai was added. > As a result we could predict something on Ignite data via the model trained > in H2O system. > > Many thanks to Michal Kurka (https://github.com/michalkurka), waiting him > on our dev-list, hope that he will help with the backward integration from > Ignite to H2O system. > > Remind, that now we have integrations with XGBoost, TensorFlow, Spark ML > via SparkMlParser and MLeap integration and waiting for the new > integrations with Dl4j or PyTorch for example > > Sincerely yours, > Alexey Zinoviev > -- - Denis
H2O integration
Hi, Igniters, I happy to announce the new part of Ignite ML functionality. The integration with the popular distributed library H2O https://github.com/h2oai was added. As a result we could predict something on Ignite data via the model trained in H2O system. Many thanks to Michal Kurka (https://github.com/michalkurka), waiting him on our dev-list, hope that he will help with the backward integration from Ignite to H2O system. Remind, that now we have integrations with XGBoost, TensorFlow, Spark ML via SparkMlParser and MLeap integration and waiting for the new integrations with Dl4j or PyTorch for example Sincerely yours, Alexey Zinoviev
Re: Re[2]: [VOTE] Apache Ignite PMC Chair
+1 Dmitry Pavlov чт, 31 окт. 2019 г. в 13:59, Вячеслав Коптилин : > +1 Dmitry Pavlov > > Thanks, > S. > > чт, 31 окт. 2019 г. в 12:31, Алексей Платонов : > > > +1 for Dmitry Pavlov > > > > чт, 31 окт. 2019 г. в 01:23, Valentin Kulichenko < > > valentin.kuliche...@gmail.com>: > > > > > +1 Dmitry Pavlov (binding) > > > > > > On Wed, Oct 30, 2019 at 12:55 PM Alex Plehanov < > plehanov.a...@gmail.com> > > > wrote: > > > > > > > + 1 Dmitry Pavlov > > > > > > > > ср, 30 окт. 2019 г. в 20:50, Pavel Kovalenko : > > > > > > > > > +1 for Dmitry Pavlov > > > > > > > > > > ср, 30 окт. 2019 г. в 18:46, Alexei Scherbakov < > > > > > alexey.scherbak...@gmail.com > > > > > >: > > > > > > > > > > > +1 for Dmitry Pavlov > > > > > > > > > > > > ср, 30 окт. 2019 г. в 18:22, aealexsandrov < > > aealexsand...@gmail.com > > > >: > > > > > > > > > > > > > +1 Alexey Goncharuk > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Sent from: > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Best regards, > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > -- BR, Sergey Antonov
Re: Re[2]: [VOTE] Apache Ignite PMC Chair
+1 Dmitry Pavlov Thanks, S. чт, 31 окт. 2019 г. в 12:31, Алексей Платонов : > +1 for Dmitry Pavlov > > чт, 31 окт. 2019 г. в 01:23, Valentin Kulichenko < > valentin.kuliche...@gmail.com>: > > > +1 Dmitry Pavlov (binding) > > > > On Wed, Oct 30, 2019 at 12:55 PM Alex Plehanov > > wrote: > > > > > + 1 Dmitry Pavlov > > > > > > ср, 30 окт. 2019 г. в 20:50, Pavel Kovalenko : > > > > > > > +1 for Dmitry Pavlov > > > > > > > > ср, 30 окт. 2019 г. в 18:46, Alexei Scherbakov < > > > > alexey.scherbak...@gmail.com > > > > >: > > > > > > > > > +1 for Dmitry Pavlov > > > > > > > > > > ср, 30 окт. 2019 г. в 18:22, aealexsandrov < > aealexsand...@gmail.com > > >: > > > > > > > > > > > +1 Alexey Goncharuk > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sent from: > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best regards, > > > > > Alexei Scherbakov > > > > > > > > > > > > > > >
Re: Clearing of injected resources on node stop
I agree this behavior is strange and causes NPEs, saw it myself in tests. Probably this is required for correct re-initialization of injected resources after node restart. Looks like clearing can be removed if node starts successfully after stopping. Could you write a test to check ? Another possible fix - remove resource injection for build-in SPIs. Do we really need it at all ? пт, 25 окт. 2019 г. в 17:58, Дмитрий Сорокин : > Hi folks! > > At the moment every resources injected by GridResourceProcessor on node > start, also is clearing on node stop, but before the stopping of > executors. That behavior on node stopping sometimes causes NPEs at calls > which performed in threads of StripedExecutor, for example, see [1]. > > Does anybody remember the motivation of cleaning injected resources? > > Also, can anybody to tell what is bad things can happen if we won't clean > (nullify) injected logger resources? > > [1] https://issues.apache.org/jira/browse/IGNITE-1606 > > — > Dmitriy Sorokin > -- Best regards, Alexei Scherbakov
Re: Re[2]: [VOTE] Apache Ignite PMC Chair
+1 for Dmitry Pavlov чт, 31 окт. 2019 г. в 01:23, Valentin Kulichenko < valentin.kuliche...@gmail.com>: > +1 Dmitry Pavlov (binding) > > On Wed, Oct 30, 2019 at 12:55 PM Alex Plehanov > wrote: > > > + 1 Dmitry Pavlov > > > > ср, 30 окт. 2019 г. в 20:50, Pavel Kovalenko : > > > > > +1 for Dmitry Pavlov > > > > > > ср, 30 окт. 2019 г. в 18:46, Alexei Scherbakov < > > > alexey.scherbak...@gmail.com > > > >: > > > > > > > +1 for Dmitry Pavlov > > > > > > > > ср, 30 окт. 2019 г. в 18:22, aealexsandrov >: > > > > > > > > > +1 Alexey Goncharuk > > > > > > > > > > > > > > > > > > > > -- > > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > Alexei Scherbakov > > > > > > > > > >
Re: [MTCGA]: new failures in builds [4738135] needs to be handled
Fixed. чт, 31 окт. 2019 г. в 08:24, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're grateful that you were > a volunteer to make the contribution to this project, but things change and > you may no longer be able to finalize your contribution. > Could you respond to this email and indicate if you wish to continue and > fix test failures or step down and some committer may revert you commit. > > *New Trusted Suite failure in master [Licenses Headers] > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders?branch=%3Cdefault%3E > Changes may lead to failure were done by > - alexey scherbakov > https://ci.ignite.apache.org/viewModification.html?modId=891809 > > - Here's a reminder of what contributors were agreed to do > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > - Should you have any questions please contact > dev@ignite.apache.org > > Best Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 08:24:03 31-10-2019 > -- Best regards, Alexei Scherbakov
