Awesome, ping us whenever you're ready! - Denis
On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <gkatzio...@gmail.com> wrote: > Hi all! > > I am sorry for being late on that. I was trying to refactor the test in > order not to be in need of any external tools or spinning up a server. > I did forked the new repo and indeed my changes there, so a pull request is > a matter of time! > > Kind regards > > *Emmanouil Gkatziouras* > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > https://github.com/gkatzioura > > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dma...@apache.org> wrote: > > > Hi Emmanouil, > > > > Do you have any questions or need any support from the community? > > > > - > > Denis > > > > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <saikat.mai...@gmail.com> > > wrote: > > > >> Hi Emmanouil, > >> > >> Can you please take a looks at dev utils, if this is something you are > >> looking for > >> https://github.com/apache/ignite/tree/master/modules/dev-utils? > >> > >> IMO, if you can release Pub/Sub server in maven and then use it as > >> dependency, that would be great. > >> > >> Regards, > >> Saikat > >> > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <saikat.mai...@gmail.com> > >> wrote: > >> > >> > Hi Emmanouil, > >> > > >> > The master branch has been initialized. I have raised a PR to migrate > >> > Flink module to ignite-extensions. > >> > > >> > PR https://github.com/apache/ignite-extensions/pull/1 > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356 > >> > > >> > Regards, > >> > Saikat > >> > > >> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras < > >> > gkatzio...@gmail.com> wrote: > >> > > >> >> Hi all! > >> >> > >> >> This is an update. The integration is tested with mocks based on the > >> >> original Ignite repo [1]. > >> >> I am currently waiting for the first commits on the new integration > >> >> project > >> >> to take place in order to create my pull request accordingly to the > >> >> initial > >> >> commits. > >> >> In order to simulate the Pub/Sub server in a test I used a test > server > >> >> implementation [2]. Unfortunately they don't have this util uploaded > >> in a > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache > License > >> >> 2.0). Is there a repo for utilities that Ignite uses for the > >> test/build? > >> >> I have also created a ticket [3] > >> >> > >> >> Kind regards > >> >> > >> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262 > >> >> > >> >> *Emmanouil Gkatziouras* > >> >> https://egkatzioura.com/ | > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/ > >> >> https://github.com/gkatzioura > >> >> > >> >> > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <saikat.mai...@gmail.com > > > >> >> wrote: > >> >> > >> >> > Hello Ilya, > >> >> > > >> >> > Thank you for your email. Yes, before we remove any integration we > >> will > >> >> > move them to another repository and release. > >> >> > > >> >> > We are thinking we will start with Pub/Sub Streamer implementation > >> and > >> >> > continue to move other integration as mentioned in the following > doc. > >> >> > > >> >> > > >> >> > > >> >> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > >> >> > > >> >> > Regards, > >> >> > Saikat > >> >> > > >> >> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev < > >> >> > ilya.kasnach...@gmail.com> > >> >> > wrote: > >> >> > > >> >> > > Hello! > >> >> > > > >> >> > > My take on this: we want to offload some of integrations, but > >> please > >> >> > commit > >> >> > > and release them to another project first, and only then we will > >> >> decide > >> >> > to > >> >> > > remove them from our repository. > >> >> > > > >> >> > > Please also make sure they are available with same artifact name > >> and > >> >> API. > >> >> > > If this has to be changed, let's wait for AI 3.0 > >> >> > > > >> >> > > Regards, > >> >> > > -- > >> >> > > Ilya Kasnacheev > >> >> > > > >> >> > > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra < > >> saikat.mai...@gmail.com>: > >> >> > > > >> >> > > > Hello Emmanouil, > >> >> > > > > >> >> > > > As discussed earlier, I discussed with Apache Bahir community > and > >> >> they > >> >> > > are > >> >> > > > interested to have Apache Ignite extensions as part of Apache > >> Bahir > >> >> > > > project. > >> >> > > > > >> >> > > > Can you please share Pub/Sub streamer implementation details > with > >> >> > Apache > >> >> > > > Bahir community and request for feedback. > >> >> > > > > >> >> > > > https://bahir.apache.org/ > >> >> > > > > >> >> > > > I have also requested for contributor access in Jira for Apache > >> >> Bahir > >> >> > > > project so that I can create issues and assign to myself. > >> >> > > > > >> >> > > > I can help with code reviews as well. > >> >> > > > > >> >> > > > Here is the email thread for reference > >> >> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html > >> >> > > > > >> >> > > > Regards, > >> >> > > > Saikat > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra < > >> >> saikat.mai...@gmail.com > >> >> > > > >> >> > > > wrote: > >> >> > > > > >> >> > > > > Hello, > >> >> > > > > > >> >> > > > > As discussed I have created following discussion threads on > our > >> >> > > migration > >> >> > > > > proposals for Apache Ignite extensions: > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html > >> >> > > > > > >> >> > > > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html > >> >> > > > > > >> >> > > > > I will share update as I hear more information. > >> >> > > > > > >> >> > > > > Regards, > >> >> > > > > Saikat > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra < > >> >> > saikat.mai...@gmail.com > >> >> > > > > >> >> > > > > wrote: > >> >> > > > > > >> >> > > > >> Hello Denis, > >> >> > > > >> > >> >> > > > >> Sure, sounds good. I will create a separate discussion > thread > >> to > >> >> get > >> >> > > > >> community feedback on whether we > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" > project. > >> >> > > > >> > >> >> > > > >> Regards, > >> >> > > > >> Saikat > >> >> > > > >> > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda < > >> dma...@apache.org> > >> >> > > wrote: > >> >> > > > >> > >> >> > > > >>> Folks, > >> >> > > > >>> > >> >> > > > >>> The concept of Apache Bahir is precisely what we need! > >> Saikat, > >> >> > thanks > >> >> > > > for > >> >> > > > >>> doing the research. Why I believe the * idea* fits us well: > >> >> > > > >>> > >> >> > > > >>> - All integrations can be stored in separate Github > >> >> repositories > >> >> > > and > >> >> > > > >>> have their dev lifecycles. We've not obliged to couple > all > >> >> the > >> >> > > > >>> integrations > >> >> > > > >>> in a single repo. For instance, Spark can be located in > a > >> >> > > dedicated > >> >> > > > >>> repo > >> >> > > > >>> while streaming integrations might be in a single one. > >> It's > >> >> up > >> >> > to > >> >> > > > us. > >> >> > > > >>> - All the repositories and integrations belong to ASF. > >> We're > >> >> not > >> >> > > > >>> dumping > >> >> > > > >>> them on Github but rather keep supporting and developing > >> in > >> >> > > > >>> accordance with > >> >> > > > >>> ASF vision and practices. > >> >> > > > >>> - There is a way to reward contributors via > committership > >> and > >> >> > PMC > >> >> > > > >>> membership. You won't get this if a project is just one > of > >> >> the > >> >> > > > >>> millions of > >> >> > > > >>> Github projects. > >> >> > > > >>> > >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a > >> >> separate > >> >> > > > >>> dev-list > >> >> > > > >>> discussion to involve more community members and referring > to > >> >> this > >> >> > > > >>> thread? > >> >> > > > >>> I think the primary question the community needs to answer > >> >> whether > >> >> > we > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions" > >> project? > >> >> > > > >>> > >> >> > > > >>> - > >> >> > > > >>> Denis > >> >> > > > >>> > >> >> > > > >>> > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev < > >> >> > > > zaleslaw....@gmail.com> > >> >> > > > >>> wrote: > >> >> > > > >>> > >> >> > > > >>> > Maybe we could move all our Streaming Integrations there, > >> but > >> >> > what > >> >> > > is > >> >> > > > >>> about > >> >> > > > >>> > maintaining and committer permissions to the new > >> repositories? > >> >> > > > >>> > I see the list of the committers and PMC members there > >> >> > > > >>> > https://bahir.apache.org/community-members/ > >> >> > > > >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to this > list > >> as > >> >> a > >> >> > > > >>> > PMC/committer to maintain Ignite integrations? > >> >> > > > >>> > If yes, I'd happy to join this project and collaborate > with > >> >> these > >> >> > > > guys > >> >> > > > >>> > together > >> >> > > > >>> > If no, and we should start from the zero level with > >> external > >> >> PRs > >> >> > - > >> >> > > > >>> hmmm, > >> >> > > > >>> > it's better to have our own external repository with > >> >> ApacheBahirr > >> >> > > > >>> ideology. > >> >> > > > >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved there > (in > >> >> > > > >>> ApacheBahirr > >> >> > > > >>> > like repository), but not all modules from the page > >> >> > > > >>> > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > >> >> > > > >>> > because > >> >> > > > >>> > they use different parts of core modules and could be > >> >> developed > >> >> > > with > >> >> > > > >>> > different velocity. > >> >> > > > >>> > > >> >> > > > >>> > In reality, before creation of new repositories we need > >> wide > >> >> > > > discussion > >> >> > > > >>> > based on the document mentioned above. > >> >> > > > >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next release > 2.8 > >> >> with > >> >> > > one > >> >> > > > >>> or two > >> >> > > > >>> > integrations like twitter or ZeroMQ > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate > >> repositories > >> >> in > >> >> > > > Apache > >> >> > > > >>> > Github to maintain them by anybody. > >> >> > > > >>> > > >> >> > > > >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra < > >> >> > > saikat.mai...@gmail.com > >> >> > > > >: > >> >> > > > >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil > >> >> > > > >>> > > > >> >> > > > >>> > > Thank you for your email. I think creating a separate > >> >> > integration > >> >> > > > >>> project > >> >> > > > >>> > > in github and also proposing it as an apache incubator > >> >> project > >> >> > > will > >> >> > > > >>> be a > >> >> > > > >>> > > good move. The other separate integration modules > >> >> > > > >>> > > > >> >> > > > >>> > > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > >> >> > > > >>> > > can > >> >> > > > >>> > > be moved to this new apache incubator project. > >> >> > > > >>> > > > >> >> > > > >>> > > There are similar integration available for Flink and > >> Spark > >> >> in > >> >> > > > Apache > >> >> > > > >>> > Bahir > >> >> > > > >>> > > https://bahir.apache.org/ > >> >> > > > >>> > > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir > >> community > >> >> > with a > >> >> > > > >>> > proposal > >> >> > > > >>> > > or we should plan to create a separate Apache Incubator > >> >> > project? > >> >> > > > >>> > > > >> >> > > > >>> > > Regards, > >> >> > > > >>> > > Saikat > >> >> > > > >>> > > > >> >> > > > >>> > > > >> >> > > > >>> > > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras < > >> >> > > > >>> > > gkatzio...@gmail.com> > >> >> > > > >>> > > wrote: > >> >> > > > >>> > > > >> >> > > > >>> > > > Hi all! > >> >> > > > >>> > > > > >> >> > > > >>> > > > Based on the discussions most probably this is going > to > >> >> be on > >> >> > > > >>> another > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone > >> project > >> >> > with > >> >> > > > the > >> >> > > > >>> > > Pub/Sub > >> >> > > > >>> > > > module and once the repository is created a pull > >> request > >> >> > shall > >> >> > > be > >> >> > > > >>> > issued. > >> >> > > > >>> > > > If there is anything else I can do in order to > >> facilitate > >> >> > this > >> >> > > > >>> please > >> >> > > > >>> > let > >> >> > > > >>> > > > me know. > >> >> > > > >>> > > > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be > able > >> to > >> >> > > create > >> >> > > > a > >> >> > > > >>> pull > >> >> > > > >>> > > > request and thus a build will take place on ignite's > >> CI. > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub > >> >> implementation > >> >> > > may I > >> >> > > > >>> > create > >> >> > > > >>> > > a > >> >> > > > >>> > > > ticket on JIRA? > >> >> > > > >>> > > > > >> >> > > > >>> > > > My next question has to do on how Ignite configures > the > >> >> build > >> >> > > > jobs > >> >> > > > >>> in > >> >> > > > >>> > > Team > >> >> > > > >>> > > > City. The unit tests for my implementation need a > local > >> >> > Pub/Sub > >> >> > > > >>> server > >> >> > > > >>> > up > >> >> > > > >>> > > > and running (they provide one for testing purposes). > >> Will > >> >> > this > >> >> > > > take > >> >> > > > >>> > > effect > >> >> > > > >>> > > > in a form of a build script inside my implementation > or > >> >> is it > >> >> > > > >>> something > >> >> > > > >>> > > > that should be configured on Team City? > >> >> > > > >>> > > > > >> >> > > > >>> > > > Kind regards, > >> >> > > > >>> > > > Emmanouil > >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > *Emmanouil Gkatziouras* > >> >> > > > >>> > > > https://egkatzioura.com/ | > >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > >> >> > > > >>> > > > https://github.com/gkatzioura > >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda < > >> >> dma...@apache.org > >> >> > > > >> >> > > > >>> wrote: > >> >> > > > >>> > > > > >> >> > > > >>> > > > > Emmanouil, Saikat, > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > After contemplating on this, let me propose another > >> way > >> >> > with > >> >> > > > this > >> >> > > > >>> > > > > particular integration that is aligned with our > >> >> > > modularization > >> >> > > > >>> > > endeavor. > >> >> > > > >>> > > > > The modularization [1] defines the Ignite core, > >> Ignite > >> >> > > modules, > >> >> > > > >>> and > >> >> > > > >>> > > > > independent integrations. The first two are to be > >> >> located > >> >> > in > >> >> > > > >>> Ignite > >> >> > > > >>> > > > > repositories and to be tested/updated by the > >> community > >> >> for > >> >> > > > every > >> >> > > > >>> > > release. > >> >> > > > >>> > > > > The modules are a subset of existing integrations > >> that > >> >> are > >> >> > > > >>> adopted > >> >> > > > >>> > > widely > >> >> > > > >>> > > > > and require community attention. > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > As for the independent integrations [1], those are > >> >> supposed > >> >> > > to > >> >> > > > be > >> >> > > > >>> > > located > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to > start > >> >> > > > >>> implementing > >> >> > > > >>> > this > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though > it > >> >> will > >> >> > be > >> >> > > > >>> located > >> >> > > > >>> > > in a > >> >> > > > >>> > > > > separate Github repo, it will be listed among > >> officially > >> >> > > > >>> available > >> >> > > > >>> > > > > integrations on the Ignite website featuring > >> Emmanouil > >> >> as a > >> >> > > > >>> > > contributor. > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > What do you think? > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > [1] > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > - > >> >> > > > >>> > > > > Denis > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra < > >> >> > > > >>> > saikat.mai...@gmail.com > >> >> > > > >>> > > > > >> >> > > > >>> > > > > wrote: > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > Hi Emmanouil, > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check > if > >> you > >> >> > are > >> >> > > > also > >> >> > > > >>> > able > >> >> > > > >>> > > to > >> >> > > > >>> > > > > > validate that the once data is added to Ignite > >> cluster > >> >> > you > >> >> > > > are > >> >> > > > >>> also > >> >> > > > >>> > > > able > >> >> > > > >>> > > > > to > >> >> > > > >>> > > > > > access it using another Ignite client or using > rest > >> >> > > > endpoints. > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are > as > >> >> > follows > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > Trigger validation of those test suites that have > >> been > >> >> > > > >>> affected by > >> >> > > > >>> > > your > >> >> > > > >>> > > > > > changes on TeamCity < > http://ci.ignite.apache.org/>: > >> < > >> >> > > > >>> > > > > http://204.14.53.153/ > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > - Locate a test suite you have to check, press > >> >> button > >> >> > > > named > >> >> > > > >>> > "..." > >> >> > > > >>> > > > that > >> >> > > > >>> > > > > > is located on the left of "Run" button. "Run > >> custom > >> >> > > build" > >> >> > > > >>> > dialog > >> >> > > > >>> > > > will > >> >> > > > >>> > > > > > appear; > >> >> > > > >>> > > > > > - Go to "Changes" tab and choose > >> >> > > > >>> > "pull/<pull-request-number>/head" > >> >> > > > >>> > > > in > >> >> > > > >>> > > > > > "Build branch" dropdown list; > >> >> > > > >>> > > > > > - Press "Run build" button and monitor tests > >> >> results. > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > @Roman rsht...@yahoo.com - can you also please > >> take a > >> >> > > look? > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > Regards, > >> >> > > > >>> > > > > > Saikat > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil > >> Gkatziouras > >> >> < > >> >> > > > >>> > > > > > gkatzio...@gmail.com> > >> >> > > > >>> > > > > > wrote: > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on > my > >> >> > personal > >> >> > > > >>> GitHub > >> >> > > > >>> > > > > account > >> >> > > > >>> > > > > > > [1] > >> >> > > > >>> > > > > > > It is based on the Kafka implementation which > is > >> of > >> >> a > >> >> > > > polling > >> >> > > > >>> > > nature. > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka. > >> Instead > >> >> it > >> >> > > > >>> provides us > >> >> > > > >>> > > > with > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka. > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a > guideline > >> >> made > >> >> > > more > >> >> > > > >>> sense > >> >> > > > >>> > > to > >> >> > > > >>> > > > > me. > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > Apart from the implementation I have also > added a > >> >> unit > >> >> > > > test. > >> >> > > > >>> > > > > > > It definitely needs some polishing and a double > >> >> check > >> >> > on > >> >> > > my > >> >> > > > >>> side > >> >> > > > >>> > to > >> >> > > > >>> > > > > make > >> >> > > > >>> > > > > > > the review smooth. > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to > >> be > >> >> > setup > >> >> > > > >>> however > >> >> > > > >>> > I > >> >> > > > >>> > > am > >> >> > > > >>> > > > > not > >> >> > > > >>> > > > > > > sure which is the best way to do so with > regards > >> to > >> >> > your > >> >> > > > >>> CI/CD. > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > [1] > >> >> > > > >>> > > > >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub > >> >> > > > >>> > > > > > > [2] > >> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/ > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras* > >> >> > > > >>> > > > > > > https://egkatzioura.com/ | > >> >> > > > >>> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/ > >> >> > > > >>> > > > > > > https://github.com/gkatzioura > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra < > >> >> > > > >>> > > saikat.mai...@gmail.com > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > > wrote: > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > Hello Emmanouil, > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > Can you please share more info about the > >> >> > > implementation? > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > We have similar implementation for data > >> streamers > >> >> > with > >> >> > > > >>> Kafka > >> >> > > > >>> > and > >> >> > > > >>> > > > > > RocketMQ > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > Please review and share if you have any > >> questions. > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > Regards, > >> >> > > > >>> > > > > > > > Saikat > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil > >> >> > Gkatziouras < > >> >> > > > >>> > > > > > > > gkatzio...@gmail.com> > >> >> > > > >>> > > > > > > > wrote: > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > Hi all, > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's > >> >> Pub/Sub > >> >> > > [1] > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the > >> >> > messaging > >> >> > > > >>> service > >> >> > > > >>> > > > > provided > >> >> > > > >>> > > > > > > by > >> >> > > > >>> > > > > > > > > GCP. > >> >> > > > >>> > > > > > > > > I have made the integration with it, but I > >> >> haven't > >> >> > > > >>> published > >> >> > > > >>> > it > >> >> > > > >>> > > > yet > >> >> > > > >>> > > > > > (I > >> >> > > > >>> > > > > > > am > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and > proceed > >> >> with > >> >> > > the > >> >> > > > >>> > > > > contribution). > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > Kind regards, > >> >> > > > >>> > > > > > > > > Emmanouil > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > [1] > >> >> https://cloud.google.com/pubsub/docs/overview > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras* > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ | > >> >> > > > >>> > > > > > > > > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/ > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda < > >> >> > > > >>> dma...@apache.org > >> >> > > > >>> > > > >> >> > > > >>> > > > > wrote: > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil, > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific > >> >> implementation > >> >> > > > >>> tailored > >> >> > > > >>> > > for a > >> >> > > > >>> > > > > > > cloud? > >> >> > > > >>> > > > > > > > If > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning > >> AWS/GCE. > >> >> Is > >> >> > > > there > >> >> > > > >>> any > >> >> > > > >>> > > > Github > >> >> > > > >>> > > > > > > > source > >> >> > > > >>> > > > > > > > > > code? > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations > >> with > >> >> > > > streaming > >> >> > > > >>> and > >> >> > > > >>> > > > > pub/sub > >> >> > > > >>> > > > > > > > > > frameworks: > >> >> > > > >>> > > > > > > > > > > >> >> https://apacheignite-mix.readme.io/docs/overview > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite master: > >> >> > > > >>> > > > > > > > > > > >> >> > > https://github.com/apache/ignite/tree/master/modules > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > Check out those resources to see what are > >> the > >> >> > most > >> >> > > > >>> suitable > >> >> > > > >>> > > > > Ignite > >> >> > > > >>> > > > > > > APIs > >> >> > > > >>> > > > > > > > > for > >> >> > > > >>> > > > > > > > > > such integrations. > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <saikat.mai...@gmail.com> > , > >> >> > @Roman > >> >> > > < > >> >> > > > >>> > > > > > rsht...@yahoo.com > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > , > >> >> > > > >>> > > > > > > > > as > >> >> > > > >>> > > > > > > > > > streaming technologies > >> >> contributors/maintainers, > >> >> > > > could > >> >> > > > >>> you > >> >> > > > >>> > > > please > >> >> > > > >>> > > > > > > join > >> >> > > > >>> > > > > > > > > this > >> >> > > > >>> > > > > > > > > > thread? > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > - > >> >> > > > >>> > > > > > > > > > Denis > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil > >> >> > > > Gkatziouras < > >> >> > > > >>> > > > > > > > > > gkatzio...@gmail.com> > >> >> > > > >>> > > > > > > > > > wrote: > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > Hello everyone, > >> >> > > > >>> > > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of > my > >> >> work > >> >> > > > >>> involves > >> >> > > > >>> > > > > Pub/Sub, > >> >> > > > >>> > > > > > > > thus I > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer. > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka. > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every > node > >> >> > creates > >> >> > > > an > >> >> > > > >>> > > instance > >> >> > > > >>> > > > > of > >> >> > > > >>> > > > > > a > >> >> > > > >>> > > > > > > > > > Pub/Sub > >> >> > > > >>> > > > > > > > > > > client. > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic > >> >> using > >> >> > the > >> >> > > > >>> same > >> >> > > > >>> > > > > > subscription > >> >> > > > >>> > > > > > > > > name. > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of the > >> nodes > >> >> > will > >> >> > > > >>> receive > >> >> > > > >>> > > the > >> >> > > > >>> > > > > > > > message, > >> >> > > > >>> > > > > > > > > > > process it and then forward it to > Ignite > >> >> using > >> >> > > the > >> >> > > > >>> > > extractor > >> >> > > > >>> > > > > > > > provided. > >> >> > > > >>> > > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > Any thoughts? > >> >> > > > >>> > > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras* > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ | > >> >> > > > >>> > > > > > > > > > > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/ > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura > >> >> > > > >>> > > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > >> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > > >> > > >
