Hi Emmanouil, The changes looks good to me. I wanted to check if you are also able to validate that the once data is added to Ignite cluster you are also able to access it using another Ignite client or using rest endpoints.
We use Teamcity for CI process, the details are as follows Trigger validation of those test suites that have been affected by your changes on TeamCity <http://ci.ignite.apache.org/>: <http://204.14.53.153/> - Locate a test suite you have to check, press button named "..." that is located on the left of "Run" button. "Run custom build" dialog will appear; - Go to "Changes" tab and choose "pull/<pull-request-number>/head" in "Build branch" dropdown list; - Press "Run build" button and monitor tests results. @Roman rsht...@yahoo.com - can you also please take a look? Regards, Saikat On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <gkatzio...@gmail.com> wrote: > Hi @Saikat Maitra > > I have my implementation on an Ignite fork on my personal GitHub account > [1] > It is based on the Kafka implementation which is of a polling nature. > GCP does not provide a service like Kafka. Instead it provides us with > Pub/Sub which is just like Kafka. > Therefore using the Kafka streamer as a guideline made more sense to me. > > Apart from the implementation I have also added a unit test. > It definitely needs some polishing and a double check on my side to make > the review smooth. > For the tests a local Pub/Sub [2] server has to be setup however I am not > sure which is the best way to do so with regards to your CI/CD. > > [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub > [2] https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/ > > *Emmanouil Gkatziouras* > https://egkatzioura.com/ | > https://www.linkedin.com/in/gkatziourasemmanouil/ > https://github.com/gkatzioura > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <saikat.mai...@gmail.com> > wrote: > > > Hello Emmanouil, > > > > Can you please share more info about the implementation? > > > > We have similar implementation for data streamers with Kafka and RocketMQ > > > > https://apacheignite-mix.readme.io/docs/kafka-streamer > > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer > > > > Please review and share if you have any questions. > > > > Regards, > > Saikat > > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras < > > gkatzio...@gmail.com> > > wrote: > > > > > Hi all, > > > > > > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1] > > > Yes it is indeed cloud specific. It is the messaging service provided > by > > > GCP. > > > I have made the integration with it, but I haven't published it yet (I > am > > > waiting for the discussion/ticket and proceed with the contribution). > > > > > > Kind regards, > > > Emmanouil > > > > > > [1] https://cloud.google.com/pubsub/docs/overview > > > > > > *Emmanouil Gkatziouras* > > > https://egkatzioura.com/ | > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > https://github.com/gkatzioura > > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <dma...@apache.org> wrote: > > > > > > > Hello Emmanouil, > > > > > > > > Are you referring to any specific implementation tailored for a > cloud? > > If > > > > I'm not mistaken you were mentioning AWS/GCE. Is there any Github > > source > > > > code? > > > > > > > > Here is a list of existing integrations with streaming and pub/sub > > > > frameworks: > > > > https://apacheignite-mix.readme.io/docs/overview > > > > > > > > with the source code in Ignite master: > > > > https://github.com/apache/ignite/tree/master/modules > > > > > > > > Check out those resources to see what are the most suitable Ignite > APIs > > > for > > > > such integrations. > > > > > > > > @Saikat Maitra <saikat.mai...@gmail.com> , @Roman <rsht...@yahoo.com > > > > , > > > as > > > > streaming technologies contributors/maintainers, could you please > join > > > this > > > > thread? > > > > > > > > - > > > > Denis > > > > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras < > > > > gkatzio...@gmail.com> > > > > wrote: > > > > > > > > > Hello everyone, > > > > > > > > > > I started using Ignite lately. Part of my work involves Pub/Sub, > > thus I > > > > > created a pub/sub streamer. > > > > > Pub/Sub is pretty close to Kafka. > > > > > Here's how the streamer works. Every node creates an instance of a > > > > Pub/Sub > > > > > client. > > > > > The clients subscribe to the same topic using the same subscription > > > name. > > > > > A message is being sent and one of the nodes will receive the > > message, > > > > > process it and then forward it to Ignite using the extractor > > provided. > > > > > > > > > > Any thoughts? > > > > > > > > > > *Emmanouil Gkatziouras* > > > > > https://egkatzioura.com/ | > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/ > > > > > https://github.com/gkatzioura > > > > > > > > > > > > > > >
