Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Ivan,

Yes, we will need to remove flink module from main ignite repo.

I can do it as part of this issue.

IGNITE-12356 Migrate Flink module to ignite-extensions

Regards,
Saikat

On Mon, Dec 30, 2019 at 12:54 AM Ivan Pavlukhin  wrote:

> Folks,
>
> Do not we need to remove flink integration sources from main ignite
> repository [1]? Is this work in progress?
>
> [1] https://github.com/apache/ignite
>
> вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras :
> >
> > Hi Saikat!
> >
> > Thank you for your time and assistance.
> >
> > Kind regards,
> > Emmanouil
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 29 Dec 2019 at 18:35, Saikat Maitra 
> wrote:
> >
> > > Hi Emmanouil,
> > >
> > > Thank you for your contribution. I have merged your changes in master.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra <
> saikat.mai...@gmail.com>
> > > wrote:
> > >
> > > > Thank you, Emmanouil
> > > >
> > > > I am reviewing the changes.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com> wrote:
> > > >
> > > >> Hi Saikat!
> > > >>
> > > >> The dependencies have been updated, and the merge conflicts
> resolved.
> > > >> Kind regards
> > > >>
> > > >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, 
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > Thank you for making the changes. I have merged my PR changes in
> > > master.
> > > >> > Can you please take the latest from master branch and resolve the
> > > >> conflicts
> > > >> > in your PR.
> > > >> >
> > > >> > I have also reviewed the changes in your PR and requested minor
> > > changes
> > > >> > specific to dependencies version and license information.
> > > >> >
> > > >> > Can you please review and share your feedback?
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > > >> > gkatzio...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi Saikat!
> > > >> > >
> > > >> > > Thank you on this one! I tried it and it worked as expected [1]
> > > >> > >
> > > >> > > Kind regards
> > > >> > > Emmanouil
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
> > > >> > >
> > > >> > > *Emmanouil Gkatziouras*
> > > >> > > https://egkatzioura.com/ |
> > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > https://github.com/gkatzioura
> > > >> > >
> > > >> > >
> > > >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> > > saikat.mai...@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Emmanouil,
> > > >> > > >
> > > >> > > > I have been able to resolve the build failures for missing
> > > modules.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> > > >> > > >
> > > >> > > > Will you please take the latest changes from my PR and run the
> > > build
> > > >> > for
> > > >> > > > your PR.
> > > >> > > >
> > > >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > > >> > > >
> > > >> > > > The new build configurations should let your module build also
> > > pass.
> > > >> > > >
> > > >> > > > The issue that was causing failure was that I had put
> ignite.zip
> > > as
> > > >> > > > artifact dependency and it was downloading the artifacts
> including
> > > >> pom
> > > >> > > file
> > > >> > > > inside the build checkout folder of ignite-extensions and as a
> > > >> result
> > > >> > the
> > > >> > > > pom.xml of ignite-extensions was getting overwritten by
> ignite's
> > > >> > pom.xml.
> > > >> > > > This was resulting in missing flink-ext module error as it
> was not
> > > >> even
> > > >> > > > present in the ignite main pom.xml.
> > > >> > > >
> > > >> > > > To resolve the issue I just had to download and extract
> artifact
> > > of
> > > >> > > > ignite.zip in a separate directory.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Saikat
> > > >> > > >
> > > >> > > >
> > > >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > >> > > > gkatzio...@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi Saikat!
> > > >> > > > >
> > > >> > > > > Thank you for your assistance on that!
> > > >> > > > >
> > > >> > > > > Kind regards
> > > >> > > > >
> > > >> > > > > *Emmanouil Gkatziouras*
> > > >> > > > > https://egkatzioura.com/ |
> > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > https://github.com/gkatzioura
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> > > >> saikat.mai...@gmail.com
> > > >> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Ivan Pavlukhin]

Folks,

Do not we need to remove flink integration sources from main ignite
repository [1]? Is this work in progress?

[1] https://github.com/apache/ignite

вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras :
>
> Hi Saikat!
>
> Thank you for your time and assistance.
>
> Kind regards,
> Emmanouil
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 29 Dec 2019 at 18:35, Saikat Maitra  wrote:
>
> > Hi Emmanouil,
> >
> > Thank you for your contribution. I have merged your changes in master.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra 
> > wrote:
> >
> > > Thank you, Emmanouil
> > >
> > > I am reviewing the changes.
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com> wrote:
> > >
> > >> Hi Saikat!
> > >>
> > >> The dependencies have been updated, and the merge conflicts resolved.
> > >> Kind regards
> > >>
> > >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, 
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > Thank you for making the changes. I have merged my PR changes in
> > master.
> > >> > Can you please take the latest from master branch and resolve the
> > >> conflicts
> > >> > in your PR.
> > >> >
> > >> > I have also reviewed the changes in your PR and requested minor
> > changes
> > >> > specific to dependencies version and license information.
> > >> >
> > >> > Can you please review and share your feedback?
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > >> > gkatzio...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Saikat!
> > >> > >
> > >> > > Thank you on this one! I tried it and it worked as expected [1]
> > >> > >
> > >> > > Kind regards
> > >> > > Emmanouil
> > >> > >
> > >> > > [1]
> > >> > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
> > >> > >
> > >> > > *Emmanouil Gkatziouras*
> > >> > > https://egkatzioura.com/ |
> > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > https://github.com/gkatzioura
> > >> > >
> > >> > >
> > >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> > saikat.mai...@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Emmanouil,
> > >> > > >
> > >> > > > I have been able to resolve the build failures for missing
> > modules.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> > >> > > >
> > >> > > > Will you please take the latest changes from my PR and run the
> > build
> > >> > for
> > >> > > > your PR.
> > >> > > >
> > >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > >> > > >
> > >> > > > The new build configurations should let your module build also
> > pass.
> > >> > > >
> > >> > > > The issue that was causing failure was that I had put ignite.zip
> > as
> > >> > > > artifact dependency and it was downloading the artifacts including
> > >> pom
> > >> > > file
> > >> > > > inside the build checkout folder of ignite-extensions and as a
> > >> result
> > >> > the
> > >> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> > >> > pom.xml.
> > >> > > > This was resulting in missing flink-ext module error as it was not
> > >> even
> > >> > > > present in the ignite main pom.xml.
> > >> > > >
> > >> > > > To resolve the issue I just had to download and extract artifact
> > of
> > >> > > > ignite.zip in a separate directory.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Saikat
> > >> > > >
> > >> > > >
> > >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > >> > > > gkatzio...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Saikat!
> > >> > > > >
> > >> > > > > Thank you for your assistance on that!
> > >> > > > >
> > >> > > > > Kind regards
> > >> > > > >
> > >> > > > > *Emmanouil Gkatziouras*
> > >> > > > > https://egkatzioura.com/ |
> > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > https://github.com/gkatzioura
> > >> > > > >
> > >> > > > >
> > >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> > >> saikat.mai...@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Emmanouil,
> > >> > > > > >
> > >> > > > > > I looked into the build logs and I observed that since I added
> > >> > > > artifacts
> > >> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull
> > the
> > >> > > > required
> > >> > > > > > dependencies and was able to run the existing tests. It is
> > >> however
> > >> > > not
> > >> > > > > > identifying new modules like I changed from flink to flink-ext
> > >> or
> > >> > > > > pub-sub.
> > >> > > > > >
> > >> > > > > > [1]
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

Thank you for your time and assistance.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 29 Dec 2019 at 18:35, Saikat Maitra  wrote:

> Hi Emmanouil,
>
> Thank you for your contribution. I have merged your changes in master.
>
> Regards,
> Saikat
>
>
> On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra 
> wrote:
>
> > Thank you, Emmanouil
> >
> > I am reviewing the changes.
> >
> > Regards,
> > Saikat
> >
> > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com> wrote:
> >
> >> Hi Saikat!
> >>
> >> The dependencies have been updated, and the merge conflicts resolved.
> >> Kind regards
> >>
> >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, 
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > Thank you for making the changes. I have merged my PR changes in
> master.
> >> > Can you please take the latest from master branch and resolve the
> >> conflicts
> >> > in your PR.
> >> >
> >> > I have also reviewed the changes in your PR and requested minor
> changes
> >> > specific to dependencies version and license information.
> >> >
> >> > Can you please review and share your feedback?
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> >
> >> >
> >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> >> > gkatzio...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi Saikat!
> >> > >
> >> > > Thank you on this one! I tried it and it worked as expected [1]
> >> > >
> >> > > Kind regards
> >> > > Emmanouil
> >> > >
> >> > > [1]
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
> >> > >
> >> > > *Emmanouil Gkatziouras*
> >> > > https://egkatzioura.com/ |
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > https://github.com/gkatzioura
> >> > >
> >> > >
> >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> saikat.mai...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Emmanouil,
> >> > > >
> >> > > > I have been able to resolve the build failures for missing
> modules.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> >> > > >
> >> > > > Will you please take the latest changes from my PR and run the
> build
> >> > for
> >> > > > your PR.
> >> > > >
> >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> >> > > >
> >> > > > The new build configurations should let your module build also
> pass.
> >> > > >
> >> > > > The issue that was causing failure was that I had put ignite.zip
> as
> >> > > > artifact dependency and it was downloading the artifacts including
> >> pom
> >> > > file
> >> > > > inside the build checkout folder of ignite-extensions and as a
> >> result
> >> > the
> >> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> >> > pom.xml.
> >> > > > This was resulting in missing flink-ext module error as it was not
> >> even
> >> > > > present in the ignite main pom.xml.
> >> > > >
> >> > > > To resolve the issue I just had to download and extract artifact
> of
> >> > > > ignite.zip in a separate directory.
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > >
> >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> >> > > > gkatzio...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Saikat!
> >> > > > >
> >> > > > > Thank you for your assistance on that!
> >> > > > >
> >> > > > > Kind regards
> >> > > > >
> >> > > > > *Emmanouil Gkatziouras*
> >> > > > > https://egkatzioura.com/ |
> >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > https://github.com/gkatzioura
> >> > > > >
> >> > > > >
> >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> >> saikat.mai...@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Emmanouil,
> >> > > > > >
> >> > > > > > I looked into the build logs and I observed that since I added
> >> > > > artifacts
> >> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull
> the
> >> > > > required
> >> > > > > > dependencies and was able to run the existing tests. It is
> >> however
> >> > > not
> >> > > > > > identifying new modules like I changed from flink to flink-ext
> >> or
> >> > > > > pub-sub.
> >> > > > > >
> >> > > > > > [1]
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> >> > > > > >
> >> > > > > > I will look into the issue and debug further.
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > > Saikat
> >> > > > > >
> >> > > > > >
> >> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> >> > > > > > gkatzio...@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Saikat!
> >> > > > > > >
> >> > > > > > > From the logs[1] it seems that TC cannot find the project.

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

Thank you for your contribution. I have merged your changes in master.

Regards,
Saikat


On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra 
wrote:

> Thank you, Emmanouil
>
> I am reviewing the changes.
>
> Regards,
> Saikat
>
> On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com> wrote:
>
>> Hi Saikat!
>>
>> The dependencies have been updated, and the merge conflicts resolved.
>> Kind regards
>>
>> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, 
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > Thank you for making the changes. I have merged my PR changes in master.
>> > Can you please take the latest from master branch and resolve the
>> conflicts
>> > in your PR.
>> >
>> > I have also reviewed the changes in your PR and requested minor changes
>> > specific to dependencies version and license information.
>> >
>> > Can you please review and share your feedback?
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> >
>> >
>> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
>> > gkatzio...@gmail.com>
>> > wrote:
>> >
>> > > Hi Saikat!
>> > >
>> > > Thank you on this one! I tried it and it worked as expected [1]
>> > >
>> > > Kind regards
>> > > Emmanouil
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra 
>> > > wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > I have been able to resolve the build failures for missing modules.
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
>> > > >
>> > > > Will you please take the latest changes from my PR and run the build
>> > for
>> > > > your PR.
>> > > >
>> > > > https://github.com/apache/ignite-extensions/pull/1/files
>> > > >
>> > > > The new build configurations should let your module build also pass.
>> > > >
>> > > > The issue that was causing failure was that I had put ignite.zip as
>> > > > artifact dependency and it was downloading the artifacts including
>> pom
>> > > file
>> > > > inside the build checkout folder of ignite-extensions and as a
>> result
>> > the
>> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
>> > pom.xml.
>> > > > This was resulting in missing flink-ext module error as it was not
>> even
>> > > > present in the ignite main pom.xml.
>> > > >
>> > > > To resolve the issue I just had to download and extract artifact of
>> > > > ignite.zip in a separate directory.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
>> > > > gkatzio...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Saikat!
>> > > > >
>> > > > > Thank you for your assistance on that!
>> > > > >
>> > > > > Kind regards
>> > > > >
>> > > > > *Emmanouil Gkatziouras*
>> > > > > https://egkatzioura.com/ |
>> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > https://github.com/gkatzioura
>> > > > >
>> > > > >
>> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
>> saikat.mai...@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Emmanouil,
>> > > > > >
>> > > > > > I looked into the build logs and I observed that since I added
>> > > > artifacts
>> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
>> > > > required
>> > > > > > dependencies and was able to run the existing tests. It is
>> however
>> > > not
>> > > > > > identifying new modules like I changed from flink to flink-ext
>> or
>> > > > > pub-sub.
>> > > > > >
>> > > > > > [1]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
>> > > > > >
>> > > > > > I will look into the issue and debug further.
>> > > > > >
>> > > > > > Regards,
>> > > > > > Saikat
>> > > > > >
>> > > > > >
>> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
>> > > > > > gkatzio...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Saikat!
>> > > > > > >
>> > > > > > > From the logs[1] it seems that TC cannot find the project.
>> > However
>> > > I
>> > > > > did
>> > > > > > > include the project on the parent pom (modules sections) [2]
>> > > > > > > I tried to reproduce locally unfortunately I am not aware of
>> the
>> > > full
>> > > > > > > context on TC. It seems as if a different parent pom is being
>> > > picked
>> > > > > up.
>> > > > > > >
>> > > > > > > Kind regards
>> > > > > > >
>> > > > > > > [1]
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
>> > > > > > > [2]
>> > > > > > >
>> > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Thank you, Emmanouil

I am reviewing the changes.

Regards,
Saikat

On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras 
wrote:

> Hi Saikat!
>
> The dependencies have been updated, and the merge conflicts resolved.
> Kind regards
>
> On Sat, 28 Dec 2019, 23:27 Saikat Maitra,  wrote:
>
> > Hi Emmanouil,
> >
> > Thank you for making the changes. I have merged my PR changes in master.
> > Can you please take the latest from master branch and resolve the
> conflicts
> > in your PR.
> >
> > I have also reviewed the changes in your PR and requested minor changes
> > specific to dependencies version and license information.
> >
> > Can you please review and share your feedback?
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > Thank you on this one! I tried it and it worked as expected [1]
> > >
> > > Kind regards
> > > Emmanouil
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra 
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I have been able to resolve the build failures for missing modules.
> > > >
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> > > >
> > > > Will you please take the latest changes from my PR and run the build
> > for
> > > > your PR.
> > > >
> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > > >
> > > > The new build configurations should let your module build also pass.
> > > >
> > > > The issue that was causing failure was that I had put ignite.zip as
> > > > artifact dependency and it was downloading the artifacts including
> pom
> > > file
> > > > inside the build checkout folder of ignite-extensions and as a result
> > the
> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> > pom.xml.
> > > > This was resulting in missing flink-ext module error as it was not
> even
> > > > present in the ignite main pom.xml.
> > > >
> > > > To resolve the issue I just had to download and extract artifact of
> > > > ignite.zip in a separate directory.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > Thank you for your assistance on that!
> > > > >
> > > > > Kind regards
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi Emmanouil,
> > > > > >
> > > > > > I looked into the build logs and I observed that since I added
> > > > artifacts
> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > > > required
> > > > > > dependencies and was able to run the existing tests. It is
> however
> > > not
> > > > > > identifying new modules like I changed from flink to flink-ext or
> > > > > pub-sub.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > > > >
> > > > > > I will look into the issue and debug further.
> > > > > >
> > > > > > Regards,
> > > > > > Saikat
> > > > > >
> > > > > >
> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > > > gkatzio...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Saikat!
> > > > > > >
> > > > > > > From the logs[1] it seems that TC cannot find the project.
> > However
> > > I
> > > > > did
> > > > > > > include the project on the parent pom (modules sections) [2]
> > > > > > > I tried to reproduce locally unfortunately I am not aware of
> the
> > > full
> > > > > > > context on TC. It seems as if a different parent pom is being
> > > picked
> > > > > up.
> > > > > > >
> > > > > > > Kind regards
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > > > >
> > > > > > > *Emmanouil Gkatziouras*
> > > > > > > https://egkatzioura.com/ |
> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > https://github.com/gkatzioura
> > > > > > >
> > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

The dependencies have been updated, and the merge conflicts resolved.
Kind regards

On Sat, 28 Dec 2019, 23:27 Saikat Maitra,  wrote:

> Hi Emmanouil,
>
> Thank you for making the changes. I have merged my PR changes in master.
> Can you please take the latest from master branch and resolve the conflicts
> in your PR.
>
> I have also reviewed the changes in your PR and requested minor changes
> specific to dependencies version and license information.
>
> Can you please review and share your feedback?
>
> Regards,
> Saikat
>
>
>
>
> On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > Thank you on this one! I tried it and it worked as expected [1]
> >
> > Kind regards
> > Emmanouil
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra 
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I have been able to resolve the build failures for missing modules.
> > >
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> > >
> > > Will you please take the latest changes from my PR and run the build
> for
> > > your PR.
> > >
> > > https://github.com/apache/ignite-extensions/pull/1/files
> > >
> > > The new build configurations should let your module build also pass.
> > >
> > > The issue that was causing failure was that I had put ignite.zip as
> > > artifact dependency and it was downloading the artifacts including pom
> > file
> > > inside the build checkout folder of ignite-extensions and as a result
> the
> > > pom.xml of ignite-extensions was getting overwritten by ignite's
> pom.xml.
> > > This was resulting in missing flink-ext module error as it was not even
> > > present in the ignite main pom.xml.
> > >
> > > To resolve the issue I just had to download and extract artifact of
> > > ignite.zip in a separate directory.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > Thank you for your assistance on that!
> > > >
> > > > Kind regards
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra  >
> > > > wrote:
> > > >
> > > > > Hi Emmanouil,
> > > > >
> > > > > I looked into the build logs and I observed that since I added
> > > artifacts
> > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > > required
> > > > > dependencies and was able to run the existing tests. It is however
> > not
> > > > > identifying new modules like I changed from flink to flink-ext or
> > > > pub-sub.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > > >
> > > > > I will look into the issue and debug further.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > > gkatzio...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Saikat!
> > > > > >
> > > > > > From the logs[1] it seems that TC cannot find the project.
> However
> > I
> > > > did
> > > > > > include the project on the parent pom (modules sections) [2]
> > > > > > I tried to reproduce locally unfortunately I am not aware of the
> > full
> > > > > > context on TC. It seems as if a different parent pom is being
> > picked
> > > > up.
> > > > > >
> > > > > > Kind regards
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > > >
> > > > > > *Emmanouil Gkatziouras*
> > > > > > https://egkatzioura.com/ |
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > https://github.com/gkatzioura
> > > > > >
> > > > > >
> > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have made changes in Ignite Extensions build configurations.
> > > > > > >
> > > > > > > It is now running the tests as expected.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

Thank you for making the changes. I have merged my PR changes in master.
Can you please take the latest from master branch and resolve the conflicts
in your PR.

I have also reviewed the changes in your PR and requested minor changes
specific to dependencies version and license information.

Can you please review and share your feedback?

Regards,
Saikat




On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras 
wrote:

> Hi Saikat!
>
> Thank you on this one! I tried it and it worked as expected [1]
>
> Kind regards
> Emmanouil
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Thu, 26 Dec 2019 at 19:50, Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > I have been able to resolve the build failures for missing modules.
> >
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
> >
> > Will you please take the latest changes from my PR and run the build for
> > your PR.
> >
> > https://github.com/apache/ignite-extensions/pull/1/files
> >
> > The new build configurations should let your module build also pass.
> >
> > The issue that was causing failure was that I had put ignite.zip as
> > artifact dependency and it was downloading the artifacts including pom
> file
> > inside the build checkout folder of ignite-extensions and as a result the
> > pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
> > This was resulting in missing flink-ext module error as it was not even
> > present in the ignite main pom.xml.
> >
> > To resolve the issue I just had to download and extract artifact of
> > ignite.zip in a separate directory.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > Thank you for your assistance on that!
> > >
> > > Kind regards
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra 
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I looked into the build logs and I observed that since I added
> > artifacts
> > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > required
> > > > dependencies and was able to run the existing tests. It is however
> not
> > > > identifying new modules like I changed from flink to flink-ext or
> > > pub-sub.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > >
> > > > I will look into the issue and debug further.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > From the logs[1] it seems that TC cannot find the project. However
> I
> > > did
> > > > > include the project on the parent pom (modules sections) [2]
> > > > > I tried to reproduce locally unfortunately I am not aware of the
> full
> > > > > context on TC. It seems as if a different parent pom is being
> picked
> > > up.
> > > > >
> > > > > Kind regards
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have made changes in Ignite Extensions build configurations.
> > > > > >
> > > > > > It is now running the tests as expected.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > > > >
> > > > > > Regards,
> > > > > > Saikat
> > > > > >
> > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > > saikat.mai...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We do run build with -DskipTests and then configure selective
> > test
> > > > > suites
> > > > > > > to run.
> > > > > > >
> > > > > > > I will check why these test suites are not getting executed.
> > > > > > >
> > > > > > > Regards
> > > > > > > Saikat
> > > > > > >

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

Thank you on this one! I tried it and it worked as expected [1]

Kind regards
Emmanouil

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Thu, 26 Dec 2019 at 19:50, Saikat Maitra  wrote:

> Hi Emmanouil,
>
> I have been able to resolve the build failures for missing modules.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
>
> Will you please take the latest changes from my PR and run the build for
> your PR.
>
> https://github.com/apache/ignite-extensions/pull/1/files
>
> The new build configurations should let your module build also pass.
>
> The issue that was causing failure was that I had put ignite.zip as
> artifact dependency and it was downloading the artifacts including pom file
> inside the build checkout folder of ignite-extensions and as a result the
> pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
> This was resulting in missing flink-ext module error as it was not even
> present in the ignite main pom.xml.
>
> To resolve the issue I just had to download and extract artifact of
> ignite.zip in a separate directory.
>
> Regards,
> Saikat
>
>
> On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > Thank you for your assistance on that!
> >
> > Kind regards
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra 
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I looked into the build logs and I observed that since I added
> artifacts
> > > dependencies to "Build apache Ignite"[1] it was able to pull the
> required
> > > dependencies and was able to run the existing tests. It is however not
> > > identifying new modules like I changed from flink to flink-ext or
> > pub-sub.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > >
> > > I will look into the issue and debug further.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > From the logs[1] it seems that TC cannot find the project. However I
> > did
> > > > include the project on the parent pom (modules sections) [2]
> > > > I tried to reproduce locally unfortunately I am not aware of the full
> > > > context on TC. It seems as if a different parent pom is being picked
> > up.
> > > >
> > > > Kind regards
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > [2]
> > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra  >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I have made changes in Ignite Extensions build configurations.
> > > > >
> > > > > It is now running the tests as expected.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We do run build with -DskipTests and then configure selective
> test
> > > > suites
> > > > > > to run.
> > > > > >
> > > > > > I will check why these test suites are not getting executed.
> > > > > >
> > > > > > Regards
> > > > > > Saikat
> > > > > >
> > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > > gkatzio...@gmail.com> wrote:
> > > > > >
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I did add a test suite and run the build with the corresponding
> > > > > arguments,
> > > > > >> however the tests did not run [1].
> > > > > >> I checked the Flink's build logs and the same message is
> displayed
> > > > there
> > > > > >> too `No tests to run` [2].
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > > >> [2]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

I have been able to resolve the build failures for missing modules.

https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build

Will you please take the latest changes from my PR and run the build for
your PR.

https://github.com/apache/ignite-extensions/pull/1/files

The new build configurations should let your module build also pass.

The issue that was causing failure was that I had put ignite.zip as
artifact dependency and it was downloading the artifacts including pom file
inside the build checkout folder of ignite-extensions and as a result the
pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
This was resulting in missing flink-ext module error as it was not even
present in the ignite main pom.xml.

To resolve the issue I just had to download and extract artifact of
ignite.zip in a separate directory.

Regards,
Saikat


On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras 
wrote:

> Hi Saikat!
>
> Thank you for your assistance on that!
>
> Kind regards
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 30 Nov 2019 at 23:51, Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > I looked into the build logs and I observed that since I added artifacts
> > dependencies to "Build apache Ignite"[1] it was able to pull the required
> > dependencies and was able to run the existing tests. It is however not
> > identifying new modules like I changed from flink to flink-ext or
> pub-sub.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> >
> > I will look into the issue and debug further.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > From the logs[1] it seems that TC cannot find the project. However I
> did
> > > include the project on the parent pom (modules sections) [2]
> > > I tried to reproduce locally unfortunately I am not aware of the full
> > > context on TC. It seems as if a different parent pom is being picked
> up.
> > >
> > > Kind regards
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > [2]
> > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra 
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I have made changes in Ignite Extensions build configurations.
> > > >
> > > > It is now running the tests as expected.
> > > >
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We do run build with -DskipTests and then configure selective test
> > > suites
> > > > > to run.
> > > > >
> > > > > I will check why these test suites are not getting executed.
> > > > >
> > > > > Regards
> > > > > Saikat
> > > > >
> > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > gkatzio...@gmail.com> wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> I did add a test suite and run the build with the corresponding
> > > > arguments,
> > > > >> however the tests did not run [1].
> > > > >> I checked the Flink's build logs and the same message is displayed
> > > there
> > > > >> too `No tests to run` [2].
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > >> [2]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
> > > > >>
> > > > >> Kind regards
> > > > >> *Emmanouil Gkatziouras*
> > > > >> https://egkatzioura.com/ |
> > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> https://github.com/gkatzioura
> > > > >>
> > > > >>
> > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > saikat.mai...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > We will need to add test suites for tests to be executed in
> build.
> > > We
> > > > >> pass
> > > > >> > them as TestSuites param. I had added Flink sink and source
> > > > testsuites.
> > > > >> >
> > > > >> > Can you please review and confirm.
> > > > >> >
> > > > >> > Regards
> > > > >> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

Thank you for your assistance on that!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 23:51, Saikat Maitra  wrote:

> Hi Emmanouil,
>
> I looked into the build logs and I observed that since I added artifacts
> dependencies to "Build apache Ignite"[1] it was able to pull the required
> dependencies and was able to run the existing tests. It is however not
> identifying new modules like I changed from flink to flink-ext or pub-sub.
>
> [1]
>
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
>
> I will look into the issue and debug further.
>
> Regards,
> Saikat
>
>
> On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > From the logs[1] it seems that TC cannot find the project. However I did
> > include the project on the parent pom (modules sections) [2]
> > I tried to reproduce locally unfortunately I am not aware of the full
> > context on TC. It seems as if a different parent pom is being picked up.
> >
> > Kind regards
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > [2]
> >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra 
> > wrote:
> >
> > > Hello,
> > >
> > > I have made changes in Ignite Extensions build configurations.
> > >
> > > It is now running the tests as expected.
> > >
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra  >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We do run build with -DskipTests and then configure selective test
> > suites
> > > > to run.
> > > >
> > > > I will check why these test suites are not getting executed.
> > > >
> > > > Regards
> > > > Saikat
> > > >
> > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com> wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I did add a test suite and run the build with the corresponding
> > > arguments,
> > > >> however the tests did not run [1].
> > > >> I checked the Flink's build logs and the same message is displayed
> > there
> > > >> too `No tests to run` [2].
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > >> [2]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
> > > >>
> > > >> Kind regards
> > > >> *Emmanouil Gkatziouras*
> > > >> https://egkatzioura.com/ |
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> https://github.com/gkatzioura
> > > >>
> > > >>
> > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> saikat.mai...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > We will need to add test suites for tests to be executed in build.
> > We
> > > >> pass
> > > >> > them as TestSuites param. I had added Flink sink and source
> > > testsuites.
> > > >> >
> > > >> > Can you please review and confirm.
> > > >> >
> > > >> > Regards
> > > >> > Saikat
> > > >> >
> > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > >> > gkatzio...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi all!
> > > >> > >
> > > >> > > I was successful building and running my pull requests. It seems
> > > that
> > > >> the
> > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > >> > > If there is something I cannot do to make them discoverable
> please
> > > >> let me
> > > >> > > know.
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
> > > >> > >
> > > >> > > *Emmanouil Gkatziouras*
> > > >> > > https://egkatzioura.com/ |
> > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > https://github.com/gkatzioura
> > > >> > >
> > > >> > >
> > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > saikat.mai...@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Emmanouil,
> > > >> > > >
> > > >> > > > I have added you as contributor in Ignite Extensions project.
> > Can
> > > >> you
> > > >> > > > please check and confirm if you are able to see the project
> and
> > > >> execute
> > > >> > > > build on your 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

I looked into the build logs and I observed that since I added artifacts
dependencies to "Build apache Ignite"[1] it was able to pull the required
dependencies and was able to run the existing tests. It is however not
identifying new modules like I changed from flink to flink-ext or pub-sub.

[1]
https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build

I will look into the issue and debug further.

Regards,
Saikat


On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras 
wrote:

> Hi Saikat!
>
> From the logs[1] it seems that TC cannot find the project. However I did
> include the project on the parent pom (modules sections) [2]
> I tried to reproduce locally unfortunately I am not aware of the full
> context on TC. It seems as if a different parent pom is being picked up.
>
> Kind regards
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> [2]
>
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 30 Nov 2019 at 22:20, Saikat Maitra 
> wrote:
>
> > Hello,
> >
> > I have made changes in Ignite Extensions build configurations.
> >
> > It is now running the tests as expected.
> >
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> >
> > Regards,
> > Saikat
> >
> > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra 
> > wrote:
> >
> > > Hi,
> > >
> > > We do run build with -DskipTests and then configure selective test
> suites
> > > to run.
> > >
> > > I will check why these test suites are not getting executed.
> > >
> > > Regards
> > > Saikat
> > >
> > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com> wrote:
> > >
> > >> Hi all,
> > >>
> > >> I did add a test suite and run the build with the corresponding
> > arguments,
> > >> however the tests did not run [1].
> > >> I checked the Flink's build logs and the same message is displayed
> there
> > >> too `No tests to run` [2].
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > >> [2]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
> > >>
> > >> Kind regards
> > >> *Emmanouil Gkatziouras*
> > >> https://egkatzioura.com/ |
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> https://github.com/gkatzioura
> > >>
> > >>
> > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra 
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > We will need to add test suites for tests to be executed in build.
> We
> > >> pass
> > >> > them as TestSuites param. I had added Flink sink and source
> > testsuites.
> > >> >
> > >> > Can you please review and confirm.
> > >> >
> > >> > Regards
> > >> > Saikat
> > >> >
> > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > >> > gkatzio...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi all!
> > >> > >
> > >> > > I was successful building and running my pull requests. It seems
> > that
> > >> the
> > >> > > tests do not run both for Flink and Pub/Sub [1]
> > >> > > If there is something I cannot do to make them discoverable please
> > >> let me
> > >> > > know.
> > >> > >
> > >> > > [1]
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
> > >> > >
> > >> > > *Emmanouil Gkatziouras*
> > >> > > https://egkatzioura.com/ |
> > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > https://github.com/gkatzioura
> > >> > >
> > >> > >
> > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > saikat.mai...@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Emmanouil,
> > >> > > >
> > >> > > > I have added you as contributor in Ignite Extensions project.
> Can
> > >> you
> > >> > > > please check and confirm if you are able to see the project and
> > >> execute
> > >> > > > build on your pull request.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Saikat
> > >> > > >
> > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > >> > > > gkatzio...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi all!
> > >> > > > >
> > >> > > > > If someone could give me read access to the Job of Ignite
> > >> Extensions
> > >> > on
> > >> > > > > Team City will greatly help me (username gkatzioura).
> > >> > > > > I suppose the builds get triggered automatically on a pull
> > >> request.
> > >> > > > >
> > >> > > > > Kind regards,
> > >> > > > > Emmanouil
> > >> > > > >
> > >> > > > > *Emmanouil 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

>From the logs[1] it seems that TC cannot find the project. However I did
include the project on the parent pom (modules sections) [2]
I tried to reproduce locally unfortunately I am not aware of the full
context on TC. It seems as if a different parent pom is being picked up.

Kind regards

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4804893=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
[2]
https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 22:20, Saikat Maitra  wrote:

> Hello,
>
> I have made changes in Ignite Extensions build configurations.
>
> It is now running the tests as expected.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
>
> Regards,
> Saikat
>
> On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra 
> wrote:
>
> > Hi,
> >
> > We do run build with -DskipTests and then configure selective test suites
> > to run.
> >
> > I will check why these test suites are not getting executed.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > gkatzio...@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> I did add a test suite and run the build with the corresponding
> arguments,
> >> however the tests did not run [1].
> >> I checked the Flink's build logs and the same message is displayed there
> >> too `No tests to run` [2].
> >>
> >> [1]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> >> [2]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
> >>
> >> Kind regards
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra 
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > We will need to add test suites for tests to be executed in build. We
> >> pass
> >> > them as TestSuites param. I had added Flink sink and source
> testsuites.
> >> >
> >> > Can you please review and confirm.
> >> >
> >> > Regards
> >> > Saikat
> >> >
> >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> >> > gkatzio...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi all!
> >> > >
> >> > > I was successful building and running my pull requests. It seems
> that
> >> the
> >> > > tests do not run both for Flink and Pub/Sub [1]
> >> > > If there is something I cannot do to make them discoverable please
> >> let me
> >> > > know.
> >> > >
> >> > > [1]
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
> >> > >
> >> > > *Emmanouil Gkatziouras*
> >> > > https://egkatzioura.com/ |
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > https://github.com/gkatzioura
> >> > >
> >> > >
> >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> saikat.mai...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Emmanouil,
> >> > > >
> >> > > > I have added you as contributor in Ignite Extensions project. Can
> >> you
> >> > > > please check and confirm if you are able to see the project and
> >> execute
> >> > > > build on your pull request.
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> >> > > > gkatzio...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi all!
> >> > > > >
> >> > > > > If someone could give me read access to the Job of Ignite
> >> Extensions
> >> > on
> >> > > > > Team City will greatly help me (username gkatzioura).
> >> > > > > I suppose the builds get triggered automatically on a pull
> >> request.
> >> > > > >
> >> > > > > Kind regards,
> >> > > > > Emmanouil
> >> > > > >
> >> > > > > *Emmanouil Gkatziouras*
> >> > > > > https://egkatzioura.com/ |
> >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > https://github.com/gkatzioura
> >> > > > >
> >> > > > >
> >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> >> > > > gkatzio...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Saikat!
> >> > > > > >
> >> > > > > > I just rebased with the flink branch. Unfortunately I do not
> >> have
> >> > > read
> >> > > > > > access to the team city link you provided and thus evaluate
> the
> >> > tests
> >> > > > for
> >> > > > > > my pull request.
> >> > > > > > I guess it has to do with ignite extensions being new.
> >> > > > > > My username is gkatzioura and email the one I am using on this
> >> > email
> >> > > > > >
> >> > > > > > Thank you for your time.
> >> > > > > >
> >> > > > > >
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello,

I have made changes in Ignite Extensions build configurations.

It is now running the tests as expected.

https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead

Regards,
Saikat

On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra 
wrote:

> Hi,
>
> We do run build with -DskipTests and then configure selective test suites
> to run.
>
> I will check why these test suites are not getting executed.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> gkatzio...@gmail.com> wrote:
>
>> Hi all,
>>
>> I did add a test suite and run the build with the corresponding arguments,
>> however the tests did not run [1].
>> I checked the Flink's build logs and the same message is displayed there
>> too `No tests to run` [2].
>>
>> [1]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
>> [2]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
>>
>> Kind regards
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra 
>> wrote:
>>
>> > Hi,
>> >
>> > We will need to add test suites for tests to be executed in build. We
>> pass
>> > them as TestSuites param. I had added Flink sink and source testsuites.
>> >
>> > Can you please review and confirm.
>> >
>> > Regards
>> > Saikat
>> >
>> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
>> > gkatzio...@gmail.com>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I was successful building and running my pull requests. It seems that
>> the
>> > > tests do not run both for Flink and Pub/Sub [1]
>> > > If there is something I cannot do to make them discoverable please
>> let me
>> > > know.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra 
>> > > wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > I have added you as contributor in Ignite Extensions project. Can
>> you
>> > > > please check and confirm if you are able to see the project and
>> execute
>> > > > build on your pull request.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
>> > > > gkatzio...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi all!
>> > > > >
>> > > > > If someone could give me read access to the Job of Ignite
>> Extensions
>> > on
>> > > > > Team City will greatly help me (username gkatzioura).
>> > > > > I suppose the builds get triggered automatically on a pull
>> request.
>> > > > >
>> > > > > Kind regards,
>> > > > > Emmanouil
>> > > > >
>> > > > > *Emmanouil Gkatziouras*
>> > > > > https://egkatzioura.com/ |
>> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > https://github.com/gkatzioura
>> > > > >
>> > > > >
>> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
>> > > > gkatzio...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Saikat!
>> > > > > >
>> > > > > > I just rebased with the flink branch. Unfortunately I do not
>> have
>> > > read
>> > > > > > access to the team city link you provided and thus evaluate the
>> > tests
>> > > > for
>> > > > > > my pull request.
>> > > > > > I guess it has to do with ignite extensions being new.
>> > > > > > My username is gkatzioura and email the one I am using on this
>> > email
>> > > > > >
>> > > > > > Thank you for your time.
>> > > > > >
>> > > > > >
>> > > > > > *Emmanouil Gkatziouras*
>> > > > > > https://egkatzioura.com/ |
>> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > https://github.com/gkatzioura
>> > > > > >
>> > > > > >
>> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
>> > saikat.mai...@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Hi Emmanouil,
>> > > > > >>
>> > > > > >> The latest build on teamcity has passed on Flink pull request.
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
>> > > > > >>
>> > > > > >> you should be able to take the changes I made in my PR and run
>> > build
>> > > > on
>> > > > > >> your pull request.
>> > > > > >>
>> > > > > >> Please let me know if you have any questions.
>> > > > > >>
>> > > > > >> Regards,
>> > > > > >> Saikat
>> > > > > >>
>> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
>> > > > saikat.mai...@gmail.com
>> > > > > >

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi,

We do run build with -DskipTests and then configure selective test suites
to run.

I will check why these test suites are not getting executed.

Regards
Saikat

On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras 
wrote:

> Hi all,
>
> I did add a test suite and run the build with the corresponding arguments,
> however the tests did not run [1].
> I checked the Flink's build logs and the same message is displayed there
> too `No tests to run` [2].
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> [2]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256
>
> Kind regards
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra 
> wrote:
>
> > Hi,
> >
> > We will need to add test suites for tests to be executed in build. We
> pass
> > them as TestSuites param. I had added Flink sink and source testsuites.
> >
> > Can you please review and confirm.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > I was successful building and running my pull requests. It seems that
> the
> > > tests do not run both for Flink and Pub/Sub [1]
> > > If there is something I cannot do to make them discoverable please let
> me
> > > know.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra 
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I have added you as contributor in Ignite Extensions project. Can you
> > > > please check and confirm if you are able to see the project and
> execute
> > > > build on your pull request.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi all!
> > > > >
> > > > > If someone could give me read access to the Job of Ignite
> Extensions
> > on
> > > > > Team City will greatly help me (username gkatzioura).
> > > > > I suppose the builds get triggered automatically on a pull request.
> > > > >
> > > > > Kind regards,
> > > > > Emmanouil
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Saikat!
> > > > > >
> > > > > > I just rebased with the flink branch. Unfortunately I do not have
> > > read
> > > > > > access to the team city link you provided and thus evaluate the
> > tests
> > > > for
> > > > > > my pull request.
> > > > > > I guess it has to do with ignite extensions being new.
> > > > > > My username is gkatzioura and email the one I am using on this
> > email
> > > > > >
> > > > > > Thank you for your time.
> > > > > >
> > > > > >
> > > > > > *Emmanouil Gkatziouras*
> > > > > > https://egkatzioura.com/ |
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > https://github.com/gkatzioura
> > > > > >
> > > > > >
> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Emmanouil,
> > > > > >>
> > > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > > > >>
> > > > > >> you should be able to take the changes I made in my PR and run
> > build
> > > > on
> > > > > >> your pull request.
> > > > > >>
> > > > > >> Please let me know if you have any questions.
> > > > > >>
> > > > > >> Regards,
> > > > > >> Saikat
> > > > > >>
> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > saikat.mai...@gmail.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Emmanouil,
> > > > > >> >
> > > > > >> > I have fixed the Flink module testsuites and have setup new
> > > project
> > > > in
> > > > > >> > teamcity.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> > > > > >> >
> > > > > >> > I am looking into setting up the build.
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Saikat
> > > > > >> >

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all,

I did add a test suite and run the build with the corresponding arguments,
however the tests did not run [1].
I checked the Flink's build logs and the same message is displayed there
too `No tests to run` [2].

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4798203=IgniteExtensions_Build=buildLog_IgniteExtensions=pull%2F2%2Fhead&_focus=248
[2]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4789199=buildLog=tree=debug=all&_focus=1256

Kind regards
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 18:58, Saikat Maitra  wrote:

> Hi,
>
> We will need to add test suites for tests to be executed in build. We pass
> them as TestSuites param. I had added Flink sink and source testsuites.
>
> Can you please review and confirm.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi all!
> >
> > I was successful building and running my pull requests. It seems that the
> > tests do not run both for Flink and Pub/Sub [1]
> > If there is something I cannot do to make them discoverable please let me
> > know.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra 
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I have added you as contributor in Ignite Extensions project. Can you
> > > please check and confirm if you are able to see the project and execute
> > > build on your pull request.
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > If someone could give me read access to the Job of Ignite Extensions
> on
> > > > Team City will greatly help me (username gkatzioura).
> > > > I suppose the builds get triggered automatically on a pull request.
> > > >
> > > > Kind regards,
> > > > Emmanouil
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > I just rebased with the flink branch. Unfortunately I do not have
> > read
> > > > > access to the team city link you provided and thus evaluate the
> tests
> > > for
> > > > > my pull request.
> > > > > I guess it has to do with ignite extensions being new.
> > > > > My username is gkatzioura and email the one I am using on this
> email
> > > > >
> > > > > Thank you for your time.
> > > > >
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Hi Emmanouil,
> > > > >>
> > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > > >>
> > > > >> you should be able to take the changes I made in my PR and run
> build
> > > on
> > > > >> your pull request.
> > > > >>
> > > > >> Please let me know if you have any questions.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > saikat.mai...@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Emmanouil,
> > > > >> >
> > > > >> > I have fixed the Flink module testsuites and have setup new
> > project
> > > in
> > > > >> > teamcity.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> > > > >> >
> > > > >> > I am looking into setting up the build.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Saikat
> > > > >> >
> > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > saikat.mai...@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Hi Emmanouil,
> > > > >> >>
> > > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > > merged
> > > > >> it
> > > > >> >> will provide the Licence, parent POMs etc and you can rebase
> from
> > > > >> master
> > > > >> >> branch and apply the changes on top of it.
> > > > >> >>
> > > > >> >> I am getting some issues with test failures in local with
> > > > >> >> GridTestProperties as the 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi,

We will need to add test suites for tests to be executed in build. We pass
them as TestSuites param. I had added Flink sink and source testsuites.

Can you please review and confirm.

Regards
Saikat

On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras 
wrote:

> Hi all!
>
> I was successful building and running my pull requests. It seems that the
> tests do not run both for Flink and Pub/Sub [1]
> If there is something I cannot do to make them discoverable please let me
> know.
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 04:57, Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > I have added you as contributor in Ignite Extensions project. Can you
> > please check and confirm if you are able to see the project and execute
> > build on your pull request.
> >
> > Regards,
> > Saikat
> >
> > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > If someone could give me read access to the Job of Ignite Extensions on
> > > Team City will greatly help me (username gkatzioura).
> > > I suppose the builds get triggered automatically on a pull request.
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > I just rebased with the flink branch. Unfortunately I do not have
> read
> > > > access to the team city link you provided and thus evaluate the tests
> > for
> > > > my pull request.
> > > > I guess it has to do with ignite extensions being new.
> > > > My username is gkatzioura and email the one I am using on this email
> > > >
> > > > Thank you for your time.
> > > >
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra  >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> The latest build on teamcity has passed on Flink pull request.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > > >>
> > > >> you should be able to take the changes I made in my PR and run build
> > on
> > > >> your pull request.
> > > >>
> > > >> Please let me know if you have any questions.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > I have fixed the Flink module testsuites and have setup new
> project
> > in
> > > >> > teamcity.
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> > > >> >
> > > >> > I am looking into setting up the build.
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > saikat.mai...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Emmanouil,
> > > >> >>
> > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > merged
> > > >> it
> > > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > > >> master
> > > >> >> branch and apply the changes on top of it.
> > > >> >>
> > > >> >> I am getting some issues with test failures in local with
> > > >> >> GridTestProperties as the test.properties is not present in this
> > new
> > > >> >> project but should be available from dependencies.
> > > >> >>
> > > >> >> Once I address this issue, I will go ahead and merge the changes
> > and
> > > >> then
> > > >> >> we can take it from there.
> > > >> >>
> > > >> >> Regards,
> > > >> >> Saikat
> > > >> >>
> > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > >> >> gkatzio...@gmail.com> wrote:
> > > >> >>
> > > >> >>> Hi all,
> > > >> >>>
> > > >> >>> I made my first pull request [1]. Since this project is brand
> new
> > > (no
> > > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > > >> branch on
> > > >> >>> Flink.
> > > >> >>> I suppose the Flink branch will be merged first. If not please
> > give
> > > me
> > > >> >>> guidelines on how I should proceed next.
> > > >> >>>
> > > >> >>> Kind regards
> > > >> >>> Emmanouil
> > > >> >>>
> > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > >> >>>
> > > >> >>> *Emmanouil Gkatziouras*
> > > >> >>> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all!

I was successful building and running my pull requests. It seems that the
tests do not run both for Flink and Pub/Sub [1]
If there is something I cannot do to make them discoverable please let me
know.

[1]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build=4796722=buildLog=tree=debug=all&_focus=283

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 04:57, Saikat Maitra  wrote:

> Hi Emmanouil,
>
> I have added you as contributor in Ignite Extensions project. Can you
> please check and confirm if you are able to see the project and execute
> build on your pull request.
>
> Regards,
> Saikat
>
> On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi all!
> >
> > If someone could give me read access to the Job of Ignite Extensions on
> > Team City will greatly help me (username gkatzioura).
> > I suppose the builds get triggered automatically on a pull request.
> >
> > Kind regards,
> > Emmanouil
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > I just rebased with the flink branch. Unfortunately I do not have read
> > > access to the team city link you provided and thus evaluate the tests
> for
> > > my pull request.
> > > I guess it has to do with ignite extensions being new.
> > > My username is gkatzioura and email the one I am using on this email
> > >
> > > Thank you for your time.
> > >
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra 
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> The latest build on teamcity has passed on Flink pull request.
> > >>
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> > >>
> > >> you should be able to take the changes I made in my PR and run build
> on
> > >> your pull request.
> > >>
> > >> Please let me know if you have any questions.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > I have fixed the Flink module testsuites and have setup new project
> in
> > >> > teamcity.
> > >> >
> > >> >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> > >> >
> > >> > I am looking into setting up the build.
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > saikat.mai...@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> Hi Emmanouil,
> > >> >>
> > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > merged
> > >> it
> > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > >> master
> > >> >> branch and apply the changes on top of it.
> > >> >>
> > >> >> I am getting some issues with test failures in local with
> > >> >> GridTestProperties as the test.properties is not present in this
> new
> > >> >> project but should be available from dependencies.
> > >> >>
> > >> >> Once I address this issue, I will go ahead and merge the changes
> and
> > >> then
> > >> >> we can take it from there.
> > >> >>
> > >> >> Regards,
> > >> >> Saikat
> > >> >>
> > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > >> >> gkatzio...@gmail.com> wrote:
> > >> >>
> > >> >>> Hi all,
> > >> >>>
> > >> >>> I made my first pull request [1]. Since this project is brand new
> > (no
> > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > >> branch on
> > >> >>> Flink.
> > >> >>> I suppose the Flink branch will be merged first. If not please
> give
> > me
> > >> >>> guidelines on how I should proceed next.
> > >> >>>
> > >> >>> Kind regards
> > >> >>> Emmanouil
> > >> >>>
> > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > >> >>>
> > >> >>> *Emmanouil Gkatziouras*
> > >> >>> https://egkatzioura.com/ |
> > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> https://github.com/gkatzioura
> > >> >>>
> > >> >>>
> > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda 
> > wrote:
> > >> >>>
> > >> >>> > Awesome, ping us whenever you're ready!
> > >> >>> >
> > >> >>> > -
> > >> >>> > Denis
> > >> >>> >
> > >> >>> >
> > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > >> >>> > gkatzio...@gmail.com>
> > >> >>> > wrote:
> > >> >>> >
> > >> >>> > > Hi all!
> > >> >>> > >
> > >> >>> > > I am sorry for being late on that. I was trying to refactor

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

I have added you as contributor in Ignite Extensions project. Can you
please check and confirm if you are able to see the project and execute
build on your pull request.

Regards,
Saikat

On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras 
wrote:

> Hi all!
>
> If someone could give me read access to the Job of Ignite Extensions on
> Team City will greatly help me (username gkatzioura).
> I suppose the builds get triggered automatically on a pull request.
>
> Kind regards,
> Emmanouil
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras 
> wrote:
>
> > Hi Saikat!
> >
> > I just rebased with the flink branch. Unfortunately I do not have read
> > access to the team city link you provided and thus evaluate the tests for
> > my pull request.
> > I guess it has to do with ignite extensions being new.
> > My username is gkatzioura and email the one I am using on this email
> >
> > Thank you for your time.
> >
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra 
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> The latest build on teamcity has passed on Flink pull request.
> >>
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
> >>
> >> you should be able to take the changes I made in my PR and run build on
> >> your pull request.
> >>
> >> Please let me know if you have any questions.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra  >
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > I have fixed the Flink module testsuites and have setup new project in
> >> > teamcity.
> >> >
> >> >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> >> >
> >> > I am looking into setting up the build.
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> saikat.mai...@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Emmanouil,
> >> >>
> >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> merged
> >> it
> >> >> will provide the Licence, parent POMs etc and you can rebase from
> >> master
> >> >> branch and apply the changes on top of it.
> >> >>
> >> >> I am getting some issues with test failures in local with
> >> >> GridTestProperties as the test.properties is not present in this new
> >> >> project but should be available from dependencies.
> >> >>
> >> >> Once I address this issue, I will go ahead and merge the changes and
> >> then
> >> >> we can take it from there.
> >> >>
> >> >> Regards,
> >> >> Saikat
> >> >>
> >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> >> gkatzio...@gmail.com> wrote:
> >> >>
> >> >>> Hi all,
> >> >>>
> >> >>> I made my first pull request [1]. Since this project is brand new
> (no
> >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> >> branch on
> >> >>> Flink.
> >> >>> I suppose the Flink branch will be merged first. If not please give
> me
> >> >>> guidelines on how I should proceed next.
> >> >>>
> >> >>> Kind regards
> >> >>> Emmanouil
> >> >>>
> >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >> >>>
> >> >>> *Emmanouil Gkatziouras*
> >> >>> https://egkatzioura.com/ |
> >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> https://github.com/gkatzioura
> >> >>>
> >> >>>
> >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda 
> wrote:
> >> >>>
> >> >>> > Awesome, ping us whenever you're ready!
> >> >>> >
> >> >>> > -
> >> >>> > Denis
> >> >>> >
> >> >>> >
> >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >> >>> > gkatzio...@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> > > Hi all!
> >> >>> > >
> >> >>> > > I am sorry for being late on that. I was trying to refactor the
> >> test
> >> >>> in
> >> >>> > > order not to be in need of any external tools or spinning up a
> >> >>> server.
> >> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >> >>> request
> >> >>> > is
> >> >>> > > a matter of time!
> >> >>> > >
> >> >>> > > Kind regards
> >> >>> > >
> >> >>> > > *Emmanouil Gkatziouras*
> >> >>> > > https://egkatzioura.com/ |
> >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > https://github.com/gkatzioura
> >> >>> > >
> >> >>> > >
> >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda 
> >> wrote:
> >> >>> > >
> >> >>> > > > Hi Emmanouil,
> >> >>> > > >
> >> >>> > > > Do you have any questions or need any support from the
> >> community?
> >> >>> > > >
> >> >>> > > > -
> >> >>> > > > Denis
> >> >>> > > >
> >> >>> > > >
> >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >> >>> saikat.mai...@gmail.com
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all!

If someone could give me read access to the Job of Ignite Extensions on
Team City will greatly help me (username gkatzioura).
I suppose the builds get triggered automatically on a pull request.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras 
wrote:

> Hi Saikat!
>
> I just rebased with the flink branch. Unfortunately I do not have read
> access to the team city link you provided and thus evaluate the tests for
> my pull request.
> I guess it has to do with ignite extensions being new.
> My username is gkatzioura and email the one I am using on this email
>
> Thank you for your time.
>
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 18:03, Saikat Maitra 
> wrote:
>
>> Hi Emmanouil,
>>
>> The latest build on teamcity has passed on Flink pull request.
>>
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
>>
>> you should be able to take the changes I made in my PR and run build on
>> your pull request.
>>
>> Please let me know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra 
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > I have fixed the Flink module testsuites and have setup new project in
>> > teamcity.
>> >
>> >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
>> >
>> > I am looking into setting up the build.
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra 
>> > wrote:
>> >
>> >> Hi Emmanouil,
>> >>
>> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
>> it
>> >> will provide the Licence, parent POMs etc and you can rebase from
>> master
>> >> branch and apply the changes on top of it.
>> >>
>> >> I am getting some issues with test failures in local with
>> >> GridTestProperties as the test.properties is not present in this new
>> >> project but should be available from dependencies.
>> >>
>> >> Once I address this issue, I will go ahead and merge the changes and
>> then
>> >> we can take it from there.
>> >>
>> >> Regards,
>> >> Saikat
>> >>
>> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> >> gkatzio...@gmail.com> wrote:
>> >>
>> >>> Hi all,
>> >>>
>> >>> I made my first pull request [1]. Since this project is brand new (no
>> >>> parent poms, licensing), I reckoned it was best to use Saikat's
>> branch on
>> >>> Flink.
>> >>> I suppose the Flink branch will be merged first. If not please give me
>> >>> guidelines on how I should proceed next.
>> >>>
>> >>> Kind regards
>> >>> Emmanouil
>> >>>
>> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> >>>
>> >>> *Emmanouil Gkatziouras*
>> >>> https://egkatzioura.com/ |
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> https://github.com/gkatzioura
>> >>>
>> >>>
>> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:
>> >>>
>> >>> > Awesome, ping us whenever you're ready!
>> >>> >
>> >>> > -
>> >>> > Denis
>> >>> >
>> >>> >
>> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> >>> > gkatzio...@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> > > Hi all!
>> >>> > >
>> >>> > > I am sorry for being late on that. I was trying to refactor the
>> test
>> >>> in
>> >>> > > order not to be in need of any external tools or spinning up a
>> >>> server.
>> >>> > > I did forked the new repo and indeed my changes there, so a pull
>> >>> request
>> >>> > is
>> >>> > > a matter of time!
>> >>> > >
>> >>> > > Kind regards
>> >>> > >
>> >>> > > *Emmanouil Gkatziouras*
>> >>> > > https://egkatzioura.com/ |
>> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > https://github.com/gkatzioura
>> >>> > >
>> >>> > >
>> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda 
>> wrote:
>> >>> > >
>> >>> > > > Hi Emmanouil,
>> >>> > > >
>> >>> > > > Do you have any questions or need any support from the
>> community?
>> >>> > > >
>> >>> > > > -
>> >>> > > > Denis
>> >>> > > >
>> >>> > > >
>> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> >>> saikat.mai...@gmail.com
>> >>> > >
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > >> Hi Emmanouil,
>> >>> > > >>
>> >>> > > >> Can you please take a looks at dev utils, if this is something
>> >>> you are
>> >>> > > >> looking for
>> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
>> ?
>> >>> > > >>
>> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
>> it as
>> >>> > > >> dependency, that would be great.
>> >>> > > >>
>> >>> > > >> Regards,
>> >>> > > >> Saikat
>> >>> > > >>
>> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> >>> > saikat.mai...@gmail.com>
>> >>> > > >> wrote:
>> >>> > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi Saikat!

I just rebased with the flink branch. Unfortunately I do not have read
access to the team city link you provided and thus evaluate the tests for
my pull request.
I guess it has to do with ignite extensions being new.
My username is gkatzioura and email the one I am using on this email

Thank you for your time.


*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 18:03, Saikat Maitra  wrote:

> Hi Emmanouil,
>
> The latest build on teamcity has passed on Flink pull request.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
>
> you should be able to take the changes I made in my PR and run build on
> your pull request.
>
> Please let me know if you have any questions.
>
> Regards,
> Saikat
>
> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > I have fixed the Flink module testsuites and have setup new project in
> > teamcity.
> >
> >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
> >
> > I am looking into setting up the build.
> >
> > Regards,
> > Saikat
> >
> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra 
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
> it
> >> will provide the Licence, parent POMs etc and you can rebase from master
> >> branch and apply the changes on top of it.
> >>
> >> I am getting some issues with test failures in local with
> >> GridTestProperties as the test.properties is not present in this new
> >> project but should be available from dependencies.
> >>
> >> Once I address this issue, I will go ahead and merge the changes and
> then
> >> we can take it from there.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> gkatzio...@gmail.com> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I made my first pull request [1]. Since this project is brand new (no
> >>> parent poms, licensing), I reckoned it was best to use Saikat's branch
> on
> >>> Flink.
> >>> I suppose the Flink branch will be merged first. If not please give me
> >>> guidelines on how I should proceed next.
> >>>
> >>> Kind regards
> >>> Emmanouil
> >>>
> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >>>
> >>> *Emmanouil Gkatziouras*
> >>> https://egkatzioura.com/ |
> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> https://github.com/gkatzioura
> >>>
> >>>
> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:
> >>>
> >>> > Awesome, ping us whenever you're ready!
> >>> >
> >>> > -
> >>> > Denis
> >>> >
> >>> >
> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >>> > gkatzio...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Hi all!
> >>> > >
> >>> > > I am sorry for being late on that. I was trying to refactor the
> test
> >>> in
> >>> > > order not to be in need of any external tools or spinning up a
> >>> server.
> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >>> request
> >>> > is
> >>> > > a matter of time!
> >>> > >
> >>> > > Kind regards
> >>> > >
> >>> > > *Emmanouil Gkatziouras*
> >>> > > https://egkatzioura.com/ |
> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > https://github.com/gkatzioura
> >>> > >
> >>> > >
> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda 
> wrote:
> >>> > >
> >>> > > > Hi Emmanouil,
> >>> > > >
> >>> > > > Do you have any questions or need any support from the community?
> >>> > > >
> >>> > > > -
> >>> > > > Denis
> >>> > > >
> >>> > > >
> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >>> saikat.mai...@gmail.com
> >>> > >
> >>> > > > wrote:
> >>> > > >
> >>> > > >> Hi Emmanouil,
> >>> > > >>
> >>> > > >> Can you please take a looks at dev utils, if this is something
> >>> you are
> >>> > > >> looking for
> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>> > > >>
> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use it
> as
> >>> > > >> dependency, that would be great.
> >>> > > >>
> >>> > > >> Regards,
> >>> > > >> Saikat
> >>> > > >>
> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >>> > saikat.mai...@gmail.com>
> >>> > > >> wrote:
> >>> > > >>
> >>> > > >> > Hi Emmanouil,
> >>> > > >> >
> >>> > > >> > The master branch has been initialized. I have raised a PR to
> >>> > migrate
> >>> > > >> > Flink module to ignite-extensions.
> >>> > > >> >
> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >>> > > >> >
> >>> > > >> > Regards,
> >>> > > >> > Saikat
> >>> > > >> >
> >>> > > >> >
> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >>> > > >> > gkatzio...@gmail.com> wrote:
> >>> > > >> >
> >>> > > >> >> Hi all!
> >>> > > >> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

The latest build on teamcity has passed on Flink pull request.

https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead

you should be able to take the changes I made in my PR and run build on
your pull request.

Please let me know if you have any questions.

Regards,
Saikat

On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra 
wrote:

> Hi Emmanouil,
>
> I have fixed the Flink module testsuites and have setup new project in
> teamcity.
>
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
>
> I am looking into setting up the build.
>
> Regards,
> Saikat
>
> On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra 
> wrote:
>
>> Hi Emmanouil,
>>
>> Thank you for your email. Yes, the plan is once the Flink PR is merged it
>> will provide the Licence, parent POMs etc and you can rebase from master
>> branch and apply the changes on top of it.
>>
>> I am getting some issues with test failures in local with
>> GridTestProperties as the test.properties is not present in this new
>> project but should be available from dependencies.
>>
>> Once I address this issue, I will go ahead and merge the changes and then
>> we can take it from there.
>>
>> Regards,
>> Saikat
>>
>> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> gkatzio...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I made my first pull request [1]. Since this project is brand new (no
>>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>>> Flink.
>>> I suppose the Flink branch will be merged first. If not please give me
>>> guidelines on how I should proceed next.
>>>
>>> Kind regards
>>> Emmanouil
>>>
>>> [1] https://github.com/apache/ignite-extensions/pull/2
>>>
>>> *Emmanouil Gkatziouras*
>>> https://egkatzioura.com/ |
>>> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> https://github.com/gkatzioura
>>>
>>>
>>> On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:
>>>
>>> > Awesome, ping us whenever you're ready!
>>> >
>>> > -
>>> > Denis
>>> >
>>> >
>>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>>> > gkatzio...@gmail.com>
>>> > wrote:
>>> >
>>> > > Hi all!
>>> > >
>>> > > I am sorry for being late on that. I was trying to refactor the test
>>> in
>>> > > order not to be in need of any external tools or spinning up a
>>> server.
>>> > > I did forked the new repo and indeed my changes there, so a pull
>>> request
>>> > is
>>> > > a matter of time!
>>> > >
>>> > > Kind regards
>>> > >
>>> > > *Emmanouil Gkatziouras*
>>> > > https://egkatzioura.com/ |
>>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > https://github.com/gkatzioura
>>> > >
>>> > >
>>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:
>>> > >
>>> > > > Hi Emmanouil,
>>> > > >
>>> > > > Do you have any questions or need any support from the community?
>>> > > >
>>> > > > -
>>> > > > Denis
>>> > > >
>>> > > >
>>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>>> saikat.mai...@gmail.com
>>> > >
>>> > > > wrote:
>>> > > >
>>> > > >> Hi Emmanouil,
>>> > > >>
>>> > > >> Can you please take a looks at dev utils, if this is something
>>> you are
>>> > > >> looking for
>>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>> > > >>
>>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>>> > > >> dependency, that would be great.
>>> > > >>
>>> > > >> Regards,
>>> > > >> Saikat
>>> > > >>
>>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>>> > saikat.mai...@gmail.com>
>>> > > >> wrote:
>>> > > >>
>>> > > >> > Hi Emmanouil,
>>> > > >> >
>>> > > >> > The master branch has been initialized. I have raised a PR to
>>> > migrate
>>> > > >> > Flink module to ignite-extensions.
>>> > > >> >
>>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>>> > > >> >
>>> > > >> > Regards,
>>> > > >> > Saikat
>>> > > >> >
>>> > > >> >
>>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>>> > > >> > gkatzio...@gmail.com> wrote:
>>> > > >> >
>>> > > >> >> Hi all!
>>> > > >> >>
>>> > > >> >> This is an update. The integration is tested with mocks based
>>> on
>>> > the
>>> > > >> >> original Ignite repo [1].
>>> > > >> >> I am currently waiting for the first commits on the new
>>> integration
>>> > > >> >> project
>>> > > >> >> to take place in order to create my pull request accordingly
>>> to the
>>> > > >> >> initial
>>> > > >> >> commits.
>>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>>> > > server
>>> > > >> >> implementation [2]. Unfortunately they don't have this util
>>> > uploaded
>>> > > >> in a
>>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>>> > > License
>>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>>> > > >> test/build?
>>> > > >> >> I have also created a ticket [3]
>>> > > >> >>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

I have fixed the Flink module testsuites and have setup new project in
teamcity.

https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview

I am looking into setting up the build.

Regards,
Saikat

On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra 
wrote:

> Hi Emmanouil,
>
> Thank you for your email. Yes, the plan is once the Flink PR is merged it
> will provide the Licence, parent POMs etc and you can rebase from master
> branch and apply the changes on top of it.
>
> I am getting some issues with test failures in local with
> GridTestProperties as the test.properties is not present in this new
> project but should be available from dependencies.
>
> Once I address this issue, I will go ahead and merge the changes and then
> we can take it from there.
>
> Regards,
> Saikat
>
> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com> wrote:
>
>> Hi all,
>>
>> I made my first pull request [1]. Since this project is brand new (no
>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>> Flink.
>> I suppose the Flink branch will be merged first. If not please give me
>> guidelines on how I should proceed next.
>>
>> Kind regards
>> Emmanouil
>>
>> [1] https://github.com/apache/ignite-extensions/pull/2
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:
>>
>> > Awesome, ping us whenever you're ready!
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> > gkatzio...@gmail.com>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I am sorry for being late on that. I was trying to refactor the test
>> in
>> > > order not to be in need of any external tools or spinning up a server.
>> > > I did forked the new repo and indeed my changes there, so a pull
>> request
>> > is
>> > > a matter of time!
>> > >
>> > > Kind regards
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > Do you have any questions or need any support from the community?
>> > > >
>> > > > -
>> > > > Denis
>> > > >
>> > > >
>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> saikat.mai...@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > >> Hi Emmanouil,
>> > > >>
>> > > >> Can you please take a looks at dev utils, if this is something you
>> are
>> > > >> looking for
>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>> > > >>
>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>> > > >> dependency, that would be great.
>> > > >>
>> > > >> Regards,
>> > > >> Saikat
>> > > >>
>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> > saikat.mai...@gmail.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi Emmanouil,
>> > > >> >
>> > > >> > The master branch has been initialized. I have raised a PR to
>> > migrate
>> > > >> > Flink module to ignite-extensions.
>> > > >> >
>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> > > >> >
>> > > >> > Regards,
>> > > >> > Saikat
>> > > >> >
>> > > >> >
>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > > >> > gkatzio...@gmail.com> wrote:
>> > > >> >
>> > > >> >> Hi all!
>> > > >> >>
>> > > >> >> This is an update. The integration is tested with mocks based on
>> > the
>> > > >> >> original Ignite repo [1].
>> > > >> >> I am currently waiting for the first commits on the new
>> integration
>> > > >> >> project
>> > > >> >> to take place in order to create my pull request accordingly to
>> the
>> > > >> >> initial
>> > > >> >> commits.
>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>> > > server
>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> > uploaded
>> > > >> in a
>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>> > > License
>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> > > >> test/build?
>> > > >> >> I have also created a ticket [3]
>> > > >> >>
>> > > >> >> Kind regards
>> > > >> >>
>> > > >> >> [1]
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > >> >> [2]
>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> > > >> >>
>> > > >> >> *Emmanouil Gkatziouras*
>> > > >> >> https://egkatzioura.com/ |
>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> https://github.com/gkatzioura
>> > > >> >>
>> > > >> >>
>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> > saikat.mai...@gmail.com
>> > > >
>> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

Thank you for your email. Yes, the plan is once the Flink PR is merged it
will provide the Licence, parent POMs etc and you can rebase from master
branch and apply the changes on top of it.

I am getting some issues with test failures in local with
GridTestProperties as the test.properties is not present in this new
project but should be available from dependencies.

Once I address this issue, I will go ahead and merge the changes and then
we can take it from there.

Regards,
Saikat

On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras 
wrote:

> Hi all,
>
> I made my first pull request [1]. Since this project is brand new (no
> parent poms, licensing), I reckoned it was best to use Saikat's branch on
> Flink.
> I suppose the Flink branch will be merged first. If not please give me
> guidelines on how I should proceed next.
>
> Kind regards
> Emmanouil
>
> [1] https://github.com/apache/ignite-extensions/pull/2
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:
>
> > Awesome, ping us whenever you're ready!
> >
> > -
> > Denis
> >
> >
> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > I am sorry for being late on that. I was trying to refactor the test in
> > > order not to be in need of any external tools or spinning up a server.
> > > I did forked the new repo and indeed my changes there, so a pull
> request
> > is
> > > a matter of time!
> > >
> > > Kind regards
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > Do you have any questions or need any support from the community?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> Can you please take a looks at dev utils, if this is something you
> are
> > > >> looking for
> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > > >>
> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > > >> dependency, that would be great.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > saikat.mai...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > The master branch has been initialized. I have raised a PR to
> > migrate
> > > >> > Flink module to ignite-extensions.
> > > >> >
> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> >
> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > > >> > gkatzio...@gmail.com> wrote:
> > > >> >
> > > >> >> Hi all!
> > > >> >>
> > > >> >> This is an update. The integration is tested with mocks based on
> > the
> > > >> >> original Ignite repo [1].
> > > >> >> I am currently waiting for the first commits on the new
> integration
> > > >> >> project
> > > >> >> to take place in order to create my pull request accordingly to
> the
> > > >> >> initial
> > > >> >> commits.
> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > > server
> > > >> >> implementation [2]. Unfortunately they don't have this util
> > uploaded
> > > >> in a
> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > > License
> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > > >> test/build?
> > > >> >> I have also created a ticket [3]
> > > >> >>
> > > >> >> Kind regards
> > > >> >>
> > > >> >> [1]
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> >>
> > > >> >> *Emmanouil Gkatziouras*
> > > >> >> https://egkatzioura.com/ |
> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> https://github.com/gkatzioura
> > > >> >>
> > > >> >>
> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >>
> > > >> >> > Hello Ilya,
> > > >> >> >
> > > >> >> > Thank you for your email. Yes, before we remove any integration
> > we
> > > >> will
> > > >> >> > move them to another repository and release.
> > > >> >> >
> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > implementation
> > > >> and
> > > >> >> > continue to move other integration as mentioned in the
> following
> > > doc.
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all,

I made my first pull request [1]. Since this project is brand new (no
parent poms, licensing), I reckoned it was best to use Saikat's branch on
Flink.
I suppose the Flink branch will be merged first. If not please give me
guidelines on how I should proceed next.

Kind regards
Emmanouil

[1] https://github.com/apache/ignite-extensions/pull/2

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:55, Denis Magda  wrote:

> Awesome, ping us whenever you're ready!
>
> -
> Denis
>
>
> On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi all!
> >
> > I am sorry for being late on that. I was trying to refactor the test in
> > order not to be in need of any external tools or spinning up a server.
> > I did forked the new repo and indeed my changes there, so a pull request
> is
> > a matter of time!
> >
> > Kind regards
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:
> >
> > > Hi Emmanouil,
> > >
> > > Do you have any questions or need any support from the community?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra  >
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> Can you please take a looks at dev utils, if this is something you are
> > >> looking for
> > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > >>
> > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > >> dependency, that would be great.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> saikat.mai...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > The master branch has been initialized. I have raised a PR to
> migrate
> > >> > Flink module to ignite-extensions.
> > >> >
> > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> >
> > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > >> > gkatzio...@gmail.com> wrote:
> > >> >
> > >> >> Hi all!
> > >> >>
> > >> >> This is an update. The integration is tested with mocks based on
> the
> > >> >> original Ignite repo [1].
> > >> >> I am currently waiting for the first commits on the new integration
> > >> >> project
> > >> >> to take place in order to create my pull request accordingly to the
> > >> >> initial
> > >> >> commits.
> > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > server
> > >> >> implementation [2]. Unfortunately they don't have this util
> uploaded
> > >> in a
> > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > License
> > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > >> test/build?
> > >> >> I have also created a ticket [3]
> > >> >>
> > >> >> Kind regards
> > >> >>
> > >> >> [1]
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > >> >>
> > >> >> *Emmanouil Gkatziouras*
> > >> >> https://egkatzioura.com/ |
> > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> https://github.com/gkatzioura
> > >> >>
> > >> >>
> > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > >> >> wrote:
> > >> >>
> > >> >> > Hello Ilya,
> > >> >> >
> > >> >> > Thank you for your email. Yes, before we remove any integration
> we
> > >> will
> > >> >> > move them to another repository and release.
> > >> >> >
> > >> >> > We are thinking we will start with Pub/Sub Streamer
> implementation
> > >> and
> > >> >> > continue to move other integration as mentioned in the following
> > doc.
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> >
> > >> >> > Regards,
> > >> >> > Saikat
> > >> >> >
> > >> >> >
> > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > >> >> > ilya.kasnach...@gmail.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Hello!
> > >> >> > >
> > >> >> > > My take on this: we want to offload some of integrations, but
> > >> please
> > >> >> > commit
> > >> >> > > and release them to another project first, and only then we
> will
> > >> >> decide
> > >> >> > to
> > >> >> > > remove them from our repository.
> > >> >> > >
> > >> >> > > Please also make sure they are available with same artifact
> name
> > >> and
> > >> >> API.
> > >> >> > > If this has to be changed, let's wait for AI 3.0
> > >> >> > >
> > >> >> > > Regards,
> > >> >> > > --
> > >> >> > > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Awesome, ping us whenever you're ready!

-
Denis


On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras 
wrote:

> Hi all!
>
> I am sorry for being late on that. I was trying to refactor the test in
> order not to be in need of any external tools or spinning up a server.
> I did forked the new repo and indeed my changes there, so a pull request is
> a matter of time!
>
> Kind regards
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:
>
> > Hi Emmanouil,
> >
> > Do you have any questions or need any support from the community?
> >
> > -
> > Denis
> >
> >
> > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra 
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Can you please take a looks at dev utils, if this is something you are
> >> looking for
> >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>
> >> IMO, if you can release Pub/Sub server in maven and then use it as
> >> dependency, that would be great.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra 
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > The master branch has been initialized. I have raised a PR to migrate
> >> > Flink module to ignite-extensions.
> >> >
> >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >> > gkatzio...@gmail.com> wrote:
> >> >
> >> >> Hi all!
> >> >>
> >> >> This is an update. The integration is tested with mocks based on the
> >> >> original Ignite repo [1].
> >> >> I am currently waiting for the first commits on the new integration
> >> >> project
> >> >> to take place in order to create my pull request accordingly to the
> >> >> initial
> >> >> commits.
> >> >> In order to simulate the Pub/Sub server in a test I used a test
> server
> >> >> implementation [2]. Unfortunately they don't have this util uploaded
> >> in a
> >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> License
> >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> >> test/build?
> >> >> I have also created a ticket [3]
> >> >>
> >> >> Kind regards
> >> >>
> >> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >> >>
> >> >> *Emmanouil Gkatziouras*
> >> >> https://egkatzioura.com/ |
> >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> https://github.com/gkatzioura
> >> >>
> >> >>
> >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra  >
> >> >> wrote:
> >> >>
> >> >> > Hello Ilya,
> >> >> >
> >> >> > Thank you for your email. Yes, before we remove any integration we
> >> will
> >> >> > move them to another repository and release.
> >> >> >
> >> >> > We are thinking we will start with Pub/Sub Streamer implementation
> >> and
> >> >> > continue to move other integration as mentioned in the following
> doc.
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> >
> >> >> > Regards,
> >> >> > Saikat
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> >> > ilya.kasnach...@gmail.com>
> >> >> > wrote:
> >> >> >
> >> >> > > Hello!
> >> >> > >
> >> >> > > My take on this: we want to offload some of integrations, but
> >> please
> >> >> > commit
> >> >> > > and release them to another project first, and only then we will
> >> >> decide
> >> >> > to
> >> >> > > remove them from our repository.
> >> >> > >
> >> >> > > Please also make sure they are available with same artifact name
> >> and
> >> >> API.
> >> >> > > If this has to be changed, let's wait for AI 3.0
> >> >> > >
> >> >> > > Regards,
> >> >> > > --
> >> >> > > Ilya Kasnacheev
> >> >> > >
> >> >> > >
> >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >> saikat.mai...@gmail.com>:
> >> >> > >
> >> >> > > > Hello Emmanouil,
> >> >> > > >
> >> >> > > > As discussed earlier, I discussed with Apache Bahir community
> and
> >> >> they
> >> >> > > are
> >> >> > > > interested to have Apache Ignite extensions as part of Apache
> >> Bahir
> >> >> > > > project.
> >> >> > > >
> >> >> > > > Can you please share Pub/Sub streamer implementation details
> with
> >> >> > Apache
> >> >> > > > Bahir community and request for feedback.
> >> >> > > >
> >> >> > > > https://bahir.apache.org/
> >> >> > > >
> >> >> > > > I have also requested for contributor access in Jira for Apache
> >> >> Bahir
> >> >> > > > project so that I can create issues and assign to myself.
> >> >> > > >
> >> >> > > > I can help with code reviews as well.
> >> >> > > >
> >> >> > > > Here is the email thread for 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all!

I am sorry for being late on that. I was trying to refactor the test in
order not to be in need of any external tools or spinning up a server.
I did forked the new repo and indeed my changes there, so a pull request is
a matter of time!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:45, Denis Magda  wrote:

> Hi Emmanouil,
>
> Do you have any questions or need any support from the community?
>
> -
> Denis
>
>
> On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra 
> wrote:
>
>> Hi Emmanouil,
>>
>> Can you please take a looks at dev utils, if this is something you are
>> looking for
>> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>
>> IMO, if you can release Pub/Sub server in maven and then use it as
>> dependency, that would be great.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra 
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > The master branch has been initialized. I have raised a PR to migrate
>> > Flink module to ignite-extensions.
>> >
>> > PR https://github.com/apache/ignite-extensions/pull/1
>> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > gkatzio...@gmail.com> wrote:
>> >
>> >> Hi all!
>> >>
>> >> This is an update. The integration is tested with mocks based on the
>> >> original Ignite repo [1].
>> >> I am currently waiting for the first commits on the new integration
>> >> project
>> >> to take place in order to create my pull request accordingly to the
>> >> initial
>> >> commits.
>> >> In order to simulate the Pub/Sub server in a test I used a test server
>> >> implementation [2]. Unfortunately they don't have this util uploaded
>> in a
>> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> test/build?
>> >> I have also created a ticket [3]
>> >>
>> >> Kind regards
>> >>
>> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>
>> >> *Emmanouil Gkatziouras*
>> >> https://egkatzioura.com/ |
>> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> https://github.com/gkatzioura
>> >>
>> >>
>> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra 
>> >> wrote:
>> >>
>> >> > Hello Ilya,
>> >> >
>> >> > Thank you for your email. Yes, before we remove any integration we
>> will
>> >> > move them to another repository and release.
>> >> >
>> >> > We are thinking we will start with Pub/Sub Streamer implementation
>> and
>> >> > continue to move other integration as mentioned in the following doc.
>> >> >
>> >> >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> >
>> >> > Regards,
>> >> > Saikat
>> >> >
>> >> >
>> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >> > ilya.kasnach...@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Hello!
>> >> > >
>> >> > > My take on this: we want to offload some of integrations, but
>> please
>> >> > commit
>> >> > > and release them to another project first, and only then we will
>> >> decide
>> >> > to
>> >> > > remove them from our repository.
>> >> > >
>> >> > > Please also make sure they are available with same artifact name
>> and
>> >> API.
>> >> > > If this has to be changed, let's wait for AI 3.0
>> >> > >
>> >> > > Regards,
>> >> > > --
>> >> > > Ilya Kasnacheev
>> >> > >
>> >> > >
>> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> saikat.mai...@gmail.com>:
>> >> > >
>> >> > > > Hello Emmanouil,
>> >> > > >
>> >> > > > As discussed earlier, I discussed with Apache Bahir community and
>> >> they
>> >> > > are
>> >> > > > interested to have Apache Ignite extensions as part of Apache
>> Bahir
>> >> > > > project.
>> >> > > >
>> >> > > > Can you please share Pub/Sub streamer implementation details with
>> >> > Apache
>> >> > > > Bahir community and request for feedback.
>> >> > > >
>> >> > > > https://bahir.apache.org/
>> >> > > >
>> >> > > > I have also requested for contributor access in Jira for Apache
>> >> Bahir
>> >> > > > project so that I can create issues and assign to myself.
>> >> > > >
>> >> > > > I can help with code reviews as well.
>> >> > > >
>> >> > > > Here is the email thread for reference
>> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >> > > >
>> >> > > > Regards,
>> >> > > > Saikat
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >> saikat.mai...@gmail.com
>> >> > >
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Hello,
>> >> > > > >
>> >> > > > > As discussed I have created 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Hi Emmanouil,

Do you have any questions or need any support from the community?

-
Denis


On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra 
wrote:

> Hi Emmanouil,
>
> Can you please take a looks at dev utils, if this is something you are
> looking for https://github.com/apache/ignite/tree/master/modules/dev-utils
> ?
>
> IMO, if you can release Pub/Sub server in maven and then use it as
> dependency, that would be great.
>
> Regards,
> Saikat
>
> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > The master branch has been initialized. I have raised a PR to migrate
> > Flink module to ignite-extensions.
> >
> > PR https://github.com/apache/ignite-extensions/pull/1
> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >
> > Regards,
> > Saikat
> >
> >
> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com> wrote:
> >
> >> Hi all!
> >>
> >> This is an update. The integration is tested with mocks based on the
> >> original Ignite repo [1].
> >> I am currently waiting for the first commits on the new integration
> >> project
> >> to take place in order to create my pull request accordingly to the
> >> initial
> >> commits.
> >> In order to simulate the Pub/Sub server in a test I used a test server
> >> implementation [2]. Unfortunately they don't have this util uploaded in
> a
> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
> >> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> >> I have also created a ticket [3]
> >>
> >> Kind regards
> >>
> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >>
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra 
> >> wrote:
> >>
> >> > Hello Ilya,
> >> >
> >> > Thank you for your email. Yes, before we remove any integration we
> will
> >> > move them to another repository and release.
> >> >
> >> > We are thinking we will start with Pub/Sub Streamer implementation and
> >> > continue to move other integration as mentioned in the following doc.
> >> >
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> > ilya.kasnach...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello!
> >> > >
> >> > > My take on this: we want to offload some of integrations, but please
> >> > commit
> >> > > and release them to another project first, and only then we will
> >> decide
> >> > to
> >> > > remove them from our repository.
> >> > >
> >> > > Please also make sure they are available with same artifact name and
> >> API.
> >> > > If this has to be changed, let's wait for AI 3.0
> >> > >
> >> > > Regards,
> >> > > --
> >> > > Ilya Kasnacheev
> >> > >
> >> > >
> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra  >:
> >> > >
> >> > > > Hello Emmanouil,
> >> > > >
> >> > > > As discussed earlier, I discussed with Apache Bahir community and
> >> they
> >> > > are
> >> > > > interested to have Apache Ignite extensions as part of Apache
> Bahir
> >> > > > project.
> >> > > >
> >> > > > Can you please share Pub/Sub streamer implementation details with
> >> > Apache
> >> > > > Bahir community and request for feedback.
> >> > > >
> >> > > > https://bahir.apache.org/
> >> > > >
> >> > > > I have also requested for contributor access in Jira for Apache
> >> Bahir
> >> > > > project so that I can create issues and assign to myself.
> >> > > >
> >> > > > I can help with code reviews as well.
> >> > > >
> >> > > > Here is the email thread for reference
> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> saikat.mai...@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > As discussed I have created following discussion threads on our
> >> > > migration
> >> > > > > proposals for Apache Ignite extensions:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > >
> >> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > >
> >> > > > > I will share update as I hear more information.
> >> > > > >
> >> > > > > Regards,
> >> > > > > Saikat
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

Can you please take a looks at dev utils, if this is something you are
looking for https://github.com/apache/ignite/tree/master/modules/dev-utils?

IMO, if you can release Pub/Sub server in maven and then use it as
dependency, that would be great.

Regards,
Saikat

On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra 
wrote:

> Hi Emmanouil,
>
> The master branch has been initialized. I have raised a PR to migrate
> Flink module to ignite-extensions.
>
> PR https://github.com/apache/ignite-extensions/pull/1
> Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>
> Regards,
> Saikat
>
>
> On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com> wrote:
>
>> Hi all!
>>
>> This is an update. The integration is tested with mocks based on the
>> original Ignite repo [1].
>> I am currently waiting for the first commits on the new integration
>> project
>> to take place in order to create my pull request accordingly to the
>> initial
>> commits.
>> In order to simulate the Pub/Sub server in a test I used a test server
>> implementation [2]. Unfortunately they don't have this util uploaded in a
>> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
>> I have also created a ticket [3]
>>
>> Kind regards
>>
>> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra 
>> wrote:
>>
>> > Hello Ilya,
>> >
>> > Thank you for your email. Yes, before we remove any integration we will
>> > move them to another repository and release.
>> >
>> > We are thinking we will start with Pub/Sub Streamer implementation and
>> > continue to move other integration as mentioned in the following doc.
>> >
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > ilya.kasnach...@gmail.com>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > My take on this: we want to offload some of integrations, but please
>> > commit
>> > > and release them to another project first, and only then we will
>> decide
>> > to
>> > > remove them from our repository.
>> > >
>> > > Please also make sure they are available with same artifact name and
>> API.
>> > > If this has to be changed, let's wait for AI 3.0
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra :
>> > >
>> > > > Hello Emmanouil,
>> > > >
>> > > > As discussed earlier, I discussed with Apache Bahir community and
>> they
>> > > are
>> > > > interested to have Apache Ignite extensions as part of Apache Bahir
>> > > > project.
>> > > >
>> > > > Can you please share Pub/Sub streamer implementation details with
>> > Apache
>> > > > Bahir community and request for feedback.
>> > > >
>> > > > https://bahir.apache.org/
>> > > >
>> > > > I have also requested for contributor access in Jira for Apache
>> Bahir
>> > > > project so that I can create issues and assign to myself.
>> > > >
>> > > > I can help with code reviews as well.
>> > > >
>> > > > Here is the email thread for reference
>> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> saikat.mai...@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > As discussed I have created following discussion threads on our
>> > > migration
>> > > > > proposals for Apache Ignite extensions:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > >
>> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > >
>> > > > > I will share update as I hear more information.
>> > > > >
>> > > > > Regards,
>> > > > > Saikat
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > saikat.mai...@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hello Denis,
>> > > > >>
>> > > > >> Sure, sounds good. I will create a separate discussion thread to
>> get
>> > > > >> community feedback on whether we
>> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> > > > >>
>> > > > >> Regards,
>> > > > >> Saikat
>> > > > >>
>> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
>> > > wrote:
>> > > > >>
>> > > > >>> Folks,
>> > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

The master branch has been initialized. I have raised a PR to migrate Flink
module to ignite-extensions.

PR https://github.com/apache/ignite-extensions/pull/1
Jira : https://issues.apache.org/jira/browse/IGNITE-12356

Regards,
Saikat


On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras 
wrote:

> Hi all!
>
> This is an update. The integration is tested with mocks based on the
> original Ignite repo [1].
> I am currently waiting for the first commits on the new integration project
> to take place in order to create my pull request accordingly to the initial
> commits.
> In order to simulate the Pub/Sub server in a test I used a test server
> implementation [2]. Unfortunately they don't have this util uploaded in a
> maven repo, thus it needs to be hosted on a maven repo (Apache License
> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> I have also created a ticket [3]
>
> Kind regards
>
> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra 
> wrote:
>
> > Hello Ilya,
> >
> > Thank you for your email. Yes, before we remove any integration we will
> > move them to another repository and release.
> >
> > We are thinking we will start with Pub/Sub Streamer implementation and
> > continue to move other integration as mentioned in the following doc.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >
> > Regards,
> > Saikat
> >
> >
> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > My take on this: we want to offload some of integrations, but please
> > commit
> > > and release them to another project first, and only then we will decide
> > to
> > > remove them from our repository.
> > >
> > > Please also make sure they are available with same artifact name and
> API.
> > > If this has to be changed, let's wait for AI 3.0
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra :
> > >
> > > > Hello Emmanouil,
> > > >
> > > > As discussed earlier, I discussed with Apache Bahir community and
> they
> > > are
> > > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > > project.
> > > >
> > > > Can you please share Pub/Sub streamer implementation details with
> > Apache
> > > > Bahir community and request for feedback.
> > > >
> > > > https://bahir.apache.org/
> > > >
> > > > I have also requested for contributor access in Jira for Apache Bahir
> > > > project so that I can create issues and assign to myself.
> > > >
> > > > I can help with code reviews as well.
> > > >
> > > > Here is the email thread for reference
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > As discussed I have created following discussion threads on our
> > > migration
> > > > > proposals for Apache Ignite extensions:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >
> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >
> > > > > I will share update as I hear more information.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > saikat.mai...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hello Denis,
> > > > >>
> > > > >> Sure, sounds good. I will create a separate discussion thread to
> get
> > > > >> community feedback on whether we
> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
> > > wrote:
> > > > >>
> > > > >>> Folks,
> > > > >>>
> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> > thanks
> > > > for
> > > > >>> doing the research. Why I believe the * idea* fits us well:
> > > > >>>
> > > > >>>- All integrations can be stored in separate Github
> repositories
> > > and
> > > > >>>have their dev lifecycles. We've not obliged to couple all the
> > > > >>> integrations
> > > > >>>in a single repo. For instance, Spark can be located in a
> > > dedicated
> > > > >>> repo
> > > > >>>while streaming integrations might 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all!

This is an update. The integration is tested with mocks based on the
original Ignite repo [1].
I am currently waiting for the first commits on the new integration project
to take place in order to create my pull request accordingly to the initial
commits.
In order to simulate the Pub/Sub server in a test I used a test server
implementation [2]. Unfortunately they don't have this util uploaded in a
maven repo, thus it needs to be hosted on a maven repo (Apache License
2.0). Is there a repo for utilities that Ignite uses for the test/build?
I have also created a ticket [3]

Kind regards

[1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
[2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
[3] https://issues.apache.org/jira/browse/IGNITE-12262

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 25 Oct 2019 at 03:35, Saikat Maitra  wrote:

> Hello Ilya,
>
> Thank you for your email. Yes, before we remove any integration we will
> move them to another repository and release.
>
> We are thinking we will start with Pub/Sub Streamer implementation and
> continue to move other integration as mentioned in the following doc.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>
> Regards,
> Saikat
>
>
> On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> wrote:
>
> > Hello!
> >
> > My take on this: we want to offload some of integrations, but please
> commit
> > and release them to another project first, and only then we will decide
> to
> > remove them from our repository.
> >
> > Please also make sure they are available with same artifact name and API.
> > If this has to be changed, let's wait for AI 3.0
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra :
> >
> > > Hello Emmanouil,
> > >
> > > As discussed earlier, I discussed with Apache Bahir community and they
> > are
> > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > project.
> > >
> > > Can you please share Pub/Sub streamer implementation details with
> Apache
> > > Bahir community and request for feedback.
> > >
> > > https://bahir.apache.org/
> > >
> > > I have also requested for contributor access in Jira for Apache Bahir
> > > project so that I can create issues and assign to myself.
> > >
> > > I can help with code reviews as well.
> > >
> > > Here is the email thread for reference
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra  >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > As discussed I have created following discussion threads on our
> > migration
> > > > proposals for Apache Ignite extensions:
> > > >
> > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >
> > > > I will share update as I hear more information.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hello Denis,
> > > >>
> > > >> Sure, sounds good. I will create a separate discussion thread to get
> > > >> community feedback on whether we
> > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
> > wrote:
> > > >>
> > > >>> Folks,
> > > >>>
> > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> thanks
> > > for
> > > >>> doing the research. Why I believe the * idea* fits us well:
> > > >>>
> > > >>>- All integrations can be stored in separate Github repositories
> > and
> > > >>>have their dev lifecycles. We've not obliged to couple all the
> > > >>> integrations
> > > >>>in a single repo. For instance, Spark can be located in a
> > dedicated
> > > >>> repo
> > > >>>while streaming integrations might be in a single one. It's up
> to
> > > us.
> > > >>>- All the repositories and integrations belong to ASF. We're not
> > > >>> dumping
> > > >>>them on Github but rather keep supporting and developing in
> > > >>> accordance with
> > > >>>ASF vision and practices.
> > > >>>- There is a way to reward contributors via committership and
> PMC
> > > >>>membership. You won't get this if a project is just one of the
> > > >>> millions of
> > > >>>Github projects.
> > > >>>
> > > >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> > > >>> dev-list
> > > >>> discussion to involve more community members and 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello Ilya,

Thank you for your email. Yes, before we remove any integration we will
move them to another repository and release.

We are thinking we will start with Pub/Sub Streamer implementation and
continue to move other integration as mentioned in the following doc.

https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations

Regards,
Saikat


On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> My take on this: we want to offload some of integrations, but please commit
> and release them to another project first, and only then we will decide to
> remove them from our repository.
>
> Please also make sure they are available with same artifact name and API.
> If this has to be changed, let's wait for AI 3.0
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 23 окт. 2019 г. в 03:23, Saikat Maitra :
>
> > Hello Emmanouil,
> >
> > As discussed earlier, I discussed with Apache Bahir community and they
> are
> > interested to have Apache Ignite extensions as part of Apache Bahir
> > project.
> >
> > Can you please share Pub/Sub streamer implementation details with Apache
> > Bahir community and request for feedback.
> >
> > https://bahir.apache.org/
> >
> > I have also requested for contributor access in Jira for Apache Bahir
> > project so that I can create issues and assign to myself.
> >
> > I can help with code reviews as well.
> >
> > Here is the email thread for reference
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
> > wrote:
> >
> > > Hello,
> > >
> > > As discussed I have created following discussion threads on our
> migration
> > > proposals for Apache Ignite extensions:
> > >
> > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >
> > > I will share update as I hear more information.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra  >
> > > wrote:
> > >
> > >> Hello Denis,
> > >>
> > >> Sure, sounds good. I will create a separate discussion thread to get
> > >> community feedback on whether we
> > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> The concept of Apache Bahir is precisely what we need! Saikat, thanks
> > for
> > >>> doing the research. Why I believe the * idea* fits us well:
> > >>>
> > >>>- All integrations can be stored in separate Github repositories
> and
> > >>>have their dev lifecycles. We've not obliged to couple all the
> > >>> integrations
> > >>>in a single repo. For instance, Spark can be located in a
> dedicated
> > >>> repo
> > >>>while streaming integrations might be in a single one. It's up to
> > us.
> > >>>- All the repositories and integrations belong to ASF. We're not
> > >>> dumping
> > >>>them on Github but rather keep supporting and developing in
> > >>> accordance with
> > >>>ASF vision and practices.
> > >>>- There is a way to reward contributors via committership and PMC
> > >>>membership. You won't get this if a project is just one of the
> > >>> millions of
> > >>>Github projects.
> > >>>
> > >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> > >>> dev-list
> > >>> discussion to involve more community members and referring to this
> > >>> thread?
> > >>> I think the primary question the community needs to answer whether we
> > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > >>>
> > >>> -
> > >>> Denis
> > >>>
> > >>>
> > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > zaleslaw@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > Maybe we could move all our Streaming Integrations there, but what
> is
> > >>> about
> > >>> > maintaining and committer permissions to the new repositories?
> > >>> > I see the list of the committers and PMC members there
> > >>> > https://bahir.apache.org/community-members/
> > >>> >
> > >>> > Could somebody from Ignite community be added to this list as a
> > >>> > PMC/committer to maintain Ignite integrations?
> > >>> > If yes, I'd happy to join this project and collaborate with these
> > guys
> > >>> > together
> > >>> > If no, and we should start from the zero level with external PRs -
> > >>> hmmm,
> > >>> > it's better to have our own external repository with ApacheBahirr
> > >>> ideology.
> > >>> >
> > >>> > Also, I agree, that all connectors could be moved there (in
> > >>> ApacheBahirr
> > >>> > like repository), but not all modules from the page
> > >>> >
> > >>> >
> > >>>
> >
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Ilya Kasnacheev]

Hello!

My take on this: we want to offload some of integrations, but please commit
and release them to another project first, and only then we will decide to
remove them from our repository.

Please also make sure they are available with same artifact name and API.
If this has to be changed, let's wait for AI 3.0

Regards,
-- 
Ilya Kasnacheev


ср, 23 окт. 2019 г. в 03:23, Saikat Maitra :

> Hello Emmanouil,
>
> As discussed earlier, I discussed with Apache Bahir community and they are
> interested to have Apache Ignite extensions as part of Apache Bahir
> project.
>
> Can you please share Pub/Sub streamer implementation details with Apache
> Bahir community and request for feedback.
>
> https://bahir.apache.org/
>
> I have also requested for contributor access in Jira for Apache Bahir
> project so that I can create issues and assign to myself.
>
> I can help with code reviews as well.
>
> Here is the email thread for reference
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>
> Regards,
> Saikat
>
>
>
>
>
>
>
> On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
> wrote:
>
> > Hello,
> >
> > As discussed I have created following discussion threads on our migration
> > proposals for Apache Ignite extensions:
> >
> >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >
> > I will share update as I hear more information.
> >
> > Regards,
> > Saikat
> >
> >
> >
> > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra 
> > wrote:
> >
> >> Hello Denis,
> >>
> >> Sure, sounds good. I will create a separate discussion thread to get
> >> community feedback on whether we
> >> should join the Bahir or kick-off "Ignite Extensions" project.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda  wrote:
> >>
> >>> Folks,
> >>>
> >>> The concept of Apache Bahir is precisely what we need! Saikat, thanks
> for
> >>> doing the research. Why I believe the * idea* fits us well:
> >>>
> >>>- All integrations can be stored in separate Github repositories and
> >>>have their dev lifecycles. We've not obliged to couple all the
> >>> integrations
> >>>in a single repo. For instance, Spark can be located in a dedicated
> >>> repo
> >>>while streaming integrations might be in a single one. It's up to
> us.
> >>>- All the repositories and integrations belong to ASF. We're not
> >>> dumping
> >>>them on Github but rather keep supporting and developing in
> >>> accordance with
> >>>ASF vision and practices.
> >>>- There is a way to reward contributors via committership and PMC
> >>>membership. You won't get this if a project is just one of the
> >>> millions of
> >>>Github projects.
> >>>
> >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> >>> dev-list
> >>> discussion to involve more community members and referring to this
> >>> thread?
> >>> I think the primary question the community needs to answer whether we
> >>> should join the Bahir or kick-off "Ignite Extensions" project?
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> zaleslaw@gmail.com>
> >>> wrote:
> >>>
> >>> > Maybe we could move all our Streaming Integrations there, but what is
> >>> about
> >>> > maintaining and committer permissions to the new repositories?
> >>> > I see the list of the committers and PMC members there
> >>> > https://bahir.apache.org/community-members/
> >>> >
> >>> > Could somebody from Ignite community be added to this list as a
> >>> > PMC/committer to maintain Ignite integrations?
> >>> > If yes, I'd happy to join this project and collaborate with these
> guys
> >>> > together
> >>> > If no, and we should start from the zero level with external PRs -
> >>> hmmm,
> >>> > it's better to have our own external repository with ApacheBahirr
> >>> ideology.
> >>> >
> >>> > Also, I agree, that all connectors could be moved there (in
> >>> ApacheBahirr
> >>> > like repository), but not all modules from the page
> >>> >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > because
> >>> > they use different parts of core modules and could be developed with
> >>> > different velocity.
> >>> >
> >>> > In reality, before creation of new repositories we need wide
> discussion
> >>> > based on the document mentioned above.
> >>> >
> >>> > P.S. Of course, we could experiment in the next release 2.8 with one
> >>> or two
> >>> > integrations like twitter or ZeroMQ
> >>> > Also, I'd like Denis Magda idea for the separate repositories in
> Apache
> >>> > Github to maintain them by anybody.
> >>> >
> >>> >
> >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra  >:
> >>> >
> >>> > > Hi Denis, Emmanouil
> >>> > >
> >>> > > Thank you for your email. I think 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Denis,

Thank you for your response. Yes, I have created the following discussion
thread for Ignite extensions migration.

http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html

Regards,
Saikat

On Tue, Oct 22, 2019 at 8:17 PM Denis Magda  wrote:

> Saikat, thanks for checking with Bahir!
>
> However, let’s run this idea through the rest of the community before
> getting down to the implementation. As an alternate option, we might adopt
> their approach for Ignite extensions but keep those within Ignite community
> in separate repositories. Would you mind starting a separate discussion
> with Ignite dev?
>
> Denis
>
> On Tuesday, October 22, 2019, Saikat Maitra 
> wrote:
>
> > Hello Emmanouil,
> >
> > As discussed earlier, I discussed with Apache Bahir community and they
> are
> > interested to have Apache Ignite extensions as part of Apache Bahir
> > project.
> >
> > Can you please share Pub/Sub streamer implementation details with Apache
> > Bahir community and request for feedback.
> >
> > https://bahir.apache.org/
> >
> > I have also requested for contributor access in Jira for Apache Bahir
> > project so that I can create issues and assign to myself.
> >
> > I can help with code reviews as well.
> >
> > Here is the email thread for reference
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
> > wrote:
> >
> > > Hello,
> > >
> > > As discussed I have created following discussion threads on our
> migration
> > > proposals for Apache Ignite extensions:
> > >
> > >
> > > http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-
> > Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-
> > project-td29829.html
> > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >
> > > I will share update as I hear more information.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra  >
> > > wrote:
> > >
> > >> Hello Denis,
> > >>
> > >> Sure, sounds good. I will create a separate discussion thread to get
> > >> community feedback on whether we
> > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> The concept of Apache Bahir is precisely what we need! Saikat, thanks
> > for
> > >>> doing the research. Why I believe the * idea* fits us well:
> > >>>
> > >>>- All integrations can be stored in separate Github repositories
> and
> > >>>have their dev lifecycles. We've not obliged to couple all the
> > >>> integrations
> > >>>in a single repo. For instance, Spark can be located in a
> dedicated
> > >>> repo
> > >>>while streaming integrations might be in a single one. It's up to
> > us.
> > >>>- All the repositories and integrations belong to ASF. We're not
> > >>> dumping
> > >>>them on Github but rather keep supporting and developing in
> > >>> accordance with
> > >>>ASF vision and practices.
> > >>>- There is a way to reward contributors via committership and PMC
> > >>>membership. You won't get this if a project is just one of the
> > >>> millions of
> > >>>Github projects.
> > >>>
> > >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> > >>> dev-list
> > >>> discussion to involve more community members and referring to this
> > >>> thread?
> > >>> I think the primary question the community needs to answer whether we
> > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > >>>
> > >>> -
> > >>> Denis
> > >>>
> > >>>
> > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > zaleslaw@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > Maybe we could move all our Streaming Integrations there, but what
> is
> > >>> about
> > >>> > maintaining and committer permissions to the new repositories?
> > >>> > I see the list of the committers and PMC members there
> > >>> > https://bahir.apache.org/community-members/
> > >>> >
> > >>> > Could somebody from Ignite community be added to this list as a
> > >>> > PMC/committer to maintain Ignite integrations?
> > >>> > If yes, I'd happy to join this project and collaborate with these
> > guys
> > >>> > together
> > >>> > If no, and we should start from the zero level with external PRs -
> > >>> hmmm,
> > >>> > it's better to have our own external repository with ApacheBahirr
> > >>> ideology.
> > >>> >
> > >>> > Also, I agree, that all connectors could be moved there (in
> > >>> ApacheBahirr
> > >>> > like repository), but not all modules from the page
> > >>> >
> > >>> >
> > >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >>> > because
> > >>> > they 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Saikat, thanks for checking with Bahir!

However, let’s run this idea through the rest of the community before
getting down to the implementation. As an alternate option, we might adopt
their approach for Ignite extensions but keep those within Ignite community
in separate repositories. Would you mind starting a separate discussion
with Ignite dev?

Denis

On Tuesday, October 22, 2019, Saikat Maitra  wrote:

> Hello Emmanouil,
>
> As discussed earlier, I discussed with Apache Bahir community and they are
> interested to have Apache Ignite extensions as part of Apache Bahir
> project.
>
> Can you please share Pub/Sub streamer implementation details with Apache
> Bahir community and request for feedback.
>
> https://bahir.apache.org/
>
> I have also requested for contributor access in Jira for Apache Bahir
> project so that I can create issues and assign to myself.
>
> I can help with code reviews as well.
>
> Here is the email thread for reference
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>
> Regards,
> Saikat
>
>
>
>
>
>
>
> On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
> wrote:
>
> > Hello,
> >
> > As discussed I have created following discussion threads on our migration
> > proposals for Apache Ignite extensions:
> >
> >
> > http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-
> Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-
> project-td29829.html
> >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >
> > I will share update as I hear more information.
> >
> > Regards,
> > Saikat
> >
> >
> >
> > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra 
> > wrote:
> >
> >> Hello Denis,
> >>
> >> Sure, sounds good. I will create a separate discussion thread to get
> >> community feedback on whether we
> >> should join the Bahir or kick-off "Ignite Extensions" project.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda  wrote:
> >>
> >>> Folks,
> >>>
> >>> The concept of Apache Bahir is precisely what we need! Saikat, thanks
> for
> >>> doing the research. Why I believe the * idea* fits us well:
> >>>
> >>>- All integrations can be stored in separate Github repositories and
> >>>have their dev lifecycles. We've not obliged to couple all the
> >>> integrations
> >>>in a single repo. For instance, Spark can be located in a dedicated
> >>> repo
> >>>while streaming integrations might be in a single one. It's up to
> us.
> >>>- All the repositories and integrations belong to ASF. We're not
> >>> dumping
> >>>them on Github but rather keep supporting and developing in
> >>> accordance with
> >>>ASF vision and practices.
> >>>- There is a way to reward contributors via committership and PMC
> >>>membership. You won't get this if a project is just one of the
> >>> millions of
> >>>Github projects.
> >>>
> >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> >>> dev-list
> >>> discussion to involve more community members and referring to this
> >>> thread?
> >>> I think the primary question the community needs to answer whether we
> >>> should join the Bahir or kick-off "Ignite Extensions" project?
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> zaleslaw@gmail.com>
> >>> wrote:
> >>>
> >>> > Maybe we could move all our Streaming Integrations there, but what is
> >>> about
> >>> > maintaining and committer permissions to the new repositories?
> >>> > I see the list of the committers and PMC members there
> >>> > https://bahir.apache.org/community-members/
> >>> >
> >>> > Could somebody from Ignite community be added to this list as a
> >>> > PMC/committer to maintain Ignite integrations?
> >>> > If yes, I'd happy to join this project and collaborate with these
> guys
> >>> > together
> >>> > If no, and we should start from the zero level with external PRs -
> >>> hmmm,
> >>> > it's better to have our own external repository with ApacheBahirr
> >>> ideology.
> >>> >
> >>> > Also, I agree, that all connectors could be moved there (in
> >>> ApacheBahirr
> >>> > like repository), but not all modules from the page
> >>> >
> >>> >
> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > because
> >>> > they use different parts of core modules and could be developed with
> >>> > different velocity.
> >>> >
> >>> > In reality, before creation of new repositories we need wide
> discussion
> >>> > based on the document mentioned above.
> >>> >
> >>> > P.S. Of course, we could experiment in the next release 2.8 with one
> >>> or two
> >>> > integrations like twitter or ZeroMQ
> >>> > Also, I'd like Denis Magda idea for the separate repositories in
> Apache
> >>> > Github to maintain them by anybody.
> >>> >
> >>> >
> >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra  >:
> >>> >
> >>> > > Hi Denis, Emmanouil
> >>> > >
> >>> > > Thank you for 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello Emmanouil,

As discussed earlier, I discussed with Apache Bahir community and they are
interested to have Apache Ignite extensions as part of Apache Bahir project.

Can you please share Pub/Sub streamer implementation details with Apache
Bahir community and request for feedback.

https://bahir.apache.org/

I have also requested for contributor access in Jira for Apache Bahir
project so that I can create issues and assign to myself.

I can help with code reviews as well.

Here is the email thread for reference
https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html

Regards,
Saikat







On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
wrote:

> Hello,
>
> As discussed I have created following discussion threads on our migration
> proposals for Apache Ignite extensions:
>
>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>
> I will share update as I hear more information.
>
> Regards,
> Saikat
>
>
>
> On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra 
> wrote:
>
>> Hello Denis,
>>
>> Sure, sounds good. I will create a separate discussion thread to get
>> community feedback on whether we
>> should join the Bahir or kick-off "Ignite Extensions" project.
>>
>> Regards,
>> Saikat
>>
>> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda  wrote:
>>
>>> Folks,
>>>
>>> The concept of Apache Bahir is precisely what we need! Saikat, thanks for
>>> doing the research. Why I believe the * idea* fits us well:
>>>
>>>- All integrations can be stored in separate Github repositories and
>>>have their dev lifecycles. We've not obliged to couple all the
>>> integrations
>>>in a single repo. For instance, Spark can be located in a dedicated
>>> repo
>>>while streaming integrations might be in a single one. It's up to us.
>>>- All the repositories and integrations belong to ASF. We're not
>>> dumping
>>>them on Github but rather keep supporting and developing in
>>> accordance with
>>>ASF vision and practices.
>>>- There is a way to reward contributors via committership and PMC
>>>membership. You won't get this if a project is just one of the
>>> millions of
>>>Github projects.
>>>
>>> Saikat, as Ignite PMC member, could you please kick-off a separate
>>> dev-list
>>> discussion to involve more community members and referring to this
>>> thread?
>>> I think the primary question the community needs to answer whether we
>>> should join the Bahir or kick-off "Ignite Extensions" project?
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev 
>>> wrote:
>>>
>>> > Maybe we could move all our Streaming Integrations there, but what is
>>> about
>>> > maintaining and committer permissions to the new repositories?
>>> > I see the list of the committers and PMC members there
>>> > https://bahir.apache.org/community-members/
>>> >
>>> > Could somebody from Ignite community be added to this list as a
>>> > PMC/committer to maintain Ignite integrations?
>>> > If yes, I'd happy to join this project and collaborate with these guys
>>> > together
>>> > If no, and we should start from the zero level with external PRs -
>>> hmmm,
>>> > it's better to have our own external repository with ApacheBahirr
>>> ideology.
>>> >
>>> > Also, I agree, that all connectors could be moved there (in
>>> ApacheBahirr
>>> > like repository), but not all modules from the page
>>> >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > because
>>> > they use different parts of core modules and could be developed with
>>> > different velocity.
>>> >
>>> > In reality, before creation of new repositories we need wide discussion
>>> > based on the document mentioned above.
>>> >
>>> > P.S. Of course, we could experiment in the next release 2.8 with one
>>> or two
>>> > integrations like twitter or ZeroMQ
>>> > Also, I'd like Denis Magda idea for the separate repositories in Apache
>>> > Github to maintain them by anybody.
>>> >
>>> >
>>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :
>>> >
>>> > > Hi Denis, Emmanouil
>>> > >
>>> > > Thank you for your email. I think creating a separate integration
>>> project
>>> > > in github and also proposing it as an apache incubator project will
>>> be a
>>> > > good move. The other separate integration modules
>>> > >
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > can
>>> > > be moved to this new apache incubator project.
>>> > >
>>> > > There are similar integration available for Flink and Spark in Apache
>>> > Bahir
>>> > > https://bahir.apache.org/
>>> > >
>>> > > Do you think we should reach out to Apache Bahir community with a
>>> > proposal
>>> > > or we should plan to create a separate Apache Incubator 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello,

As discussed I have created following discussion threads on our migration
proposals for Apache Ignite extensions:

http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html

https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html

I will share update as I hear more information.

Regards,
Saikat



On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra 
wrote:

> Hello Denis,
>
> Sure, sounds good. I will create a separate discussion thread to get
> community feedback on whether we
> should join the Bahir or kick-off "Ignite Extensions" project.
>
> Regards,
> Saikat
>
> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda  wrote:
>
>> Folks,
>>
>> The concept of Apache Bahir is precisely what we need! Saikat, thanks for
>> doing the research. Why I believe the * idea* fits us well:
>>
>>- All integrations can be stored in separate Github repositories and
>>have their dev lifecycles. We've not obliged to couple all the
>> integrations
>>in a single repo. For instance, Spark can be located in a dedicated
>> repo
>>while streaming integrations might be in a single one. It's up to us.
>>- All the repositories and integrations belong to ASF. We're not
>> dumping
>>them on Github but rather keep supporting and developing in accordance
>> with
>>ASF vision and practices.
>>- There is a way to reward contributors via committership and PMC
>>membership. You won't get this if a project is just one of the
>> millions of
>>Github projects.
>>
>> Saikat, as Ignite PMC member, could you please kick-off a separate
>> dev-list
>> discussion to involve more community members and referring to this thread?
>> I think the primary question the community needs to answer whether we
>> should join the Bahir or kick-off "Ignite Extensions" project?
>>
>> -
>> Denis
>>
>>
>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev 
>> wrote:
>>
>> > Maybe we could move all our Streaming Integrations there, but what is
>> about
>> > maintaining and committer permissions to the new repositories?
>> > I see the list of the committers and PMC members there
>> > https://bahir.apache.org/community-members/
>> >
>> > Could somebody from Ignite community be added to this list as a
>> > PMC/committer to maintain Ignite integrations?
>> > If yes, I'd happy to join this project and collaborate with these guys
>> > together
>> > If no, and we should start from the zero level with external PRs - hmmm,
>> > it's better to have our own external repository with ApacheBahirr
>> ideology.
>> >
>> > Also, I agree, that all connectors could be moved there (in ApacheBahirr
>> > like repository), but not all modules from the page
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > because
>> > they use different parts of core modules and could be developed with
>> > different velocity.
>> >
>> > In reality, before creation of new repositories we need wide discussion
>> > based on the document mentioned above.
>> >
>> > P.S. Of course, we could experiment in the next release 2.8 with one or
>> two
>> > integrations like twitter or ZeroMQ
>> > Also, I'd like Denis Magda idea for the separate repositories in Apache
>> > Github to maintain them by anybody.
>> >
>> >
>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :
>> >
>> > > Hi Denis, Emmanouil
>> > >
>> > > Thank you for your email. I think creating a separate integration
>> project
>> > > in github and also proposing it as an apache incubator project will
>> be a
>> > > good move. The other separate integration modules
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > can
>> > > be moved to this new apache incubator project.
>> > >
>> > > There are similar integration available for Flink and Spark in Apache
>> > Bahir
>> > > https://bahir.apache.org/
>> > >
>> > > Do you think we should reach out to Apache Bahir community with a
>> > proposal
>> > > or we should plan to create a separate Apache Incubator project?
>> > >
>> > > Regards,
>> > > Saikat
>> > >
>> > >
>> > >
>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
>> > > gkatzio...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi all!
>> > > >
>> > > > Based on the discussions most probably this is going to be on
>> another
>> > > > GitHub repo. Therefore I will prepare a standalone project with the
>> > > Pub/Sub
>> > > > module and once the repository is created a pull request shall be
>> > issued.
>> > > > If there is anything else I can do in order to facilitate this
>> please
>> > let
>> > > > me know.
>> > > >
>> > > > If a ticket is created for this issue, I shall be able to create a
>> pull
>> > > > request and thus a build will take place on ignite's CI.
>> > > > If everyone is aligned with adding a Pub/Sub implementation may I

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello Denis,

Sure, sounds good. I will create a separate discussion thread to get
community feedback on whether we
should join the Bahir or kick-off "Ignite Extensions" project.

Regards,
Saikat

On Thu, Oct 17, 2019 at 2:21 PM Denis Magda  wrote:

> Folks,
>
> The concept of Apache Bahir is precisely what we need! Saikat, thanks for
> doing the research. Why I believe the * idea* fits us well:
>
>- All integrations can be stored in separate Github repositories and
>have their dev lifecycles. We've not obliged to couple all the
> integrations
>in a single repo. For instance, Spark can be located in a dedicated repo
>while streaming integrations might be in a single one. It's up to us.
>- All the repositories and integrations belong to ASF. We're not dumping
>them on Github but rather keep supporting and developing in accordance
> with
>ASF vision and practices.
>- There is a way to reward contributors via committership and PMC
>membership. You won't get this if a project is just one of the millions
> of
>Github projects.
>
> Saikat, as Ignite PMC member, could you please kick-off a separate dev-list
> discussion to involve more community members and referring to this thread?
> I think the primary question the community needs to answer whether we
> should join the Bahir or kick-off "Ignite Extensions" project?
>
> -
> Denis
>
>
> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev 
> wrote:
>
> > Maybe we could move all our Streaming Integrations there, but what is
> about
> > maintaining and committer permissions to the new repositories?
> > I see the list of the committers and PMC members there
> > https://bahir.apache.org/community-members/
> >
> > Could somebody from Ignite community be added to this list as a
> > PMC/committer to maintain Ignite integrations?
> > If yes, I'd happy to join this project and collaborate with these guys
> > together
> > If no, and we should start from the zero level with external PRs - hmmm,
> > it's better to have our own external repository with ApacheBahirr
> ideology.
> >
> > Also, I agree, that all connectors could be moved there (in ApacheBahirr
> > like repository), but not all modules from the page
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > because
> > they use different parts of core modules and could be developed with
> > different velocity.
> >
> > In reality, before creation of new repositories we need wide discussion
> > based on the document mentioned above.
> >
> > P.S. Of course, we could experiment in the next release 2.8 with one or
> two
> > integrations like twitter or ZeroMQ
> > Also, I'd like Denis Magda idea for the separate repositories in Apache
> > Github to maintain them by anybody.
> >
> >
> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :
> >
> > > Hi Denis, Emmanouil
> > >
> > > Thank you for your email. I think creating a separate integration
> project
> > > in github and also proposing it as an apache incubator project will be
> a
> > > good move. The other separate integration modules
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > can
> > > be moved to this new apache incubator project.
> > >
> > > There are similar integration available for Flink and Spark in Apache
> > Bahir
> > > https://bahir.apache.org/
> > >
> > > Do you think we should reach out to Apache Bahir community with a
> > proposal
> > > or we should plan to create a separate Apache Incubator project?
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > Based on the discussions most probably this is going to be on another
> > > > GitHub repo. Therefore I will prepare a standalone project with the
> > > Pub/Sub
> > > > module and once the repository is created a pull request shall be
> > issued.
> > > > If there is anything else I can do in order to facilitate this please
> > let
> > > > me know.
> > > >
> > > > If a ticket is created for this issue, I shall be able to create a
> pull
> > > > request and thus a build will take place on ignite's CI.
> > > > If everyone is aligned with adding a Pub/Sub implementation may I
> > create
> > > a
> > > > ticket on JIRA?
> > > >
> > > > My next question has to do on how Ignite configures the build jobs in
> > > Team
> > > > City. The unit tests for my implementation need a local Pub/Sub
> server
> > up
> > > > and running (they provide one for testing purposes). Will this take
> > > effect
> > > > in a form of a build script inside my implementation or is it
> something
> > > > that should be configured on Team City?
> > > >
> > > > Kind regards,
> > > > Emmanouil
> > > >
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Folks,

The concept of Apache Bahir is precisely what we need! Saikat, thanks for
doing the research. Why I believe the * idea* fits us well:

   - All integrations can be stored in separate Github repositories and
   have their dev lifecycles. We've not obliged to couple all the integrations
   in a single repo. For instance, Spark can be located in a dedicated repo
   while streaming integrations might be in a single one. It's up to us.
   - All the repositories and integrations belong to ASF. We're not dumping
   them on Github but rather keep supporting and developing in accordance with
   ASF vision and practices.
   - There is a way to reward contributors via committership and PMC
   membership. You won't get this if a project is just one of the millions of
   Github projects.

Saikat, as Ignite PMC member, could you please kick-off a separate dev-list
discussion to involve more community members and referring to this thread?
I think the primary question the community needs to answer whether we
should join the Bahir or kick-off "Ignite Extensions" project?

-
Denis


On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev 
wrote:

> Maybe we could move all our Streaming Integrations there, but what is about
> maintaining and committer permissions to the new repositories?
> I see the list of the committers and PMC members there
> https://bahir.apache.org/community-members/
>
> Could somebody from Ignite community be added to this list as a
> PMC/committer to maintain Ignite integrations?
> If yes, I'd happy to join this project and collaborate with these guys
> together
> If no, and we should start from the zero level with external PRs - hmmm,
> it's better to have our own external repository with ApacheBahirr ideology.
>
> Also, I agree, that all connectors could be moved there (in ApacheBahirr
> like repository), but not all modules from the page
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> because
> they use different parts of core modules and could be developed with
> different velocity.
>
> In reality, before creation of new repositories we need wide discussion
> based on the document mentioned above.
>
> P.S. Of course, we could experiment in the next release 2.8 with one or two
> integrations like twitter or ZeroMQ
> Also, I'd like Denis Magda idea for the separate repositories in Apache
> Github to maintain them by anybody.
>
>
> чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :
>
> > Hi Denis, Emmanouil
> >
> > Thank you for your email. I think creating a separate integration project
> > in github and also proposing it as an apache incubator project will be a
> > good move. The other separate integration modules
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > can
> > be moved to this new apache incubator project.
> >
> > There are similar integration available for Flink and Spark in Apache
> Bahir
> > https://bahir.apache.org/
> >
> > Do you think we should reach out to Apache Bahir community with a
> proposal
> > or we should plan to create a separate Apache Incubator project?
> >
> > Regards,
> > Saikat
> >
> >
> >
> > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > Based on the discussions most probably this is going to be on another
> > > GitHub repo. Therefore I will prepare a standalone project with the
> > Pub/Sub
> > > module and once the repository is created a pull request shall be
> issued.
> > > If there is anything else I can do in order to facilitate this please
> let
> > > me know.
> > >
> > > If a ticket is created for this issue, I shall be able to create a pull
> > > request and thus a build will take place on ignite's CI.
> > > If everyone is aligned with adding a Pub/Sub implementation may I
> create
> > a
> > > ticket on JIRA?
> > >
> > > My next question has to do on how Ignite configures the build jobs in
> > Team
> > > City. The unit tests for my implementation need a local Pub/Sub server
> up
> > > and running (they provide one for testing purposes). Will this take
> > effect
> > > in a form of a build script inside my implementation or is it something
> > > that should be configured on Team City?
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Tue, 15 Oct 2019 at 23:27, Denis Magda  wrote:
> > >
> > > > Emmanouil, Saikat,
> > > >
> > > > After contemplating on this, let me propose another way with this
> > > > particular integration that is aligned with our modularization
> > endeavor.
> > > > The modularization [1] defines the Ignite core, Ignite modules, and
> > > > independent integrations. The first two are to be located in Ignite
> > > > repositories and 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Ilya Kasnacheev]

Hello!

Historically I support some integrations, so I can final review/merge
integrations commit until someone else steps in.

Regards,
-- 
Ilya Kasnacheev


чт, 17 окт. 2019 г. в 12:54, Alexey Zinoviev :

> Maybe we could move all our Streaming Integrations there, but what is about
> maintaining and committer permissions to the new repositories?
> I see the list of the committers and PMC members there
> https://bahir.apache.org/community-members/
>
> Could somebody from Ignite community be added to this list as a
> PMC/committer to maintain Ignite integrations?
> If yes, I'd happy to join this project and collaborate with these guys
> together
> If no, and we should start from the zero level with external PRs - hmmm,
> it's better to have our own external repository with ApacheBahirr ideology.
>
> Also, I agree, that all connectors could be moved there (in ApacheBahirr
> like repository), but not all modules from the page
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> because
> they use different parts of core modules and could be developed with
> different velocity.
>
> In reality, before creation of new repositories we need wide discussion
> based on the document mentioned above.
>
> P.S. Of course, we could experiment in the next release 2.8 with one or two
> integrations like twitter or ZeroMQ
> Also, I'd like Denis Magda idea for the separate repositories in Apache
> Github to maintain them by anybody.
>
>
> чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :
>
> > Hi Denis, Emmanouil
> >
> > Thank you for your email. I think creating a separate integration project
> > in github and also proposing it as an apache incubator project will be a
> > good move. The other separate integration modules
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > can
> > be moved to this new apache incubator project.
> >
> > There are similar integration available for Flink and Spark in Apache
> Bahir
> > https://bahir.apache.org/
> >
> > Do you think we should reach out to Apache Bahir community with a
> proposal
> > or we should plan to create a separate Apache Incubator project?
> >
> > Regards,
> > Saikat
> >
> >
> >
> > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > Based on the discussions most probably this is going to be on another
> > > GitHub repo. Therefore I will prepare a standalone project with the
> > Pub/Sub
> > > module and once the repository is created a pull request shall be
> issued.
> > > If there is anything else I can do in order to facilitate this please
> let
> > > me know.
> > >
> > > If a ticket is created for this issue, I shall be able to create a pull
> > > request and thus a build will take place on ignite's CI.
> > > If everyone is aligned with adding a Pub/Sub implementation may I
> create
> > a
> > > ticket on JIRA?
> > >
> > > My next question has to do on how Ignite configures the build jobs in
> > Team
> > > City. The unit tests for my implementation need a local Pub/Sub server
> up
> > > and running (they provide one for testing purposes). Will this take
> > effect
> > > in a form of a build script inside my implementation or is it something
> > > that should be configured on Team City?
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Tue, 15 Oct 2019 at 23:27, Denis Magda  wrote:
> > >
> > > > Emmanouil, Saikat,
> > > >
> > > > After contemplating on this, let me propose another way with this
> > > > particular integration that is aligned with our modularization
> > endeavor.
> > > > The modularization [1] defines the Ignite core, Ignite modules, and
> > > > independent integrations. The first two are to be located in Ignite
> > > > repositories and to be tested/updated by the community for every
> > release.
> > > > The modules are a subset of existing integrations that are adopted
> > widely
> > > > and require community attention.
> > > >
> > > > As for the independent integrations [1], those are supposed to be
> > located
> > > > in a separate GitHub repo. I would advise us to start implementing
> this
> > > > concept with the Pub/Sub integration. Even though it will be located
> > in a
> > > > separate Github repo, it will be listed among officially available
> > > > integrations on the Ignite website featuring Emmanouil as a
> > contributor.
> > > >
> > > > What do you think?
> > > >
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >
> > > >
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Alexey Zinoviev]

Maybe we could move all our Streaming Integrations there, but what is about
maintaining and committer permissions to the new repositories?
I see the list of the committers and PMC members there
https://bahir.apache.org/community-members/

Could somebody from Ignite community be added to this list as a
PMC/committer to maintain Ignite integrations?
If yes, I'd happy to join this project and collaborate with these guys
together
If no, and we should start from the zero level with external PRs - hmmm,
it's better to have our own external repository with ApacheBahirr ideology.

Also, I agree, that all connectors could be moved there (in ApacheBahirr
like repository), but not all modules from the page
https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
because
they use different parts of core modules and could be developed with
different velocity.

In reality, before creation of new repositories we need wide discussion
based on the document mentioned above.

P.S. Of course, we could experiment in the next release 2.8 with one or two
integrations like twitter or ZeroMQ
Also, I'd like Denis Magda idea for the separate repositories in Apache
Github to maintain them by anybody.


чт, 17 окт. 2019 г. в 05:02, Saikat Maitra :

> Hi Denis, Emmanouil
>
> Thank you for your email. I think creating a separate integration project
> in github and also proposing it as an apache incubator project will be a
> good move. The other separate integration modules
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> can
> be moved to this new apache incubator project.
>
> There are similar integration available for Flink and Spark in Apache Bahir
> https://bahir.apache.org/
>
> Do you think we should reach out to Apache Bahir community with a proposal
> or we should plan to create a separate Apache Incubator project?
>
> Regards,
> Saikat
>
>
>
> On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi all!
> >
> > Based on the discussions most probably this is going to be on another
> > GitHub repo. Therefore I will prepare a standalone project with the
> Pub/Sub
> > module and once the repository is created a pull request shall be issued.
> > If there is anything else I can do in order to facilitate this please let
> > me know.
> >
> > If a ticket is created for this issue, I shall be able to create a pull
> > request and thus a build will take place on ignite's CI.
> > If everyone is aligned with adding a Pub/Sub implementation may I create
> a
> > ticket on JIRA?
> >
> > My next question has to do on how Ignite configures the build jobs in
> Team
> > City. The unit tests for my implementation need a local Pub/Sub server up
> > and running (they provide one for testing purposes). Will this take
> effect
> > in a form of a build script inside my implementation or is it something
> > that should be configured on Team City?
> >
> > Kind regards,
> > Emmanouil
> >
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Tue, 15 Oct 2019 at 23:27, Denis Magda  wrote:
> >
> > > Emmanouil, Saikat,
> > >
> > > After contemplating on this, let me propose another way with this
> > > particular integration that is aligned with our modularization
> endeavor.
> > > The modularization [1] defines the Ignite core, Ignite modules, and
> > > independent integrations. The first two are to be located in Ignite
> > > repositories and to be tested/updated by the community for every
> release.
> > > The modules are a subset of existing integrations that are adopted
> widely
> > > and require community attention.
> > >
> > > As for the independent integrations [1], those are supposed to be
> located
> > > in a separate GitHub repo. I would advise us to start implementing this
> > > concept with the Pub/Sub integration. Even though it will be located
> in a
> > > separate Github repo, it will be listed among officially available
> > > integrations on the Ignite website featuring Emmanouil as a
> contributor.
> > >
> > > What do you think?
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >
> > >
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra  >
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > The changes looks good to me. I wanted to check if you are also able
> to
> > > > validate that the once data is added to Ignite cluster you are also
> > able
> > > to
> > > > access it using another Ignite client or using rest endpoints.
> > > >
> > > > We use Teamcity for CI process, the details are as follows
> > > >
> > > > Trigger validation of those test suites that have been affected by
> your
> > > > 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Denis, Emmanouil

Thank you for your email. I think creating a separate integration project
in github and also proposing it as an apache incubator project will be a
good move. The other separate integration modules
https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
can
be moved to this new apache incubator project.

There are similar integration available for Flink and Spark in Apache Bahir
https://bahir.apache.org/

Do you think we should reach out to Apache Bahir community with a proposal
or we should plan to create a separate Apache Incubator project?

Regards,
Saikat



On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras 
wrote:

> Hi all!
>
> Based on the discussions most probably this is going to be on another
> GitHub repo. Therefore I will prepare a standalone project with the Pub/Sub
> module and once the repository is created a pull request shall be issued.
> If there is anything else I can do in order to facilitate this please let
> me know.
>
> If a ticket is created for this issue, I shall be able to create a pull
> request and thus a build will take place on ignite's CI.
> If everyone is aligned with adding a Pub/Sub implementation may I create a
> ticket on JIRA?
>
> My next question has to do on how Ignite configures the build jobs in Team
> City. The unit tests for my implementation need a local Pub/Sub server up
> and running (they provide one for testing purposes). Will this take effect
> in a form of a build script inside my implementation or is it something
> that should be configured on Team City?
>
> Kind regards,
> Emmanouil
>
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Tue, 15 Oct 2019 at 23:27, Denis Magda  wrote:
>
> > Emmanouil, Saikat,
> >
> > After contemplating on this, let me propose another way with this
> > particular integration that is aligned with our modularization endeavor.
> > The modularization [1] defines the Ignite core, Ignite modules, and
> > independent integrations. The first two are to be located in Ignite
> > repositories and to be tested/updated by the community for every release.
> > The modules are a subset of existing integrations that are adopted widely
> > and require community attention.
> >
> > As for the independent integrations [1], those are supposed to be located
> > in a separate GitHub repo. I would advise us to start implementing this
> > concept with the Pub/Sub integration. Even though it will be located in a
> > separate Github repo, it will be listed among officially available
> > integrations on the Ignite website featuring Emmanouil as a contributor.
> >
> > What do you think?
> >
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >
> >
> >
> > -
> > Denis
> >
> >
> > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra 
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > The changes looks good to me. I wanted to check if you are also able to
> > > validate that the once data is added to Ignite cluster you are also
> able
> > to
> > > access it using another Ignite client or using rest endpoints.
> > >
> > > We use Teamcity for CI process, the details are as follows
> > >
> > > Trigger validation of those test suites that have been affected by your
> > > changes on TeamCity : <
> > http://204.14.53.153/
> > > >
> > >
> > >- Locate a test suite you have to check, press button named "..."
> that
> > >is located on the left of "Run" button. "Run custom build" dialog
> will
> > >appear;
> > >- Go to "Changes" tab and choose "pull//head"
> in
> > >"Build branch" dropdown list;
> > >- Press "Run build" button and monitor tests results.
> > >
> > >
> > >
> > > @Roman rsht...@yahoo.com - can you also please take a look?
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi @Saikat Maitra
> > > >
> > > > I have my implementation on an Ignite fork on my personal GitHub
> > account
> > > > [1]
> > > > It is based on the Kafka implementation which is of a polling nature.
> > > > GCP does not provide a service like Kafka. Instead it provides us
> with
> > > > Pub/Sub which is just like Kafka.
> > > > Therefore using the Kafka streamer as a guideline made more sense to
> > me.
> > > >
> > > > Apart from the implementation I have also added a unit test.
> > > > It definitely needs some polishing and a double check on my side to
> > make
> > > > the review smooth.
> > > > For the tests a local Pub/Sub [2] server has to be setup however I am
> > not
> > > > sure which is the best way to do so with regards to your CI/CD.
> > > >
> > > > [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > [2] 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all!

Based on the discussions most probably this is going to be on another
GitHub repo. Therefore I will prepare a standalone project with the Pub/Sub
module and once the repository is created a pull request shall be issued.
If there is anything else I can do in order to facilitate this please let
me know.

If a ticket is created for this issue, I shall be able to create a pull
request and thus a build will take place on ignite's CI.
If everyone is aligned with adding a Pub/Sub implementation may I create a
ticket on JIRA?

My next question has to do on how Ignite configures the build jobs in Team
City. The unit tests for my implementation need a local Pub/Sub server up
and running (they provide one for testing purposes). Will this take effect
in a form of a build script inside my implementation or is it something
that should be configured on Team City?

Kind regards,
Emmanouil


*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Tue, 15 Oct 2019 at 23:27, Denis Magda  wrote:

> Emmanouil, Saikat,
>
> After contemplating on this, let me propose another way with this
> particular integration that is aligned with our modularization endeavor.
> The modularization [1] defines the Ignite core, Ignite modules, and
> independent integrations. The first two are to be located in Ignite
> repositories and to be tested/updated by the community for every release.
> The modules are a subset of existing integrations that are adopted widely
> and require community attention.
>
> As for the independent integrations [1], those are supposed to be located
> in a separate GitHub repo. I would advise us to start implementing this
> concept with the Pub/Sub integration. Even though it will be located in a
> separate Github repo, it will be listed among officially available
> integrations on the Ignite website featuring Emmanouil as a contributor.
>
> What do you think?
>
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>
>
>
> -
> Denis
>
>
> On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra 
> wrote:
>
> > Hi Emmanouil,
> >
> > The changes looks good to me. I wanted to check if you are also able to
> > validate that the once data is added to Ignite cluster you are also able
> to
> > access it using another Ignite client or using rest endpoints.
> >
> > We use Teamcity for CI process, the details are as follows
> >
> > Trigger validation of those test suites that have been affected by your
> > changes on TeamCity : <
> http://204.14.53.153/
> > >
> >
> >- Locate a test suite you have to check, press button named "..." that
> >is located on the left of "Run" button. "Run custom build" dialog will
> >appear;
> >- Go to "Changes" tab and choose "pull//head" in
> >"Build branch" dropdown list;
> >- Press "Run build" button and monitor tests results.
> >
> >
> >
> > @Roman rsht...@yahoo.com - can you also please take a look?
> >
> > Regards,
> > Saikat
> >
> > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi @Saikat Maitra
> > >
> > > I have my implementation on an Ignite fork on my personal GitHub
> account
> > > [1]
> > > It is based on the Kafka implementation which is of a polling nature.
> > > GCP does not provide a service like Kafka. Instead it provides us with
> > > Pub/Sub which is just like Kafka.
> > > Therefore using the Kafka streamer as a guideline made more sense to
> me.
> > >
> > > Apart from the implementation I have also added a unit test.
> > > It definitely needs some polishing and a double check on my side to
> make
> > > the review smooth.
> > > For the tests a local Pub/Sub [2] server has to be setup however I am
> not
> > > sure which is the best way to do so with regards to your CI/CD.
> > >
> > > [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > [2] https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra 
> > > wrote:
> > >
> > > > Hello Emmanouil,
> > > >
> > > > Can you please share more info about the implementation?
> > > >
> > > > We have similar implementation for data streamers with Kafka and
> > RocketMQ
> > > >
> > > > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >
> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >
> > > > Please review and share if you have any questions.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > By mentioning Pub/Sub I mean Google Cloud's 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Emmanouil, Saikat,

After contemplating on this, let me propose another way with this
particular integration that is aligned with our modularization endeavor.
The modularization [1] defines the Ignite core, Ignite modules, and
independent integrations. The first two are to be located in Ignite
repositories and to be tested/updated by the community for every release.
The modules are a subset of existing integrations that are adopted widely
and require community attention.

As for the independent integrations [1], those are supposed to be located
in a separate GitHub repo. I would advise us to start implementing this
concept with the Pub/Sub integration. Even though it will be located in a
separate Github repo, it will be listed among officially available
integrations on the Ignite website featuring Emmanouil as a contributor.

What do you think?


[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations



-
Denis


On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra 
wrote:

> Hi Emmanouil,
>
> The changes looks good to me. I wanted to check if you are also able to
> validate that the once data is added to Ignite cluster you are also able to
> access it using another Ignite client or using rest endpoints.
>
> We use Teamcity for CI process, the details are as follows
>
> Trigger validation of those test suites that have been affected by your
> changes on TeamCity :  >
>
>- Locate a test suite you have to check, press button named "..." that
>is located on the left of "Run" button. "Run custom build" dialog will
>appear;
>- Go to "Changes" tab and choose "pull//head" in
>"Build branch" dropdown list;
>- Press "Run build" button and monitor tests results.
>
>
>
> @Roman rsht...@yahoo.com - can you also please take a look?
>
> Regards,
> Saikat
>
> On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi @Saikat Maitra
> >
> > I have my implementation on an Ignite fork on my personal GitHub account
> > [1]
> > It is based on the Kafka implementation which is of a polling nature.
> > GCP does not provide a service like Kafka. Instead it provides us with
> > Pub/Sub which is just like Kafka.
> > Therefore using the Kafka streamer as a guideline made more sense to me.
> >
> > Apart from the implementation I have also added a unit test.
> > It definitely needs some polishing and a double check on my side to make
> > the review smooth.
> > For the tests a local Pub/Sub [2] server has to be setup however I am not
> > sure which is the best way to do so with regards to your CI/CD.
> >
> > [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > [2] https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra 
> > wrote:
> >
> > > Hello Emmanouil,
> > >
> > > Can you please share more info about the implementation?
> > >
> > > We have similar implementation for data streamers with Kafka and
> RocketMQ
> > >
> > > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > >
> > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > >
> > > Please review and share if you have any questions.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
> > > > Yes it is indeed cloud specific. It is the messaging service provided
> > by
> > > > GCP.
> > > > I have made the integration with it, but I haven't published it yet
> (I
> > am
> > > > waiting for the discussion/ticket and proceed with the contribution).
> > > >
> > > > Kind regards,
> > > > Emmanouil
> > > >
> > > > [1] https://cloud.google.com/pubsub/docs/overview
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda  wrote:
> > > >
> > > > > Hello Emmanouil,
> > > > >
> > > > > Are you referring to any specific implementation tailored for a
> > cloud?
> > > If
> > > > > I'm not mistaken you were mentioning AWS/GCE. Is there any Github
> > > source
> > > > > code?
> > > > >
> > > > > Here is a list of existing integrations with streaming and pub/sub
> > > > > frameworks:
> > > > > https://apacheignite-mix.readme.io/docs/overview
> > > > >
> > > > > with the source code in Ignite master:
> > > > > https://github.com/apache/ignite/tree/master/modules
> > > > >
> > > > > Check out those resources to see what are the most suitable Ignite
> > APIs
> > > > for
> > > > > such integrations.
> > > > >
> 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Emmanouil,

The changes looks good to me. I wanted to check if you are also able to
validate that the once data is added to Ignite cluster you are also able to
access it using another Ignite client or using rest endpoints.

We use Teamcity for CI process, the details are as follows

Trigger validation of those test suites that have been affected by your
changes on TeamCity : 

   - Locate a test suite you have to check, press button named "..." that
   is located on the left of "Run" button. "Run custom build" dialog will
   appear;
   - Go to "Changes" tab and choose "pull//head" in
   "Build branch" dropdown list;
   - Press "Run build" button and monitor tests results.



@Roman rsht...@yahoo.com - can you also please take a look?

Regards,
Saikat

On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras 
wrote:

> Hi @Saikat Maitra
>
> I have my implementation on an Ignite fork on my personal GitHub account
> [1]
> It is based on the Kafka implementation which is of a polling nature.
> GCP does not provide a service like Kafka. Instead it provides us with
> Pub/Sub which is just like Kafka.
> Therefore using the Kafka streamer as a guideline made more sense to me.
>
> Apart from the implementation I have also added a unit test.
> It definitely needs some polishing and a double check on my side to make
> the review smooth.
> For the tests a local Pub/Sub [2] server has to be setup however I am not
> sure which is the best way to do so with regards to your CI/CD.
>
> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> [2] https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 12 Oct 2019 at 02:11, Saikat Maitra 
> wrote:
>
> > Hello Emmanouil,
> >
> > Can you please share more info about the implementation?
> >
> > We have similar implementation for data streamers with Kafka and RocketMQ
> >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >
> > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >
> > Please review and share if you have any questions.
> >
> > Regards,
> > Saikat
> >
> >
> >
> > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
> > > Yes it is indeed cloud specific. It is the messaging service provided
> by
> > > GCP.
> > > I have made the integration with it, but I haven't published it yet (I
> am
> > > waiting for the discussion/ticket and proceed with the contribution).
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > > [1] https://cloud.google.com/pubsub/docs/overview
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Fri, 11 Oct 2019 at 21:45, Denis Magda  wrote:
> > >
> > > > Hello Emmanouil,
> > > >
> > > > Are you referring to any specific implementation tailored for a
> cloud?
> > If
> > > > I'm not mistaken you were mentioning AWS/GCE. Is there any Github
> > source
> > > > code?
> > > >
> > > > Here is a list of existing integrations with streaming and pub/sub
> > > > frameworks:
> > > > https://apacheignite-mix.readme.io/docs/overview
> > > >
> > > > with the source code in Ignite master:
> > > > https://github.com/apache/ignite/tree/master/modules
> > > >
> > > > Check out those resources to see what are the most suitable Ignite
> APIs
> > > for
> > > > such integrations.
> > > >
> > > > @Saikat Maitra  , @Roman  >
> > ,
> > > as
> > > > streaming technologies contributors/maintainers, could you please
> join
> > > this
> > > > thread?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras <
> > > > gkatzio...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello everyone,
> > > > >
> > > > > I started using Ignite lately. Part of my work involves Pub/Sub,
> > thus I
> > > > > created a pub/sub streamer.
> > > > > Pub/Sub is pretty close to Kafka.
> > > > > Here's how the streamer works. Every node creates an instance of a
> > > > Pub/Sub
> > > > > client.
> > > > > The clients subscribe to the same topic using the same subscription
> > > name.
> > > > > A message is being sent and one of the nodes will receive the
> > message,
> > > > > process it and then forward it to Ignite using the extractor
> > provided.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi @Saikat Maitra

I have my implementation on an Ignite fork on my personal GitHub account [1]
It is based on the Kafka implementation which is of a polling nature.
GCP does not provide a service like Kafka. Instead it provides us with
Pub/Sub which is just like Kafka.
Therefore using the Kafka streamer as a guideline made more sense to me.

Apart from the implementation I have also added a unit test.
It definitely needs some polishing and a double check on my side to make
the review smooth.
For the tests a local Pub/Sub [2] server has to be setup however I am not
sure which is the best way to do so with regards to your CI/CD.

[1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
[2] https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 12 Oct 2019 at 02:11, Saikat Maitra  wrote:

> Hello Emmanouil,
>
> Can you please share more info about the implementation?
>
> We have similar implementation for data streamers with Kafka and RocketMQ
>
> https://apacheignite-mix.readme.io/docs/kafka-streamer
>
> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>
> Please review and share if you have any questions.
>
> Regards,
> Saikat
>
>
>
> On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
> > Yes it is indeed cloud specific. It is the messaging service provided by
> > GCP.
> > I have made the integration with it, but I haven't published it yet (I am
> > waiting for the discussion/ticket and proceed with the contribution).
> >
> > Kind regards,
> > Emmanouil
> >
> > [1] https://cloud.google.com/pubsub/docs/overview
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Fri, 11 Oct 2019 at 21:45, Denis Magda  wrote:
> >
> > > Hello Emmanouil,
> > >
> > > Are you referring to any specific implementation tailored for a cloud?
> If
> > > I'm not mistaken you were mentioning AWS/GCE. Is there any Github
> source
> > > code?
> > >
> > > Here is a list of existing integrations with streaming and pub/sub
> > > frameworks:
> > > https://apacheignite-mix.readme.io/docs/overview
> > >
> > > with the source code in Ignite master:
> > > https://github.com/apache/ignite/tree/master/modules
> > >
> > > Check out those resources to see what are the most suitable Ignite APIs
> > for
> > > such integrations.
> > >
> > > @Saikat Maitra  , @Roman 
> ,
> > as
> > > streaming technologies contributors/maintainers, could you please join
> > this
> > > thread?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras <
> > > gkatzio...@gmail.com>
> > > wrote:
> > >
> > > > Hello everyone,
> > > >
> > > > I started using Ignite lately. Part of my work involves Pub/Sub,
> thus I
> > > > created a pub/sub streamer.
> > > > Pub/Sub is pretty close to Kafka.
> > > > Here's how the streamer works. Every node creates an instance of a
> > > Pub/Sub
> > > > client.
> > > > The clients subscribe to the same topic using the same subscription
> > name.
> > > > A message is being sent and one of the nodes will receive the
> message,
> > > > process it and then forward it to Ignite using the extractor
> provided.
> > > >
> > > > Any thoughts?
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hello Emmanouil,

Can you please share more info about the implementation?

We have similar implementation for data streamers with Kafka and RocketMQ

https://apacheignite-mix.readme.io/docs/kafka-streamer

https://apacheignite-mix.readme.io/docs/rocketmq-streamer

Please review and share if you have any questions.

Regards,
Saikat



On Fri, Oct 11, 2019 at 4:56 PM Emmanouil Gkatziouras 
wrote:

> Hi all,
>
> By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
> Yes it is indeed cloud specific. It is the messaging service provided by
> GCP.
> I have made the integration with it, but I haven't published it yet (I am
> waiting for the discussion/ticket and proceed with the contribution).
>
> Kind regards,
> Emmanouil
>
> [1] https://cloud.google.com/pubsub/docs/overview
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 11 Oct 2019 at 21:45, Denis Magda  wrote:
>
> > Hello Emmanouil,
> >
> > Are you referring to any specific implementation tailored for a cloud? If
> > I'm not mistaken you were mentioning AWS/GCE. Is there any Github source
> > code?
> >
> > Here is a list of existing integrations with streaming and pub/sub
> > frameworks:
> > https://apacheignite-mix.readme.io/docs/overview
> >
> > with the source code in Ignite master:
> > https://github.com/apache/ignite/tree/master/modules
> >
> > Check out those resources to see what are the most suitable Ignite APIs
> for
> > such integrations.
> >
> > @Saikat Maitra  , @Roman  ,
> as
> > streaming technologies contributors/maintainers, could you please join
> this
> > thread?
> >
> > -
> > Denis
> >
> >
> > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras <
> > gkatzio...@gmail.com>
> > wrote:
> >
> > > Hello everyone,
> > >
> > > I started using Ignite lately. Part of my work involves Pub/Sub, thus I
> > > created a pub/sub streamer.
> > > Pub/Sub is pretty close to Kafka.
> > > Here's how the streamer works. Every node creates an instance of a
> > Pub/Sub
> > > client.
> > > The clients subscribe to the same topic using the same subscription
> name.
> > > A message is being sent and one of the nodes will receive the message,
> > > process it and then forward it to Ignite using the extractor provided.
> > >
> > > Any thoughts?
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras]

Hi all,

By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
Yes it is indeed cloud specific. It is the messaging service provided by
GCP.
I have made the integration with it, but I haven't published it yet (I am
waiting for the discussion/ticket and proceed with the contribution).

Kind regards,
Emmanouil

[1] https://cloud.google.com/pubsub/docs/overview

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 11 Oct 2019 at 21:45, Denis Magda  wrote:

> Hello Emmanouil,
>
> Are you referring to any specific implementation tailored for a cloud? If
> I'm not mistaken you were mentioning AWS/GCE. Is there any Github source
> code?
>
> Here is a list of existing integrations with streaming and pub/sub
> frameworks:
> https://apacheignite-mix.readme.io/docs/overview
>
> with the source code in Ignite master:
> https://github.com/apache/ignite/tree/master/modules
>
> Check out those resources to see what are the most suitable Ignite APIs for
> such integrations.
>
> @Saikat Maitra  , @Roman  , as
> streaming technologies contributors/maintainers, could you please join this
> thread?
>
> -
> Denis
>
>
> On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras <
> gkatzio...@gmail.com>
> wrote:
>
> > Hello everyone,
> >
> > I started using Ignite lately. Part of my work involves Pub/Sub, thus I
> > created a pub/sub streamer.
> > Pub/Sub is pretty close to Kafka.
> > Here's how the streamer works. Every node creates an instance of a
> Pub/Sub
> > client.
> > The clients subscribe to the same topic using the same subscription name.
> > A message is being sent and one of the nodes will receive the message,
> > process it and then forward it to Ignite using the extractor provided.
> >
> > Any thoughts?
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda]

Hello Emmanouil,

Are you referring to any specific implementation tailored for a cloud? If
I'm not mistaken you were mentioning AWS/GCE. Is there any Github source
code?

Here is a list of existing integrations with streaming and pub/sub
frameworks:
https://apacheignite-mix.readme.io/docs/overview

with the source code in Ignite master:
https://github.com/apache/ignite/tree/master/modules

Check out those resources to see what are the most suitable Ignite APIs for
such integrations.

@Saikat Maitra  , @Roman  , as
streaming technologies contributors/maintainers, could you please join this
thread?

-
Denis


On Thu, Oct 10, 2019 at 7:58 AM Emmanouil Gkatziouras 
wrote:

> Hello everyone,
>
> I started using Ignite lately. Part of my work involves Pub/Sub, thus I
> created a pub/sub streamer.
> Pub/Sub is pretty close to Kafka.
> Here's how the streamer works. Every node creates an instance of a Pub/Sub
> client.
> The clients subscribe to the same topic using the same subscription name.
> A message is being sent and one of the nodes will receive the message,
> process it and then forward it to Ignite using the extractor provided.
>
> Any thoughts?
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>