Hi Ivan,
Yes, we will need to remove flink module from main ignite repo.
I can do it as part of this issue.
IGNITE-12356 Migrate Flink module to ignite-extensions
Regards,
Saikat
On Mon, Dec 30, 2019 at 12:54 AM Ivan Pavlukhin wrote:
> Folks,
>
> Do not we need to remove flink integration
Folks,
Do not we need to remove flink integration sources from main ignite
repository [1]? Is this work in progress?
[1] https://github.com/apache/ignite
вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras :
>
> Hi Saikat!
>
> Thank you for your time and assistance.
>
> Kind regards,
> Emmanouil
Hi Saikat!
Thank you for your time and assistance.
Kind regards,
Emmanouil
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura
On Sun, 29 Dec 2019 at 18:35, Saikat Maitra wrote:
> Hi Emmanouil,
>
> Thank you for
Hi Emmanouil,
Thank you for your contribution. I have merged your changes in master.
Regards,
Saikat
On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra
wrote:
> Thank you, Emmanouil
>
> I am reviewing the changes.
>
> Regards,
> Saikat
>
> On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
Thank you, Emmanouil
I am reviewing the changes.
Regards,
Saikat
On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras
wrote:
> Hi Saikat!
>
> The dependencies have been updated, and the merge conflicts resolved.
> Kind regards
>
> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, wrote:
>
> > Hi
Hi Saikat!
The dependencies have been updated, and the merge conflicts resolved.
Kind regards
On Sat, 28 Dec 2019, 23:27 Saikat Maitra, wrote:
> Hi Emmanouil,
>
> Thank you for making the changes. I have merged my PR changes in master.
> Can you please take the latest from master branch and
Hi Emmanouil,
Thank you for making the changes. I have merged my PR changes in master.
Can you please take the latest from master branch and resolve the conflicts
in your PR.
I have also reviewed the changes in your PR and requested minor changes
specific to dependencies version and license
Hi Saikat!
Thank you on this one! I tried it and it worked as expected [1]
Kind regards
Emmanouil
[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4888357=buildLog&_focus=2046#_state=2046
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
Hi Emmanouil,
I have been able to resolve the build failures for missing modules.
https://ci.ignite.apache.org/viewLog.html?buildId=4883394=IgniteExtensions_Build
Will you please take the latest changes from my PR and run the build for
your PR.
Hi Saikat!
Thank you for your assistance on that!
Kind regards
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura
On Sat, 30 Nov 2019 at 23:51, Saikat Maitra wrote:
> Hi Emmanouil,
>
> I looked into the build
Hi Emmanouil,
I looked into the build logs and I observed that since I added artifacts
dependencies to "Build apache Ignite"[1] it was able to pull the required
dependencies and was able to run the existing tests. It is however not
identifying new modules like I changed from flink to flink-ext or
Hi Saikat!
>From the logs[1] it seems that TC cannot find the project. However I did
include the project on the parent pom (modules sections) [2]
I tried to reproduce locally unfortunately I am not aware of the full
context on TC. It seems as if a different parent pom is being picked up.
Kind
Hello,
I have made changes in Ignite Extensions build configurations.
It is now running the tests as expected.
https://ci.ignite.apache.org/viewLog.html?buildId=4804480=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
Regards,
Saikat
On Thu, Nov 28, 2019 at 1:29 PM
Hi,
We do run build with -DskipTests and then configure selective test suites
to run.
I will check why these test suites are not getting executed.
Regards
Saikat
On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras
wrote:
> Hi all,
>
> I did add a test suite and run the build with the
Hi all,
I did add a test suite and run the build with the corresponding arguments,
however the tests did not run [1].
I checked the Flink's build logs and the same message is displayed there
too `No tests to run` [2].
[1]
Hi,
We will need to add test suites for tests to be executed in build. We pass
them as TestSuites param. I had added Flink sink and source testsuites.
Can you please review and confirm.
Regards
Saikat
On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras
wrote:
> Hi all!
>
> I was successful
Hi all!
I was successful building and running my pull requests. It seems that the
tests do not run both for Flink and Pub/Sub [1]
If there is something I cannot do to make them discoverable please let me
know.
[1]
Hi Emmanouil,
I have added you as contributor in Ignite Extensions project. Can you
please check and confirm if you are able to see the project and execute
build on your pull request.
Regards,
Saikat
On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras
wrote:
> Hi all!
>
> If someone could
Hi all!
If someone could give me read access to the Job of Ignite Extensions on
Team City will greatly help me (username gkatzioura).
I suppose the builds get triggered automatically on a pull request.
Kind regards,
Emmanouil
*Emmanouil Gkatziouras*
https://egkatzioura.com/ |
Hi Saikat!
I just rebased with the flink branch. Unfortunately I do not have read
access to the team city link you provided and thus evaluate the tests for
my pull request.
I guess it has to do with ignite extensions being new.
My username is gkatzioura and email the one I am using on this email
Hi Emmanouil,
The latest build on teamcity has passed on Flink pull request.
https://ci.ignite.apache.org/viewLog.html?buildId=4788928=IgniteExtensions_Build=buildResultsDiv_IgniteExtensions=pull%2F1%2Fhead
you should be able to take the changes I made in my PR and run build on
your pull
Hi Emmanouil,
I have fixed the Flink module testsuites and have setup new project in
teamcity.
https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions=projectOverview
I am looking into setting up the build.
Regards,
Saikat
On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra
wrote:
>
Hi Emmanouil,
Thank you for your email. Yes, the plan is once the Flink PR is merged it
will provide the Licence, parent POMs etc and you can rebase from master
branch and apply the changes on top of it.
I am getting some issues with test failures in local with
GridTestProperties as the
Hi all,
I made my first pull request [1]. Since this project is brand new (no
parent poms, licensing), I reckoned it was best to use Saikat's branch on
Flink.
I suppose the Flink branch will be merged first. If not please give me
guidelines on how I should proceed next.
Kind regards
Emmanouil
Awesome, ping us whenever you're ready!
-
Denis
On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras
wrote:
> Hi all!
>
> I am sorry for being late on that. I was trying to refactor the test in
> order not to be in need of any external tools or spinning up a server.
> I did forked the new
Hi all!
I am sorry for being late on that. I was trying to refactor the test in
order not to be in need of any external tools or spinning up a server.
I did forked the new repo and indeed my changes there, so a pull request is
a matter of time!
Kind regards
*Emmanouil Gkatziouras*
Hi Emmanouil,
Do you have any questions or need any support from the community?
-
Denis
On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra
wrote:
> Hi Emmanouil,
>
> Can you please take a looks at dev utils, if this is something you are
> looking for
Hi Emmanouil,
Can you please take a looks at dev utils, if this is something you are
looking for https://github.com/apache/ignite/tree/master/modules/dev-utils?
IMO, if you can release Pub/Sub server in maven and then use it as
dependency, that would be great.
Regards,
Saikat
On Sun, Nov 10,
Hi Emmanouil,
The master branch has been initialized. I have raised a PR to migrate Flink
module to ignite-extensions.
PR https://github.com/apache/ignite-extensions/pull/1
Jira : https://issues.apache.org/jira/browse/IGNITE-12356
Regards,
Saikat
On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
Hi all!
This is an update. The integration is tested with mocks based on the
original Ignite repo [1].
I am currently waiting for the first commits on the new integration project
to take place in order to create my pull request accordingly to the initial
commits.
In order to simulate the Pub/Sub
Hello Ilya,
Thank you for your email. Yes, before we remove any integration we will
move them to another repository and release.
We are thinking we will start with Pub/Sub Streamer implementation and
continue to move other integration as mentioned in the following doc.
Hello!
My take on this: we want to offload some of integrations, but please commit
and release them to another project first, and only then we will decide to
remove them from our repository.
Please also make sure they are available with same artifact name and API.
If this has to be changed,
Hi Denis,
Thank you for your response. Yes, I have created the following discussion
thread for Ignite extensions migration.
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html
Regards,
Saikat
Saikat, thanks for checking with Bahir!
However, let’s run this idea through the rest of the community before
getting down to the implementation. As an alternate option, we might adopt
their approach for Ignite extensions but keep those within Ignite community
in separate repositories. Would you
Hello Emmanouil,
As discussed earlier, I discussed with Apache Bahir community and they are
interested to have Apache Ignite extensions as part of Apache Bahir project.
Can you please share Pub/Sub streamer implementation details with Apache
Bahir community and request for feedback.
Hello,
As discussed I have created following discussion threads on our migration
proposals for Apache Ignite extensions:
http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
Hello Denis,
Sure, sounds good. I will create a separate discussion thread to get
community feedback on whether we
should join the Bahir or kick-off "Ignite Extensions" project.
Regards,
Saikat
On Thu, Oct 17, 2019 at 2:21 PM Denis Magda wrote:
> Folks,
>
> The concept of Apache Bahir is
Folks,
The concept of Apache Bahir is precisely what we need! Saikat, thanks for
doing the research. Why I believe the * idea* fits us well:
- All integrations can be stored in separate Github repositories and
have their dev lifecycles. We've not obliged to couple all the integrations
Hello!
Historically I support some integrations, so I can final review/merge
integrations commit until someone else steps in.
Regards,
--
Ilya Kasnacheev
чт, 17 окт. 2019 г. в 12:54, Alexey Zinoviev :
> Maybe we could move all our Streaming Integrations there, but what is about
> maintaining
Maybe we could move all our Streaming Integrations there, but what is about
maintaining and committer permissions to the new repositories?
I see the list of the committers and PMC members there
https://bahir.apache.org/community-members/
Could somebody from Ignite community be added to this list
Hi Denis, Emmanouil
Thank you for your email. I think creating a separate integration project
in github and also proposing it as an apache incubator project will be a
good move. The other separate integration modules
Hi all!
Based on the discussions most probably this is going to be on another
GitHub repo. Therefore I will prepare a standalone project with the Pub/Sub
module and once the repository is created a pull request shall be issued.
If there is anything else I can do in order to facilitate this please
Emmanouil, Saikat,
After contemplating on this, let me propose another way with this
particular integration that is aligned with our modularization endeavor.
The modularization [1] defines the Ignite core, Ignite modules, and
independent integrations. The first two are to be located in Ignite
Hi Emmanouil,
The changes looks good to me. I wanted to check if you are also able to
validate that the once data is added to Ignite cluster you are also able to
access it using another Ignite client or using rest endpoints.
We use Teamcity for CI process, the details are as follows
Trigger
Hi @Saikat Maitra
I have my implementation on an Ignite fork on my personal GitHub account [1]
It is based on the Kafka implementation which is of a polling nature.
GCP does not provide a service like Kafka. Instead it provides us with
Pub/Sub which is just like Kafka.
Therefore using the Kafka
Hello Emmanouil,
Can you please share more info about the implementation?
We have similar implementation for data streamers with Kafka and RocketMQ
https://apacheignite-mix.readme.io/docs/kafka-streamer
https://apacheignite-mix.readme.io/docs/rocketmq-streamer
Please review and share if you
Hi all,
By mentioning Pub/Sub I mean Google Cloud's Pub/Sub [1]
Yes it is indeed cloud specific. It is the messaging service provided by
GCP.
I have made the integration with it, but I haven't published it yet (I am
waiting for the discussion/ticket and proceed with the contribution).
Kind
Hello Emmanouil,
Are you referring to any specific implementation tailored for a cloud? If
I'm not mistaken you were mentioning AWS/GCE. Is there any Github source
code?
Here is a list of existing integrations with streaming and pub/sub
frameworks:
Hello everyone,
I started using Ignite lately. Part of my work involves Pub/Sub, thus I
created a pub/sub streamer.
Pub/Sub is pretty close to Kafka.
Here's how the streamer works. Every node creates an instance of a Pub/Sub
client.
The clients subscribe to the same topic using the same
49 matches
Mail list logo
