Hello, Please find the PR link below.
https://github.com/apache/ignite-extensions/pull/1/files Regards, Saikat On Thu, Dec 26, 2019 at 1:43 PM Saikat Maitra <saikat.mai...@gmail.com> wrote: > Hello > > I have been able to resolve the build issue in teamcity and was able to > pass the build failures. > > > https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build > > The issue that was causing failure was that I had put ignite.zip as > artifact dependency and it was downloading the artifacts including pom file > inside the build checkout folder of ignite-extensions and as a result the > pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml. > This was resulting in missing flink-ext module error as it was not even > present in the ignite main pom.xml. > > To resolve the issue I just had to download and extract artifact of > ignite.zip in a separate directory. > > If I can get a +1 on this PR then I can go ahead and merge the changes. > > Regards, > Saikat > > > > > > > On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra <saikat.mai...@gmail.com> > wrote: > >> Hi Ivan, >> >> Thank you for sharing your feedback. I will try to install all modules in >> local agent's repository and run the build. >> >> I will share updates. >> >> Regards, >> Saikat >> >> On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin <vololo...@gmail.com> >> wrote: >> >>> Saikat, >>> >>> Let me share my understanding how do we run the majority of our jobs >>> executing test suites. Hope it will help. >>> 1. ~Build Apache Ignite~ in a separate run and publish everything as a >>> single artifact ignite.zip (it contains ignite working directory with >>> compiled modules, e.g. modules/core/target and others). >>> 2. Download and extract ignite.zip and run particular test suites >>> against extracted files. >>> >>> In a such setup test suites run the same as you do it on your local >>> machine and it always works with a single multi-module maven project >>> (apache-ignite). So, all dependencies are resolved in the working >>> directory. >>> >>> For a separate IgniteExtensions maven project you need to install all >>> dependencies from downloaded ignite.zip into .m2/repository local to >>> the agent. I suppose we can do it as follows: >>> 1. Download ignite.zip and extract it into "ignite" subfolder. >>> 2. Run a maven command installing all modules into local maven >>> repository. Something like "mvn install" but here we need to make sure >>> that nothing is compiled once again. >>> 3. Build IgniteExtensions. >>> >>> There might be a better way to do the same, I just tried to describe a >>> schema how it can be done. >>> >>> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <saikat.mai...@gmail.com>: >>> > >>> > Hi Ivan, Ilya >>> > >>> > Thank you for your reply. I have added you as dev in IgniteExtensions >>> > project and you should be able to check the build configurations. >>> > >>> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ >>> similar to >>> > other projects then I receive below error >>> > >>> > [ERROR] [ERROR] Could not find the selected project in the reactor: >>> > :ignite-flink-ext @ >>> > >>> > I do not see the same error when I am running the build in local and >>> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my >>> > local .m2 repository but it is failing in teamcity. >>> > >>> > Regards, >>> > Saikat >>> > >>> > >>> > >>> > >>> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com> >>> > wrote: >>> > >>> > > Hello! >>> > > >>> > > I think that your build should depend on an Apache Ignite build(s) >>> or just >>> > > use an already released version. >>> > > >>> > > In the same fashion as all our teamcity depend on "Build Apache >>> Ignite" and >>> > > use its artifacts. >>> > > >>> > > Here you want Ignite snapshot but Teamcity does not know where to >>> take it >>> > > from. >>> > > >>> > > Regards, >>> > > -- >>> > > Ilya Kasnacheev >>> > > >>> > > >>> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <saikat.mai...@gmail.com>: >>> > > >>> > > > Hello, >>> > > > >>> > > > I am running into a problem specific to teamcity build for Ignite >>> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT >>> I am >>> > > > getting an error message during build as below >>> > > > >>> > > > [06:24:12][Step 4/5] Failed to execute goal on project >>> ignite-flink-ext: >>> > > > Could not resolve dependencies for project >>> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The >>> following >>> > > > artifacts could not be resolved: >>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT, >>> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find >>> > > artifact >>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com >>> ( >>> > > > https://h2database.com/m2-repo) >>> > > > >>> > > > >>> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I >>> receive >>> > > > below error >>> > > > >>> > > > [ERROR] [ERROR] Could not find the selected project in the reactor: >>> > > > :ignite-flink-ext @ >>> > > > >>> > > > Build url >>> > > > >>> > > > >>> > > >>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv >>> > > > >>> > > > Can you please let me know if you faced similar problem with >>> teamcity >>> > > > build? >>> > > > >>> > > > I can set Ignite Extensions dependencies to released ignite-core >>> > > artifacts >>> > > > version like 2.7.6 and build works fine. >>> > > > >>> > > > Regards, >>> > > > Saikat >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra < >>> saikat.mai...@gmail.com> >>> > > > wrote: >>> > > > >>> > > > > Hello Denis, >>> > > > > >>> > > > > Thank you for your email and sharing your thoughts on the release >>> > > > process. >>> > > > > I will update the artifact id and dependencies for >>> ignite-extensions >>> > > > > accordingly. >>> > > > > >>> > > > > I had created Ignite-Extensions project as a root level project >>> and in >>> > > > > teamcity I was facing issues pulling dependencies for >>> 2.8.0-SNAPSHOT >>> > > > > whereas I was able to pull dependencies for ignite-core 2.7.6 >>> from >>> > > maven >>> > > > > central. I will look into it further why teamcity build was not >>> able to >>> > > > > pull snapshot dependencies. >>> > > > > >>> > > > > I will also create "ignite-core-2.9+" branch for the upcoming >>> release >>> > > > > process. >>> > > > > >>> > > > > Thank you, >>> > > > > Saikat >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dma...@apache.org> >>> wrote: >>> > > > > >>> > > > >> Hi Saikat, >>> > > > >> >>> > > > >> Thanks for driving this activity forward and raising the >>> question. Let >>> > > > me >>> > > > >> share my thoughts below and let's see what the broader community >>> > > thinks. >>> > > > >> >>> > > > >> Each extension needs to have its own version unrelated to the >>> core and >>> > > > >> Maven's groupId parameter for extension artifacts should be >>> > > > >> "org.apache.ignite.ext". For instance, the very first release >>> of Flink >>> > > > in >>> > > > >> the form of extension should be pulled from Maven this way >>> > > > >> >>> > > > >> <dependency> >>> > > > >> <groupId>org.apache.ignite.ext</groupId> >>> > > > >> <artifactId>ignite-flink</artifactId> >>> > > > >> <version>1.0.0</version> >>> > > > >> </dependency> >>> > > > >> >>> > > > >> When it comes to the releases, all the extensions need to be >>> verified >>> > > > for >>> > > > >> an upcoming release and updated if needed (with the version >>> increase >>> > > > only >>> > > > >> for those updated). Thus, looks like the extensions master >>> needs to be >>> > > > >> linked to the latest Ignite core snapshot. Whenever the core >>> will be >>> > > > being >>> > > > >> prepared and any extensions need to be modified we can take this >>> > > > approach: >>> > > > >> >>> > > > >> - Create a branch of extensions for the upcoming core >>> release. For >>> > > > >> instance, "ignite-core-2.9+" branch. That's just the branch >>> name >>> > > (and >>> > > > >> not >>> > > > >> any Maven artifact name) with "+" sign implying that the >>> updated >>> > > > >> extensions >>> > > > >> will work for Ignite 2.9 and later until we need to update >>> them >>> > > again >>> > > > >> creating a release branch like "ignite-core-2.14+" >>> > > > >> - If only a subset of the extensions was updated then we >>> need to >>> > > > >> release >>> > > > >> those extensions to Maven. The goal is to avoid the practice >>> of >>> > > > >> publishing >>> > > > >> Flink or any other extension to Maven for every core release >>> if >>> > > there >>> > > > >> are >>> > > > >> no changes. >>> > > > >> - As for a ZIP archive, we should prepare the archive for a >>> > > download >>> > > > >> with the name like "ignite-core-2.9+" >>> > > > >> >>> > > > >> >>> > > > >> - >>> > > > >> Denis >>> > > > >> >>> > > > >> >>> > > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra < >>> > > saikat.mai...@gmail.com> >>> > > > >> wrote: >>> > > > >> >>> > > > >> > Hello, >>> > > > >> > >>> > > > >> > I wanted to connect and discuss on the release process for >>> > > > >> > ignite-extensions. As of today all our integrations since >>> released >>> > > > >> together >>> > > > >> > were able to run build based on latest snapshot for example >>> the >>> > > > current >>> > > > >> > build depends on 2.8.0-SNAPSHOT. If we are making >>> ignite-extensions >>> > > as >>> > > > >> > separate project with different release cycle then it make >>> sense to >>> > > > have >>> > > > >> > dependencies on core modules based on released artifact for >>> example >>> > > > the >>> > > > >> > dependency for ignite-core would be 2.7.6 >>> > > > >> > >>> > > > >> > Please review and share your thoughts. >>> > > > >> > >>> > > > >> > PR https://github.com/apache/ignite-extensions/pull/1 >>> > > > >> > >>> > > > >> > Regards >>> > > > >> > Saikat >>> > > > >> > >>> > > > >> >>> > > > > >>> > > > >>> > > >>> >>> >>> >>> -- >>> Best regards, >>> Ivan Pavlukhin >>> >>
