Re: Jenkins Pipeline Build Missing Deploy Phase?

[Pascal Schumacher]

Hi Zoran,

thanks for looking into this!

The last build was successful, but nothing was deployed.

I guess we are affected by "MDEPLOY-193: deployAtEnd does not deploy 
artifacts for multi-module project with build extensions" 
https://issues.apache.org/jira/browse/MDEPLOY-193 (or 
https://issues.apache.org/jira/browse/MDEPLOY-226 or 
https://issues.apache.org/jira/browse/MDEPLOY-201). :(


- Pascal

Am 19.03.2018 um 08:58 schrieb Zoran Regvart:

Hello,
no luck, the build failed on `bom-generator-maven-plugin` because no
Maven artefact was attached for that module in Maven reactor. I've
replaced the `install` goal in `Build` stage with `deploy` as at that
point the Maven reactor is populated.
Previously deploying to ASF Maven repository was done by Jenkins Maven
plugin which kept track of all produced artefacts so it can be
performed separately from the main Maven build. With pipeline that is
no longer possible.
I've added `deployAtEnd` flag so we deploy to the repository only if
the build passes. This doesn't mean that the -SNAPSHOT artefacts in
pass code checks or unit/integration tests, but that's how it used to
work with in the previous job constalation[1].

zoran

[1] https://builds.apache.org/view/C/view/Apache Camel/job/Camel.trunk.notest/

On Sun, Mar 18, 2018 at 9:23 PM, Zoran Regvart  wrote:

Hi Pascal,
good catch! The `Deploy` stage was using `install` phase whereas it
should invoke the maven-deploy-plugin (i.e. `deploy:deploy`). Let's
see if artifacts get deployed in pipeline build 140[1].

I've also configured 4h polling period, seems that master wasn't built
from 15th, I'm not 100% sure if this is an issue with gitpubsub[2] or
with the way the job is configured. From what I understand Jenkins
should be notified of any change to any ASF git repository.

zoran

[1] 
https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel/job/master/140/
[2] https://www.apache.org/dev/gitpubsub.html

On Sun, Mar 18, 2018 at 12:42 PM, Pascal Schumacher
 wrote:

Hello,

Jenkins successfully build camel a few hours ago, but when I try to build
only camel-core locally I get:

[INFO] Building Camel :: Core 2.22.0-SNAPSHOT
[INFO] [ jar
]-
Downloading from apache.snapshots:
http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/maven-metadata.xml
Downloading from apache.snapshots:
http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/camel-package-maven-plugin-2.22.0-SNAPSHOT.pom
[WARNING] The POM for
org.apache.camel:camel-package-maven-plugin:jar:2.22.0-SNAPSHOT is missing,
no dependency information available
Downloading from apache.snapshots:
http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/camel-package-maven-plugin-2.22.0-SNAPSHOT.jar

[ERROR] Plugin org.apache.camel:camel-package-maven-plugin:2.22.0-SNAPSHOT
or one of its dependencies could not be resolved: Could not find artifact
org.apache.camel:camel-package-maven-plugin:jar:2.22.0-SNAPSHOT in
apache.snapshots (http://repository.apache.org/snapshots/) -> [Help 1]

https://repository.apache.org/content/groups/snapshots/org/apache/camel/apache-camel/
does not contains 2.22.0-SNAPSHOT and the build output of
https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel/job/master/139/console
does not contain any maven-deploy-plugin output.

So I guess the Jenkins pipeline build is missing the deploy phase, or are
camel-snapshot supposed to be deployed my some other means?

Cheers,

Pascal





--
Zoran Regvart

Re: Spring Boot 2 migration started - Status and things to help with

[Claus Ibsen]

Hi

I have created sub-tasks in JIRA to make it easier to see the outstanding tasks
https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-11430



On Mon, Mar 19, 2018 at 3:03 PM, Tomohisa Igarashi
 wrote:
> Hi Claus,
>
> I want to take some from the list, if there's no taker I would just start 
> from 1). These definitely help me to get familiar with spring-boot :-)
>
> Thanks,
> Tomo
>
>
> On Sat, 2018-03-17 at 17:31 +0100, Claus Ibsen wrote:
>> Hi
>>
>> I started the migration from spring boot 1 to 2 in a new branch
>> boot2ga as the older branch boot2 was out of date. Then I have copied
>> over the good parts and got the project to build and the
>> camel-spring-boot is now using SB2.
>>
>> To support both Spring Boot 1.x and 2.x at the same time is too
>> complex, and therefore we are migrating to SB2 and dropping SB1
>> support in the upcoming Camel 2.22.0 release.
>>
>>
>> The branch is at (will push in a little while):
>>
>> https://github.com/apache/camel/tree/boot2ga
>>
>>
>> Outstanding work
>> =
>>
>>
>> 1)
>> There were some newer unit tests and a few minor new changes to some
>> of the actuator stuff that was done on master branch that was not
>> migrated to the older boot2 branch. So we need to check upon this and
>> then add those again. Its not as easy as the actuator api has changed
>> alot from SB1 to SB2.
>>
>> 2)
>> Migrate the itests/camel-itest-spring-boot from SB1 to SB2. I think we
>> had to do some changes here as well. Those has not yet been migrated.
>>
>> 3)
>> Jackson was upgraded from 2.8 to 2.9 as SB2 uses this newer version.
>> This may affect the camel-swagger component.
>>
>> 4)
>> Look at upgrading other dependencies that SB2 supports/uses. For
>> example metrics. And there is also the camel-lucene / camel-solr
>> components that can be upgraded to newer versions.
>>
>> 5)
>> Run all of the unit tests.
>>
>> 6)
>> Check the examples that uses spring-boot and check that they work and
>> if there is something to update/modify.
>>
>> 7)
>> Update to Karaf 4.2.0 when its GA as we have upgraded to Spring
>> Framework 5 by default now.
>>
>> 8)
>> Look at the camel-archetype-spring-boot if it need migration as well.
>>
>> 9)
>> Migrate the application.properties / yaml to the new SB2 naming. They
>> changed the prefix and some keys to new names. We need to do this in
>> our examples / camel-spring-boot unit tests etc. See the SB2 migration
>> guide: 
>> https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide
>>
>> 10)
>> Maybe there is some new feature etc in SB2 we can leverage. I noticed
>> there is some InfoContributor you can do in actuator to provide some
>> info details. Maybe we can do a CamelContextInfoContributor that can
>> give you the camel version etc.
>>
>>
>>
>>
>>
>>



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Spring Boot 2 migration started - Status and things to help with

[Tomohisa Igarashi]

Hi Claus,

I want to take some from the list, if there's no taker I would just start from 
1). These definitely help me to get familiar with spring-boot :-)

Thanks,
Tomo


On Sat, 2018-03-17 at 17:31 +0100, Claus Ibsen wrote:
> Hi
> 
> I started the migration from spring boot 1 to 2 in a new branch
> boot2ga as the older branch boot2 was out of date. Then I have copied
> over the good parts and got the project to build and the
> camel-spring-boot is now using SB2.
> 
> To support both Spring Boot 1.x and 2.x at the same time is too
> complex, and therefore we are migrating to SB2 and dropping SB1
> support in the upcoming Camel 2.22.0 release.
> 
> 
> The branch is at (will push in a little while):
> 
> https://github.com/apache/camel/tree/boot2ga
> 
> 
> Outstanding work
> =
> 
> 
> 1)
> There were some newer unit tests and a few minor new changes to some
> of the actuator stuff that was done on master branch that was not
> migrated to the older boot2 branch. So we need to check upon this and
> then add those again. Its not as easy as the actuator api has changed
> alot from SB1 to SB2.
> 
> 2)
> Migrate the itests/camel-itest-spring-boot from SB1 to SB2. I think we
> had to do some changes here as well. Those has not yet been migrated.
> 
> 3)
> Jackson was upgraded from 2.8 to 2.9 as SB2 uses this newer version.
> This may affect the camel-swagger component.
> 
> 4)
> Look at upgrading other dependencies that SB2 supports/uses. For
> example metrics. And there is also the camel-lucene / camel-solr
> components that can be upgraded to newer versions.
> 
> 5)
> Run all of the unit tests.
> 
> 6)
> Check the examples that uses spring-boot and check that they work and
> if there is something to update/modify.
> 
> 7)
> Update to Karaf 4.2.0 when its GA as we have upgraded to Spring
> Framework 5 by default now.
> 
> 8)
> Look at the camel-archetype-spring-boot if it need migration as well.
> 
> 9)
> Migrate the application.properties / yaml to the new SB2 naming. They
> changed the prefix and some keys to new names. We need to do this in
> our examples / camel-spring-boot unit tests etc. See the SB2 migration
> guide: 
> https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide
> 
> 10)
> Maybe there is some new feature etc in SB2 we can leverage. I noticed
> there is some InfoContributor you can do in actuator to provide some
> info details. Maybe we can do a CamelContextInfoContributor that can
> give you the camel version etc.
> 
> 
> 
> 
> 
> 

Re: Spring Boot 2 migration started - Status and things to help with

[Claus Ibsen]

Hi

So the tests/camel-itest-spring-boot can be run from within the IDEA /
Eclipse if you run the unit tests (eg right click -> run test)
But when running from command line it fails.

For example CamelBeanioTest etc

Well at least we are getting closer.


-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Spring Boot 2 migration started - Status and things to help with

[Claus Ibsen]

Hi



On Mon, Mar 19, 2018 at 8:04 AM, Claus Ibsen  wrote:
> Hi
>
> After running full testing we have failures on
>
> - camel-spring-cloud
> - camel-spring-cloud-netflix

I have fixed these two - they require using latest releases of
spring-cloud which is not yet on maven central, so we pull them from
spring maven repo.


> - camel-itest (not sure if this is a spring issue)

there is an error related to camel-spring-security / spring-security
upgraded and it fails with some password encoder stuff.


> - camel-example-spring-boot-rest-jpa
>

this now fails with the auto generated ids for the @Entity. If you
assign the values as 1, and 2 in the data.sql script then it works.

There is also a problem with

- camel-example-spring-cloud-servicecall



> And of course all the itests (maybe Nicola have time to look into this)
> - camel-itest-spring-boot
>
>
>
> On Sat, Mar 17, 2018 at 5:31 PM, Claus Ibsen  wrote:
>> Hi
>>
>> I started the migration from spring boot 1 to 2 in a new branch
>> boot2ga as the older branch boot2 was out of date. Then I have copied
>> over the good parts and got the project to build and the
>> camel-spring-boot is now using SB2.
>>
>> To support both Spring Boot 1.x and 2.x at the same time is too
>> complex, and therefore we are migrating to SB2 and dropping SB1
>> support in the upcoming Camel 2.22.0 release.
>>
>>
>> The branch is at (will push in a little while):
>>
>> https://github.com/apache/camel/tree/boot2ga
>>
>>
>> Outstanding work
>> =
>>
>>
>> 1)
>> There were some newer unit tests and a few minor new changes to some
>> of the actuator stuff that was done on master branch that was not
>> migrated to the older boot2 branch. So we need to check upon this and
>> then add those again. Its not as easy as the actuator api has changed
>> alot from SB1 to SB2.
>>
>> 2)
>> Migrate the itests/camel-itest-spring-boot from SB1 to SB2. I think we
>> had to do some changes here as well. Those has not yet been migrated.
>>
>> 3)
>> Jackson was upgraded from 2.8 to 2.9 as SB2 uses this newer version.
>> This may affect the camel-swagger component.
>>
>> 4)
>> Look at upgrading other dependencies that SB2 supports/uses. For
>> example metrics. And there is also the camel-lucene / camel-solr
>> components that can be upgraded to newer versions.
>>
>> 5)
>> Run all of the unit tests.
>>
>> 6)
>> Check the examples that uses spring-boot and check that they work and
>> if there is something to update/modify.
>>
>> 7)
>> Update to Karaf 4.2.0 when its GA as we have upgraded to Spring
>> Framework 5 by default now.
>>
>> 8)
>> Look at the camel-archetype-spring-boot if it need migration as well.
>>
>> 9)
>> Migrate the application.properties / yaml to the new SB2 naming. They
>> changed the prefix and some keys to new names. We need to do this in
>> our examples / camel-spring-boot unit tests etc. See the SB2 migration
>> guide: 
>> https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide
>>
>> 10)
>> Maybe there is some new feature etc in SB2 we can leverage. I noticed
>> there is some InfoContributor you can do in actuator to provide some
>> info details. Maybe we can do a CamelContextInfoContributor that can
>> give you the camel version etc.
>>
>>
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> -
>> http://davsclaus.com @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Jenkins Pipeline Build Missing Deploy Phase?

[Zoran Regvart]

Hello,
no luck, the build failed on `bom-generator-maven-plugin` because no
Maven artefact was attached for that module in Maven reactor. I've
replaced the `install` goal in `Build` stage with `deploy` as at that
point the Maven reactor is populated.
Previously deploying to ASF Maven repository was done by Jenkins Maven
plugin which kept track of all produced artefacts so it can be
performed separately from the main Maven build. With pipeline that is
no longer possible.
I've added `deployAtEnd` flag so we deploy to the repository only if
the build passes. This doesn't mean that the -SNAPSHOT artefacts in
pass code checks or unit/integration tests, but that's how it used to
work with in the previous job constalation[1].

zoran

[1] https://builds.apache.org/view/C/view/Apache Camel/job/Camel.trunk.notest/

On Sun, Mar 18, 2018 at 9:23 PM, Zoran Regvart  wrote:
> Hi Pascal,
> good catch! The `Deploy` stage was using `install` phase whereas it
> should invoke the maven-deploy-plugin (i.e. `deploy:deploy`). Let's
> see if artifacts get deployed in pipeline build 140[1].
>
> I've also configured 4h polling period, seems that master wasn't built
> from 15th, I'm not 100% sure if this is an issue with gitpubsub[2] or
> with the way the job is configured. From what I understand Jenkins
> should be notified of any change to any ASF git repository.
>
> zoran
>
> [1] 
> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel/job/master/140/
> [2] https://www.apache.org/dev/gitpubsub.html
>
> On Sun, Mar 18, 2018 at 12:42 PM, Pascal Schumacher
>  wrote:
>> Hello,
>>
>> Jenkins successfully build camel a few hours ago, but when I try to build
>> only camel-core locally I get:
>>
>> [INFO] Building Camel :: Core 2.22.0-SNAPSHOT
>> [INFO] [ jar
>> ]-
>> Downloading from apache.snapshots:
>> http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/maven-metadata.xml
>> Downloading from apache.snapshots:
>> http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/camel-package-maven-plugin-2.22.0-SNAPSHOT.pom
>> [WARNING] The POM for
>> org.apache.camel:camel-package-maven-plugin:jar:2.22.0-SNAPSHOT is missing,
>> no dependency information available
>> Downloading from apache.snapshots:
>> http://repository.apache.org/snapshots/org/apache/camel/camel-package-maven-plugin/2.22.0-SNAPSHOT/camel-package-maven-plugin-2.22.0-SNAPSHOT.jar
>>
>> [ERROR] Plugin org.apache.camel:camel-package-maven-plugin:2.22.0-SNAPSHOT
>> or one of its dependencies could not be resolved: Could not find artifact
>> org.apache.camel:camel-package-maven-plugin:jar:2.22.0-SNAPSHOT in
>> apache.snapshots (http://repository.apache.org/snapshots/) -> [Help 1]
>>
>> https://repository.apache.org/content/groups/snapshots/org/apache/camel/apache-camel/
>> does not contains 2.22.0-SNAPSHOT and the build output of
>> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel/job/master/139/console
>> does not contain any maven-deploy-plugin output.
>>
>> So I guess the Jenkins pipeline build is missing the deploy phase, or are
>> camel-snapshot supposed to be deployed my some other means?
>>
>> Cheers,
>>
>> Pascal
>>
>>
>
>
>
> --
> Zoran Regvart



-- 
Zoran Regvart

Re: Spring Boot 2 migration started - Status and things to help with

[Claus Ibsen]

Hi

After running full testing we have failures on

- camel-spring-cloud
- camel-spring-cloud-netflix
- camel-itest (not sure if this is a spring issue)
- camel-example-spring-boot-rest-jpa

And of course all the itests (maybe Nicola have time to look into this)
- camel-itest-spring-boot



On Sat, Mar 17, 2018 at 5:31 PM, Claus Ibsen  wrote:
> Hi
>
> I started the migration from spring boot 1 to 2 in a new branch
> boot2ga as the older branch boot2 was out of date. Then I have copied
> over the good parts and got the project to build and the
> camel-spring-boot is now using SB2.
>
> To support both Spring Boot 1.x and 2.x at the same time is too
> complex, and therefore we are migrating to SB2 and dropping SB1
> support in the upcoming Camel 2.22.0 release.
>
>
> The branch is at (will push in a little while):
>
> https://github.com/apache/camel/tree/boot2ga
>
>
> Outstanding work
> =
>
>
> 1)
> There were some newer unit tests and a few minor new changes to some
> of the actuator stuff that was done on master branch that was not
> migrated to the older boot2 branch. So we need to check upon this and
> then add those again. Its not as easy as the actuator api has changed
> alot from SB1 to SB2.
>
> 2)
> Migrate the itests/camel-itest-spring-boot from SB1 to SB2. I think we
> had to do some changes here as well. Those has not yet been migrated.
>
> 3)
> Jackson was upgraded from 2.8 to 2.9 as SB2 uses this newer version.
> This may affect the camel-swagger component.
>
> 4)
> Look at upgrading other dependencies that SB2 supports/uses. For
> example metrics. And there is also the camel-lucene / camel-solr
> components that can be upgraded to newer versions.
>
> 5)
> Run all of the unit tests.
>
> 6)
> Check the examples that uses spring-boot and check that they work and
> if there is something to update/modify.
>
> 7)
> Update to Karaf 4.2.0 when its GA as we have upgraded to Spring
> Framework 5 by default now.
>
> 8)
> Look at the camel-archetype-spring-boot if it need migration as well.
>
> 9)
> Migrate the application.properties / yaml to the new SB2 naming. They
> changed the prefix and some keys to new names. We need to do this in
> our examples / camel-spring-boot unit tests etc. See the SB2 migration
> guide: 
> https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide
>
> 10)
> Maybe there is some new feature etc in SB2 we can leverage. I noticed
> there is some InfoContributor you can do in actuator to provide some
> info details. Maybe we can do a CamelContextInfoContributor that can
> give you the camel version etc.
>
>
>
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2