Re: Understanding the snapshot and release process

[Christofer Dutz]

Well the big advantage of running the jobs on ASF Jenkins is that you can 
auto-deploy to Nexus (ASF Maven Repo) as well as commit to Git without having 
to store some ones credentials on an external machine. Infra is very helpful in 
getting even complicated setups running (I manage the Windows based flex 
builds, which need Flashplayer, Browsers and Selenium). I would be more than 
happy to help setup everything.

Chris

Am 28.05.17, 17:26 schrieb "John D. Ament" :

On Sun, May 28, 2017 at 11:12 AM Christofer Dutz 
wrote:

> The Maven build should work on the ASF Jenkins and hereby publish the
> snapshots. I think that in the flex project I added this to ensure its
> added to the generated "site".
>
> Which one is the hosted Jenkins you are referring to? The asf Jenkins?
>
>
Yes, the ASF jenkins.


> And what should be the problem with Travis? Why are you using two ci
> servers?
>

AFAIK, Edgent only uses travis, and doesn't currently run jobs on ASF
Jenkins.  At least if they do, I was unable to find a job with a name that
was easy to identify.


> Chris
>
>
>
> Von meinem Samsung Galaxy Smartphone gesendet.
>
>
>  Ursprüngliche Nachricht 
> Von: "John D. Ament" 
> Datum: 28.05.17 16:20 (GMT+01:00)
> An: dev@edgent.apache.org
> Betreff: Re: Understanding the snapshot and release process
>
> The maven based build seems to be much more maintainable.  I'm glad you
> guys came together to solve this problem :-)
>
> RE distribution of snapshots.  We only presently support publishing via 
our
> hosted jenkins, however Edgent currently builds in Travis.  So that would
> need to be changed, or some other solution.
>
> You also can drop the repositories/pluginRepositories sections of the new
> pom.  Its not needed.
>
>
> John
>
> On Sun, May 28, 2017 at 10:03 AM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Hi guys,
> >
> > Strange people these Germans … if it rains, they complain about it and
> how
> > much they would like to go swimming in the sun … now that it’s 32° and
> the
> > sun is shining, no one wants to go swimming, cause it’s too fu***g hot
> ;-)
> > … well this way I had some time to finish the maven migration … I just
> > pushed the changes to the “feature/maven” branch of my fork on github:
> > https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
> >
> > Here a short list on what little monsters I found siting under some
> rocks:
> >
> > 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> > jar module
> > 2) AppServiceTest in provides/direct relies on system-properties to load
> > jar produced by another module → Ideally this should be refactored to
> find
> > the jar on the classpath
> > 3) HttpServerTest in console/server relies on console.war
> > 4) console/servlets module compiled to something with a name completely
> > unrelated to the project
> > 5) test/svt requires samples project → Some modules rely on samples.
> While
> > I think, this is ok for Samples themselves, nothing outside the samples
> > should rely on samples (my opinion)
> > 6) Almost all tests relying on successful SSL handshakes in
> > wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> > have time, I could imagine this to be related to non US JVM encryption
> > issues we are also seeing in the Flex project)
> > 7) Some unit-tests should more be integration-tests
> >
> > I didn’t migrate the stuff in “platform” as I didn’t quite understand
> what
> > they are used for and what they should do.
> >
> > Hope you like the changes. With a build like this it should be easy to
> > setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> > machines with a simple one or two-liner.
> >
> > Chris
> >
> >
> > Am 27.05.17, 19:16 schrieb "Christofer Dutz"  >:
> >
> > I did have some free time this weekend, so I thought I’d give the
> > Maven thing a try. And right now I haven finished, but I think I’m about
> > 80% done. So far there have been one or two little monsters under some
> > rocks … but they weren’t too nasty ;-)
> >
> > Regarding the classpath stuff in the manifest: I really dislike this
> > way of setting the classpath as it usually causes a lot of problems when
> > using it. It requires libs to be in a predefined directory structure and
> > follow a given naming convention. Try to use this in a project built 
with
> > Maven (just picking this as I saw this would be a problem for a 

Re: Understanding the snapshot and release process

[John D. Ament]

On Sun, May 28, 2017 at 11:12 AM Christofer Dutz 
wrote:

> The Maven build should work on the ASF Jenkins and hereby publish the
> snapshots. I think that in the flex project I added this to ensure its
> added to the generated "site".
>
> Which one is the hosted Jenkins you are referring to? The asf Jenkins?
>
>
Yes, the ASF jenkins.


> And what should be the problem with Travis? Why are you using two ci
> servers?
>

AFAIK, Edgent only uses travis, and doesn't currently run jobs on ASF
Jenkins.  At least if they do, I was unable to find a job with a name that
was easy to identify.


> Chris
>
>
>
> Von meinem Samsung Galaxy Smartphone gesendet.
>
>
>  Ursprüngliche Nachricht 
> Von: "John D. Ament" 
> Datum: 28.05.17 16:20 (GMT+01:00)
> An: dev@edgent.apache.org
> Betreff: Re: Understanding the snapshot and release process
>
> The maven based build seems to be much more maintainable.  I'm glad you
> guys came together to solve this problem :-)
>
> RE distribution of snapshots.  We only presently support publishing via our
> hosted jenkins, however Edgent currently builds in Travis.  So that would
> need to be changed, or some other solution.
>
> You also can drop the repositories/pluginRepositories sections of the new
> pom.  Its not needed.
>
>
> John
>
> On Sun, May 28, 2017 at 10:03 AM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Hi guys,
> >
> > Strange people these Germans … if it rains, they complain about it and
> how
> > much they would like to go swimming in the sun … now that it’s 32° and
> the
> > sun is shining, no one wants to go swimming, cause it’s too fu***g hot
> ;-)
> > … well this way I had some time to finish the maven migration … I just
> > pushed the changes to the “feature/maven” branch of my fork on github:
> > https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
> >
> > Here a short list on what little monsters I found siting under some
> rocks:
> >
> > 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> > jar module
> > 2) AppServiceTest in provides/direct relies on system-properties to load
> > jar produced by another module → Ideally this should be refactored to
> find
> > the jar on the classpath
> > 3) HttpServerTest in console/server relies on console.war
> > 4) console/servlets module compiled to something with a name completely
> > unrelated to the project
> > 5) test/svt requires samples project → Some modules rely on samples.
> While
> > I think, this is ok for Samples themselves, nothing outside the samples
> > should rely on samples (my opinion)
> > 6) Almost all tests relying on successful SSL handshakes in
> > wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> > have time, I could imagine this to be related to non US JVM encryption
> > issues we are also seeing in the Flex project)
> > 7) Some unit-tests should more be integration-tests
> >
> > I didn’t migrate the stuff in “platform” as I didn’t quite understand
> what
> > they are used for and what they should do.
> >
> > Hope you like the changes. With a build like this it should be easy to
> > setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> > machines with a simple one or two-liner.
> >
> > Chris
> >
> >
> > Am 27.05.17, 19:16 schrieb "Christofer Dutz"  >:
> >
> > I did have some free time this weekend, so I thought I’d give the
> > Maven thing a try. And right now I haven finished, but I think I’m about
> > 80% done. So far there have been one or two little monsters under some
> > rocks … but they weren’t too nasty ;-)
> >
> > Regarding the classpath stuff in the manifest: I really dislike this
> > way of setting the classpath as it usually causes a lot of problems when
> > using it. It requires libs to be in a predefined directory structure and
> > follow a given naming convention. Try to use this in a project built with
> > Maven (just picking this as I saw this would be a problem for a Maven
> built
> > project)
> >
> > Regarding tests: I did see that there are several Unit tests I would
> > probably more call integration-tests. Especially the ones that require
> > loading of a war (console.war). So I guess you have both some classical
> > unit-tests as well as integration-tests … no need to treat them the same
> > way. Maven ususally has the unit-test and the integration-test phase for
> > exactly this.
> >
> > Regarding jacoco: So I’ll activate that in the maven build too
> >
> > Regarding automatic build: For Flex I setup the build to do a
> > multibranch pipeline build. So whenever someone checks something in
> > “develop”, “releae/*” or “feature/*” it automatically builds that
> > particular branch. However develop and “release/*” are the only ones
> > uploading SNAPSHOT versions to the ASF Nexus.
> >
> > Regarding my commitment: I think Edgent is the tool I was looking for
> > in order 

Re: Understanding the snapshot and release process

[John D. Ament]

The maven based build seems to be much more maintainable.  I'm glad you
guys came together to solve this problem :-)

RE distribution of snapshots.  We only presently support publishing via our
hosted jenkins, however Edgent currently builds in Travis.  So that would
need to be changed, or some other solution.

You also can drop the repositories/pluginRepositories sections of the new
pom.  Its not needed.


John

On Sun, May 28, 2017 at 10:03 AM Christofer Dutz 
wrote:

> Hi guys,
>
> Strange people these Germans … if it rains, they complain about it and how
> much they would like to go swimming in the sun … now that it’s 32° and the
> sun is shining, no one wants to go swimming, cause it’s too fu***g hot ;-)
> … well this way I had some time to finish the maven migration … I just
> pushed the changes to the “feature/maven” branch of my fork on github:
> https://github.com/chrisdutz/incubator-edgent/tree/feature/maven
>
> Here a short list on what little monsters I found siting under some rocks:
>
> 1) Usage of test-jars --> Bad practice → Move code to separate test-util
> jar module
> 2) AppServiceTest in provides/direct relies on system-properties to load
> jar produced by another module → Ideally this should be refactored to find
> the jar on the classpath
> 3) HttpServerTest in console/server relies on console.war
> 4) console/servlets module compiled to something with a name completely
> unrelated to the project
> 5) test/svt requires samples project → Some modules rely on samples. While
> I think, this is ok for Samples themselves, nothing outside the samples
> should rely on samples (my opinion)
> 6) Almost all tests relying on successful SSL handshakes in
> wsclient-javax.websocket are failing (I’ll investigate this as soon as I
> have time, I could imagine this to be related to non US JVM encryption
> issues we are also seeing in the Flex project)
> 7) Some unit-tests should more be integration-tests
>
> I didn’t migrate the stuff in “platform” as I didn’t quite understand what
> they are used for and what they should do.
>
> Hope you like the changes. With a build like this it should be easy to
> setup the CI to auto distribute SNAPSHOTs and execute releases on ASF
> machines with a simple one or two-liner.
>
> Chris
>
>
> Am 27.05.17, 19:16 schrieb "Christofer Dutz" :
>
> I did have some free time this weekend, so I thought I’d give the
> Maven thing a try. And right now I haven finished, but I think I’m about
> 80% done. So far there have been one or two little monsters under some
> rocks … but they weren’t too nasty ;-)
>
> Regarding the classpath stuff in the manifest: I really dislike this
> way of setting the classpath as it usually causes a lot of problems when
> using it. It requires libs to be in a predefined directory structure and
> follow a given naming convention. Try to use this in a project built with
> Maven (just picking this as I saw this would be a problem for a Maven built
> project)
>
> Regarding tests: I did see that there are several Unit tests I would
> probably more call integration-tests. Especially the ones that require
> loading of a war (console.war). So I guess you have both some classical
> unit-tests as well as integration-tests … no need to treat them the same
> way. Maven ususally has the unit-test and the integration-test phase for
> exactly this.
>
> Regarding jacoco: So I’ll activate that in the maven build too
>
> Regarding automatic build: For Flex I setup the build to do a
> multibranch pipeline build. So whenever someone checks something in
> “develop”, “releae/*” or “feature/*” it automatically builds that
> particular branch. However develop and “release/*” are the only ones
> uploading SNAPSHOT versions to the ASF Nexus.
>
> Regarding my commitment: I think Edgent is the tool I was looking for
> in order to make my SCADA project feature-perfect. So I have a strong
> interest in the project maturing. I do have plans for a longer involvement
> in the project. Probably I would be trying to create an open-source “plc”
> connector (which would probably have to be developed outside of Apache due
> to libraries still being GPL). But I do plan on staying around some time ;-)
>
> Regarding the usage: Maven or Gradle based application developers
> could directly utilize SNAPSHOT and release artifacts as they are released.
> Additionally, I would create a so-called assembly that produces the same
> zip or tar-gz archives the users have been using. So there shouldn’t be too
> much of a change.
>
> So now I’ll try to finish the rest of the connector modules maven
> conversion (
>
> Chris
>
> Am 25.05.17, 19:37 schrieb "Dale LaBossiere" :
>
> Awaiting others to chime in on this “switch to mvn” subject.
> A “don’t care” (+/- 0) response is preferred over silence.
>
> In the mean time...
>
> > On May 24, 2017, at 11:41 AM, 

Re: Understanding the snapshot and release process

[Christofer Dutz]

Hi guys,

Strange people these Germans … if it rains, they complain about it and how much 
they would like to go swimming in the sun … now that it’s 32° and the sun is 
shining, no one wants to go swimming, cause it’s too fu***g hot ;-) … well this 
way I had some time to finish the maven migration … I just pushed the changes 
to the “feature/maven” branch of my fork on github: 
https://github.com/chrisdutz/incubator-edgent/tree/feature/maven

Here a short list on what little monsters I found siting under some rocks:

1) Usage of test-jars --> Bad practice → Move code to separate test-util jar 
module
2) AppServiceTest in provides/direct relies on system-properties to load jar 
produced by another module → Ideally this should be refactored to find the jar 
on the classpath
3) HttpServerTest in console/server relies on console.war
4) console/servlets module compiled to something with a name completely 
unrelated to the project
5) test/svt requires samples project → Some modules rely on samples. While I 
think, this is ok for Samples themselves, nothing outside the samples should 
rely on samples (my opinion)
6) Almost all tests relying on successful SSL handshakes in 
wsclient-javax.websocket are failing (I’ll investigate this as soon as I have 
time, I could imagine this to be related to non US JVM encryption issues we are 
also seeing in the Flex project)
7) Some unit-tests should more be integration-tests

I didn’t migrate the stuff in “platform” as I didn’t quite understand what they 
are used for and what they should do.

Hope you like the changes. With a build like this it should be easy to setup 
the CI to auto distribute SNAPSHOTs and execute releases on ASF machines with a 
simple one or two-liner.

Chris


Am 27.05.17, 19:16 schrieb "Christofer Dutz" :

I did have some free time this weekend, so I thought I’d give the Maven 
thing a try. And right now I haven finished, but I think I’m about 80% done. So 
far there have been one or two little monsters under some rocks … but they 
weren’t too nasty ;-)

Regarding the classpath stuff in the manifest: I really dislike this way of 
setting the classpath as it usually causes a lot of problems when using it. It 
requires libs to be in a predefined directory structure and follow a given 
naming convention. Try to use this in a project built with Maven (just picking 
this as I saw this would be a problem for a Maven built project)

Regarding tests: I did see that there are several Unit tests I would 
probably more call integration-tests. Especially the ones that require loading 
of a war (console.war). So I guess you have both some classical unit-tests as 
well as integration-tests … no need to treat them the same way. Maven ususally 
has the unit-test and the integration-test phase for exactly this.

Regarding jacoco: So I’ll activate that in the maven build too 

Regarding automatic build: For Flex I setup the build to do a multibranch 
pipeline build. So whenever someone checks something in “develop”, “releae/*” 
or “feature/*” it automatically builds that particular branch. However develop 
and “release/*” are the only ones uploading SNAPSHOT versions to the ASF Nexus.

Regarding my commitment: I think Edgent is the tool I was looking for in 
order to make my SCADA project feature-perfect. So I have a strong interest in 
the project maturing. I do have plans for a longer involvement in the project. 
Probably I would be trying to create an open-source “plc” connector (which 
would probably have to be developed outside of Apache due to libraries still 
being GPL). But I do plan on staying around some time ;-)

Regarding the usage: Maven or Gradle based application developers could 
directly utilize SNAPSHOT and release artifacts as they are released. 
Additionally, I would create a so-called assembly that produces the same zip or 
tar-gz archives the users have been using. So there shouldn’t be too much of a 
change.

So now I’ll try to finish the rest of the connector modules maven 
conversion (

Chris

Am 25.05.17, 19:37 schrieb "Dale LaBossiere" :

Awaiting others to chime in on this “switch to mvn” subject.
A “don’t care” (+/- 0) response is preferred over silence.

In the mean time...

> On May 24, 2017, at 11:41 AM, Christofer Dutz 
 wrote:
> ...
> I just checked out everything and managed to get things imported in 
IntelliJ after a little struggle … this is a good job you did :-)
Can’t take all the credit (or blame :-)
> 
> Right now, it seems as if there were Ant+Gradle+Eclipse build files 
in there. While it might seem ... 
gradle is the only CLI way to build edgent.  Any ant build.xml files 
are either (a) residual cruft that should be removed (my bad) or (b) needed to 
leverage the ant machinery