On Sun, May 28, 2017 at 11:12 AM Christofer Dutz <[email protected]> wrote:
> The Maven build should work on the ASF Jenkins and hereby publish the > snapshots. I think that in the flex project I added this to ensure its > added to the generated "site". > > Which one is the hosted Jenkins you are referring to? The asf Jenkins? > > Yes, the ASF jenkins. > And what should be the problem with Travis? Why are you using two ci > servers? > AFAIK, Edgent only uses travis, and doesn't currently run jobs on ASF Jenkins. At least if they do, I was unable to find a job with a name that was easy to identify. > Chris > > > > Von meinem Samsung Galaxy Smartphone gesendet. > > > -------- Ursprüngliche Nachricht -------- > Von: "John D. Ament" <[email protected]> > Datum: 28.05.17 16:20 (GMT+01:00) > An: [email protected] > Betreff: Re: Understanding the snapshot and release process > > The maven based build seems to be much more maintainable. I'm glad you > guys came together to solve this problem :-) > > RE distribution of snapshots. We only presently support publishing via our > hosted jenkins, however Edgent currently builds in Travis. So that would > need to be changed, or some other solution. > > You also can drop the repositories/pluginRepositories sections of the new > pom. Its not needed. > > > John > > On Sun, May 28, 2017 at 10:03 AM Christofer Dutz < > [email protected]> > wrote: > > > Hi guys, > > > > Strange people these Germans … if it rains, they complain about it and > how > > much they would like to go swimming in the sun … now that it’s 32° and > the > > sun is shining, no one wants to go swimming, cause it’s too fu***g hot > ;-) > > … well this way I had some time to finish the maven migration … I just > > pushed the changes to the “feature/maven” branch of my fork on github: > > https://github.com/chrisdutz/incubator-edgent/tree/feature/maven > > > > Here a short list on what little monsters I found siting under some > rocks: > > > > 1) Usage of test-jars --> Bad practice → Move code to separate test-util > > jar module > > 2) AppServiceTest in provides/direct relies on system-properties to load > > jar produced by another module → Ideally this should be refactored to > find > > the jar on the classpath > > 3) HttpServerTest in console/server relies on console.war > > 4) console/servlets module compiled to something with a name completely > > unrelated to the project > > 5) test/svt requires samples project → Some modules rely on samples. > While > > I think, this is ok for Samples themselves, nothing outside the samples > > should rely on samples (my opinion) > > 6) Almost all tests relying on successful SSL handshakes in > > wsclient-javax.websocket are failing (I’ll investigate this as soon as I > > have time, I could imagine this to be related to non US JVM encryption > > issues we are also seeing in the Flex project) > > 7) Some unit-tests should more be integration-tests > > > > I didn’t migrate the stuff in “platform” as I didn’t quite understand > what > > they are used for and what they should do. > > > > Hope you like the changes. With a build like this it should be easy to > > setup the CI to auto distribute SNAPSHOTs and execute releases on ASF > > machines with a simple one or two-liner. > > > > Chris > > > > > > Am 27.05.17, 19:16 schrieb "Christofer Dutz" <[email protected] > >: > > > > I did have some free time this weekend, so I thought I’d give the > > Maven thing a try. And right now I haven finished, but I think I’m about > > 80% done. So far there have been one or two little monsters under some > > rocks … but they weren’t too nasty ;-) > > > > Regarding the classpath stuff in the manifest: I really dislike this > > way of setting the classpath as it usually causes a lot of problems when > > using it. It requires libs to be in a predefined directory structure and > > follow a given naming convention. Try to use this in a project built with > > Maven (just picking this as I saw this would be a problem for a Maven > built > > project) > > > > Regarding tests: I did see that there are several Unit tests I would > > probably more call integration-tests. Especially the ones that require > > loading of a war (console.war). So I guess you have both some classical > > unit-tests as well as integration-tests … no need to treat them the same > > way. Maven ususally has the unit-test and the integration-test phase for > > exactly this. > > > > Regarding jacoco: So I’ll activate that in the maven build too > > > > Regarding automatic build: For Flex I setup the build to do a > > multibranch pipeline build. So whenever someone checks something in > > “develop”, “releae/*” or “feature/*” it automatically builds that > > particular branch. However develop and “release/*” are the only ones > > uploading SNAPSHOT versions to the ASF Nexus. > > > > Regarding my commitment: I think Edgent is the tool I was looking for > > in order to make my SCADA project feature-perfect. So I have a strong > > interest in the project maturing. I do have plans for a longer > involvement > > in the project. Probably I would be trying to create an open-source “plc” > > connector (which would probably have to be developed outside of Apache > due > > to libraries still being GPL). But I do plan on staying around some time > ;-) > > > > Regarding the usage: Maven or Gradle based application developers > > could directly utilize SNAPSHOT and release artifacts as they are > released. > > Additionally, I would create a so-called assembly that produces the same > > zip or tar-gz archives the users have been using. So there shouldn’t be > too > > much of a change. > > > > So now I’ll try to finish the rest of the connector modules maven > > conversion ( > > > > Chris > > > > Am 25.05.17, 19:37 schrieb "Dale LaBossiere" <[email protected]>: > > > > Awaiting others to chime in on this “switch to mvn” subject. > > A “don’t care” (+/- 0) response is preferred over silence. > > > > In the mean time... > > > > > On May 24, 2017, at 11:41 AM, Christofer Dutz < > > [email protected]> wrote: > > > ... > > > I just checked out everything and managed to get things > imported > > in IntelliJ after a little struggle … this is a good job you did :-) > > Can’t take all the credit (or blame :-) > > > > > > Right now, it seems as if there were Ant+Gradle+Eclipse build > > files in there. While it might seem ... > > gradle is the only CLI way to build edgent. Any ant build.xml > > files are either (a) residual cruft that should be removed (my bad) or > (b) > > needed to leverage the ant machinery for invoking retrolambda in building > > java7/android compatible versions of the jars (machinery that wasn’t > > converted to pure gradle due to time/effort/value at the time). > Attempting > > to use ant at the top level tells you it doesn’t work :-) As for Eclipse > > .project/.classpath, yup those are live and haven’t been a maintenance > > issue. > > > > fwiw, I just removed all build.xml except the top level one and > > those under platform and “gradle release” worked fine so I’ll create a PR > > to clean them up as a first step. > > > > > ...JavaDoc is generated automatically when running a release > > build together with the usual Maven project reports (is even configured > in > > the apache parent POM together with rat, deployment etc.) > > Javadoc was complicated by creating groupings as well as > excluding > > it for non-API classes. > > > > > ...I didn’t quite understand the “Manifest” thing, but the jar > > plugin does generate this with reasonable defaults, and can be extended > to > > also export the dependencies into that (even if I don’t recommend that). > > An edgent jar’s manifest class-path includes references to its > > immediate dependent edgent jars (not transitive) as well as references to > > its external jar dependencies (transitively). I agree that "compiled in" > > references to specific versions of external dependencies may not be a > great > > idea / is perhaps best eliminated. > > > > My recollection is that by default gradle did not generate any > > manifest class-path. > > > > e.g., just to make this a bit more concrete, this is from > > edgent.connectors.kafka.jar/MANIFEST.MF (see > connectors/kafka/build.gradle > > for more info) > > > > Class-Path: ../../../lib/edgent.api.topology.jar > > ../../../ext/gson-2.2 > > .4.jar ../../../ext/slf4j-api-1.7.12.jar > > ../../../ext/metrics-core-3. > > 1.2.jar ../ext/kafka_2.10-0.8.2.2.jar > > ../ext/kafka-clients-0.8.2.2.ja > > r ../ext/log4j-1.2.16.jar ../ext/metrics-core-2.2.0.jar > > ../ext/scala- > > library-2.10.4.jar ../ext/zkclient-0.3.jar > > ../ext/zookeeper-3.4.6.jar > > > > edgent.api.topology.jar itself has lots of edgent jar > dependencies > > (captured in its manifest) but that’s of no concern to the kafka > > connector. I’m not sure maintaining this is a requirement and, at least > > for gradle, eliminating it would have greatly simplified things. > > > > Test execution environment was also an issue. perhaps more a > > result of structural issues. i.e., the gradle config treated a > component’s > > tests as unit tests, run against the associated component’s .class files. > > The ant system treated them more like integration tests, run against the > > jar files that would be bundled into a binary-release bundle; more like > the > > environment that Edgent-based applications would use. That was very > > important. And due to some manual setup requirements for some connector > > tests are excluded from execution by default. > > > > > … > > > In the Flex project, I also setup the build to run SonarQube > > analysis and automatically generate the documentation from markdown > and/or > > asciidoctor (which I think is very convenient) even automatically update > > and deploy the project website. > > > > We include use of jacoco > > We have travis integration for auto-PR validation. > > It would be nice to have some periodic build/regression testing > > run on the main (master) branch. > > > > > I could offer to create a fork on GitHub, create a feature > > branch there and try to whip up a set of poms that add Maven as fourth > > build system to the list … you could check it out and play around with > it. > > But I’d only do this, if there is any interest in it. > > > > I don’t blame you! > > > > One worry I have is that it ends up like the gradle effort… it > was > > ~trivial to get started but there was then a lot of effort required > > identify and flesh out equivalence with the ant based build result > > artifacts. > > If you’re on board for the long haul in doing a conversion then > > that lessens my concern. > > > > I’m on board to help but don’t want to inherit a “now finish it” > > task :-) > > > > A full switch to a ~simple mvn-based build system would be the > > goal IMO — we’d want to toss gradle. The effort ultimately also entails > > updating a bunch of doc. Not a killer, just not to be overlooked. > > > > It’s unclear to me how the Edgent-based app developers will be > > affected by all of this. OK, they build their Edgent-based app with some > > tool that can utilize a maven repo. Then what? > > > > Today the story is simple: extract the binary-tgz (or a subset of > > it) on the target (edge device), copy your app code jar/classes to the > > target, set the CLASSPATH and go. (assume we also eliminate > > building/distributing a binary-tgz) > > > > Can elaborate on that part of the story? > > > > Thanks! > > — Dale > > > > > > > > > > > > >
