Re: Launchpad stable and launchpad unstable
Hi Bertrand, Am 29.11.16 um 13:25 schrieb Bertrand Delacretaz: Hi Sandro, On Mon, Nov 28, 2016 at 11:45 PM, Sandro Boehme wrote: ...If I checkout and build the latest trunk I would expect that the built projects are available at the runtime of the server. This way I can set breakpoints in the debugger, examine the inner workings and maybe provide an improvement Yes, as a Sling developer that's what you want, and also for testing the whole thing in automated tests. OTOH for Sling to be always release ready we need a launchpad that refers only to released modules, so we need both. I agree that we need both! ...If the launchpad provides only stable versions I would always first have to switch to the snapshot versions using the script and I currently I can't do that, right?... The launchpad/builder/set-sling-snapshots.sh script should allow you to do that but when I wrote it it didn't fully work, see comments inside. -Bertrand I tried it with the current trunk and http://localhost:8080/ returned a 503 with 'AuthenticationSupport service missing. Cannot authenticate request.' 'mvn dependency:resolve | grep org.apache.sling | grep -v SNAPSHOT' returned: [INFO] org.apache.sling:org.apache.sling.commons.scheduler:jar:2.5.2:provided [INFO] org.apache.sling:org.apache.sling.repoinit.parser:jar:1.1.0:provided [INFO] org.apache.sling:org.apache.sling.commons.classloader:jar:1.3.6:provided [INFO]org.apache.sling:org.apache.sling.xss:jar:1.0.16:provided [INFO] org.apache.sling:org.apache.sling.provisioning.model:jar:1.7.0:provided [INFO]org.apache.sling:org.apache.sling.commons.mime:jar:2.1.10:provided [INFO]org.apache.sling:org.apache.sling.event.dea:jar:1.1.0:provided [INFO]org.apache.sling:org.apache.sling.auth.core:jar:1.3.22:provided When I look in the /logs/error.log after startup I see exceptions like 'java.lang.NoClassDefFoundError: javax/jcr/RepositoryException' I feel like I'm always just complaining but I'm afraid it would take way too much time to understand how the launchpad translates the txt files into installed Maven artifacts. Let alone fixing something there. OTOH I think the aspect I've mentioned in my last email is not unimportant. Best, Sandro
Re: Launchpad stable and launchpad unstable
Hi Sandro, On Mon, Nov 28, 2016 at 11:45 PM, Sandro Boehme wrote: > ...If I checkout and build the latest trunk I would expect that the built > projects are available at the runtime of the server. This way I can set > breakpoints in the debugger, examine the inner workings and maybe provide an > improvement Yes, as a Sling developer that's what you want, and also for testing the whole thing in automated tests. OTOH for Sling to be always release ready we need a launchpad that refers only to released modules, so we need both. > ...If the launchpad provides only stable versions I would always first have to > switch to the snapshot versions using the script and I currently I can't do > that, right?... The launchpad/builder/set-sling-snapshots.sh script should allow you to do that but when I wrote it it didn't fully work, see comments inside. -Bertrand
Re: Launchpad stable and launchpad unstable
- the launchpad maintained in trunk should always be the "stable" launchpad, referencing only released dependencies of the sling bundles and oft the integration tests project If I checkout and build the latest trunk I would expect that the built projects are available at the runtime of the server. This way I can set breakpoints in the debugger, examine the inner workings and maybe provide an improvement. If the launchpad provides only stable versions I would always first have to switch to the snapshot versions using the script and I currently I can't do that, right? Best, Sandro PS: Thanks Bertrand for pointing me to this discussion!
Re: Launchpad stable and launchpad unstable
On Wed, 2016-11-02 at 12:02 +0100, Carsten Ziegeler wrote: > Bertrand Delacretaz wrote > > On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler > org> wrote: > > > Bertrand Delacretaz wrote > > > > On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler > > > che.org> wrote: > > > > > ...what problem are we exactly trying to solve? > > > > > > > > ...As described in the original post here - having a stable > > > > launchpad > > > > that's always ready to release, while also continuously testing > > > > the > > > > integration of the latest snapshots. > > > > > > > > > > So if the integration tests always test the latest snapshots then > > > there > > > is nothing we have do to, right?... > > > > We need to test the launchpad that we release, don't we? > > Yeah, sorry that I'm a little bit picky here. All I'm trying that we > get > a description of the full picture. > > We want to test the *same* integration tests against the current > launchpad (which has no snapshot deps) and against the same launchpad > but replacing all sling deps to the latest snapshot versions. > And that might lead us into the situation you describe below. > > > > > Depending on the combination of bundles that release might have > > different behavior than combining the latest snapshots, although > > maybe > > it's not that likely. > > Right, I would argue that it is not very likely. > > > > > As a first step we might manually run the integration tests against > > the launchpad that's about to be released, that's better than > > nothing > > but some tests will fail and we need a manual analysis of that - > > which > > is not impossible if the goal is to release say every 3 months. > > Instead of coming up with a gigantic approach that will not be used > or > only once every two years, I think lets deal with it once we're > there. > Something simple like an exclusion list activated by the same > mechanism > that switches to snapshot dependencies. I've tried to summarise the current state of the discussions at https://cwiki.apache.org/confluence/display/SLING/Moving+the+Sling+La unchpad+to+use+released+artifacts+only I think currently we need to decide/finalise: - how to transform a 'stable' launchpad into an 'unstable' one - how to exclude tests from running on an 'stable' launchpad but keep the running on an 'unstable' launchpad. Comment on the wiki page more than welcome, I have probably missed something :-) Thanks, Robert > > Carsten >
Re: Launchpad stable and launchpad unstable
Bertrand Delacretaz wrote > On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler wrote: >> Bertrand Delacretaz wrote >>> On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler >>> wrote: ...what problem are we exactly trying to solve? >>> ...As described in the original post here - having a stable launchpad >>> that's always ready to release, while also continuously testing the >>> integration of the latest snapshots. >>> >> So if the integration tests always test the latest snapshots then there >> is nothing we have do to, right?... > > We need to test the launchpad that we release, don't we? Yeah, sorry that I'm a little bit picky here. All I'm trying that we get a description of the full picture. We want to test the *same* integration tests against the current launchpad (which has no snapshot deps) and against the same launchpad but replacing all sling deps to the latest snapshot versions. And that might lead us into the situation you describe below. > > Depending on the combination of bundles that release might have > different behavior than combining the latest snapshots, although maybe > it's not that likely. Right, I would argue that it is not very likely. > > As a first step we might manually run the integration tests against > the launchpad that's about to be released, that's better than nothing > but some tests will fail and we need a manual analysis of that - which > is not impossible if the goal is to release say every 3 months. Instead of coming up with a gigantic approach that will not be used or only once every two years, I think lets deal with it once we're there. Something simple like an exclusion list activated by the same mechanism that switches to snapshot dependencies. Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Launchpad stable and launchpad unstable
On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler wrote: > Bertrand Delacretaz wrote >> On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler >> wrote: >>> ...what problem are we exactly trying to solve? >> ...As described in the original post here - having a stable launchpad >> that's always ready to release, while also continuously testing the >> integration of the latest snapshots. >> > So if the integration tests always test the latest snapshots then there > is nothing we have do to, right?... We need to test the launchpad that we release, don't we? Depending on the combination of bundles that release might have different behavior than combining the latest snapshots, although maybe it's not that likely. As a first step we might manually run the integration tests against the launchpad that's about to be released, that's better than nothing but some tests will fail and we need a manual analysis of that - which is not impossible if the goal is to release say every 3 months. -Bertrand
Re: Launchpad stable and launchpad unstable
Bertrand Delacretaz wrote > On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler wrote: >> ...what problem are we exactly trying to solve? > > As described in the original post here - having a stable launchpad > that's always ready to release, while also continuously testing the > integration of the latest snapshots. > So if the integration tests always test the latest snapshots then there is nothing we have do to, right? Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Launchpad stable and launchpad unstable
On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler wrote: > ...what problem are we exactly trying to solve? As described in the original post here - having a stable launchpad that's always ready to release, while also continuously testing the integration of the latest snapshots. -Bertrand
Re: Launchpad stable and launchpad unstable
Maybe I'm missing something, but what problem are we exactly trying to solve? Carsten Robert Munteanu wrote > On Mon, 2016-10-31 at 11:17 +0100, Carsten Ziegeler wrote: >> Bertrand Delacretaz wrote >>> On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu >> g> wrote: ...How would we handle launchpad tests that rely on functionality introduced in a SNAPSHOT dependency?... >>> >>> I suppose we'll need an annotation to indicate the requirements for >>> a >>> specific test. >>> >> >> Isn't that a little bit complicated? Who removes the annotation when >> things are released? > > How about an annotation or an assumption mechanism which looks for a > specific bundle + version to be present? That would not need further > maintenance and can be removed at any time, should we want to remove > it: > > @NeedsBundle("org.apache.sling.commons.json", "2.1.0") > @Test > public void validateSomething() { ... } > > or > > @Test > public void validateSomething() { > assumeThat(bundleIsPresent("org.apache.sling.commons.json", > "2.1.0")); > ... > } > >> >> In general, can we come up with a full proposal instead of discussing >> individual single points? I suggest we setup a wiki where we outline >> the >> full process and then can fill in the details > > I can come up with a full proposal but there are some things which are > not totally clear to me, so that's why I'm asking :-) > > Robert > -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Launchpad stable and launchpad unstable
On Tue, Nov 1, 2016 at 10:08 AM, Robert Munteanu wrote: ... > @NeedsBundle("org.apache.sling.commons.json", "2.1.0") ... > assumeThat(bundleIsPresent("org.apache.sling.commons.json", ... I like the idea, maybe with names like @NeedsBundleVersion and bundleVersionPresent(...) to be more explicit. Maybe we need something similar for package versions, but the mechanism will be similar. -Bertrand
Re: Launchpad stable and launchpad unstable
On Mon, 2016-10-31 at 11:17 +0100, Carsten Ziegeler wrote: > Bertrand Delacretaz wrote > > On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu > g> wrote: > > > ...How would we handle launchpad tests that rely on functionality > > > introduced in a SNAPSHOT dependency?... > > > > I suppose we'll need an annotation to indicate the requirements for > > a > > specific test. > > > > Isn't that a little bit complicated? Who removes the annotation when > things are released? How about an annotation or an assumption mechanism which looks for a specific bundle + version to be present? That would not need further maintenance and can be removed at any time, should we want to remove it: @NeedsBundle("org.apache.sling.commons.json", "2.1.0") @Test public void validateSomething() { ... } or @Test public void validateSomething() { assumeThat(bundleIsPresent("org.apache.sling.commons.json", "2.1.0")); ... } > > In general, can we come up with a full proposal instead of discussing > individual single points? I suggest we setup a wiki where we outline > the > full process and then can fill in the details I can come up with a full proposal but there are some things which are not totally clear to me, so that's why I'm asking :-) Robert
Re: Launchpad stable and launchpad unstable
Bertrand Delacretaz wrote > On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu wrote: >> ...How would we handle launchpad tests that rely on functionality >> introduced in a SNAPSHOT dependency?... > > I suppose we'll need an annotation to indicate the requirements for a > specific test. > Isn't that a little bit complicated? Who removes the annotation when things are released? In general, can we come up with a full proposal instead of discussing individual single points? I suggest we setup a wiki where we outline the full process and then can fill in the details Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Launchpad stable and launchpad unstable
On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu wrote: > ...How would we handle launchpad tests that rely on functionality > introduced in a SNAPSHOT dependency?... I suppose we'll need an annotation to indicate the requirements for a specific test. -Bertrand
Re: Launchpad stable and launchpad unstable
On Fri, 2016-10-14 at 15:38 +0200, Bertrand Delacretaz wrote: > I've committed an experimental script [1] which modifies the > launchpad/builder provisioning model files to remove the version > numbers for all Sling artifacts. See comments in the script for > details (and enjoy the sed regexp ;-) > > When doing so, the Slingstart plugin uses the latest snapshot instead > of a specified version, IIUC. > > The script doesn't fully work yet (see comments in the script - > patches or hints welcome) but once it does the process for running > our > integration tests on this all-snapshots launchpad would be: For the record, it doesn't work for me either, but misses more SNAPSHOT versions: $ mvn dependency:resolve | grep org.apache.sling | grep -v SNAPSHOT OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 [INFO] --- maven-dependency-plugin:2.10:resolve (default-cli) @ org.apache.sling.launchpad --- [INFO]org.apache.sling:org.apache.sling.servlets.resolver:jar:2.4.6 :provided [INFO]org.apache.sling:org.apache.sling.launchpad.base:war:webapp:5 .6.0-2.6.12:provided [INFO]org.apache.sling:org.apache.sling.commons.scheduler:jar:2.5.2 :provided [INFO]org.apache.sling:org.apache.sling.commons.log.webconsole:jar: 1.0.0:provided [INFO]org.apache.sling:org.apache.sling.commons.log:jar:5.0.0:provi ded [INFO]org.apache.sling:org.apache.sling.resourceresolver:jar:1.5.0: provided [INFO]org.apache.sling:org.apache.sling.launchpad.base:jar:5.6.0- 2.6.12:provided [INFO]org.apache.sling:org.apache.sling.launchpad.base:jar:app:5.6. 0-2.6.12:provided [INFO]org.apache.sling:org.apache.sling.event.dea:jar:1.1.0:provide d [INFO]org.apache.sling:org.apache.sling.api:jar:2.15.0:provided [INFO]org.apache.sling:org.apache.sling.installer.core:jar:3.7.0:pr ovided Robert
Re: Launchpad stable and launchpad unstable
On Fri, 2016-10-14 at 15:45 +0200, Carsten Ziegeler wrote: > Bertrand Delacretaz wrote > > On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu > > wrote: > > > ...If we make building an unstable launchpad really simple, e.g. > > > an addition > > > profile or a mojo property, I don't think we lose anything... > > > > I've committed an experimental script [1] which modifies the > > launchpad/builder provisioning model files to remove the version > > numbers for all Sling artifacts. See comments in the script for > > details (and enjoy the sed regexp ;-) > > > > When doing so, the Slingstart plugin uses the latest snapshot > > instead > > of a specified version, IIUC. > > > > The script doesn't fully work yet (see comments in the script - > > patches or hints welcome) but once it does the process for running > > our > > integration tests on this all-snapshots launchpad would be: > > > > 1) Run this script in launchpad/builder, build and deploy that, > > with a > > specific classifier or other marker > > 2) Run the launchpad/testing build using that special launchpad jar > > 3) Revert the code changes caused by step 1), unless working in a > > throwaway folder > > > > Would that work for you guys? > > > > Sorry I already replied to your commit as I didn't see this mail :( > > I think a script is fine and thanks for taking it up. > My initial thoughts were to do this in the slingstart maven plugin > This would give a little bit more control - if required, like > enabling > snapshots only for a single feature or something. Or having a > blacklist. > > But maybe that's not needed anyways How would we handle launchpad tests that rely on functionality introduced in a SNAPSHOT dependency? These would fail with the stable launchpad. We could make then run conditionally just with the unstable launchpad but OTOH we need to remember to remove that check after the bundle which provides the new functionality ( or the bug fix ) is released and included in the launchpad. Robert
Re: Launchpad stable and launchpad unstable
Bertrand Delacretaz wrote > On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu wrote: >> ...If we make building an unstable launchpad really simple, e.g. an addition >> profile or a mojo property, I don't think we lose anything... > > I've committed an experimental script [1] which modifies the > launchpad/builder provisioning model files to remove the version > numbers for all Sling artifacts. See comments in the script for > details (and enjoy the sed regexp ;-) > > When doing so, the Slingstart plugin uses the latest snapshot instead > of a specified version, IIUC. > > The script doesn't fully work yet (see comments in the script - > patches or hints welcome) but once it does the process for running our > integration tests on this all-snapshots launchpad would be: > > 1) Run this script in launchpad/builder, build and deploy that, with a > specific classifier or other marker > 2) Run the launchpad/testing build using that special launchpad jar > 3) Revert the code changes caused by step 1), unless working in a > throwaway folder > > Would that work for you guys? > Sorry I already replied to your commit as I didn't see this mail :( I think a script is fine and thanks for taking it up. My initial thoughts were to do this in the slingstart maven plugin This would give a little bit more control - if required, like enabling snapshots only for a single feature or something. Or having a blacklist. But maybe that's not needed anyways Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Launchpad stable and launchpad unstable
On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu wrote: >...If we make building an unstable launchpad really simple, e.g. an addition > profile or a mojo property, I don't think we lose anything... I've committed an experimental script [1] which modifies the launchpad/builder provisioning model files to remove the version numbers for all Sling artifacts. See comments in the script for details (and enjoy the sed regexp ;-) When doing so, the Slingstart plugin uses the latest snapshot instead of a specified version, IIUC. The script doesn't fully work yet (see comments in the script - patches or hints welcome) but once it does the process for running our integration tests on this all-snapshots launchpad would be: 1) Run this script in launchpad/builder, build and deploy that, with a specific classifier or other marker 2) Run the launchpad/testing build using that special launchpad jar 3) Revert the code changes caused by step 1), unless working in a throwaway folder Would that work for you guys? -Bertrand (BTW the resulting launchpad with all snapshots doesn't start, might be a good indicator that we need this! [1] https://svn.apache.org/repos/asf/sling/trunk/launchpad/builder/set-sling-snapshots.sh
Re: Launchpad stable and launchpad unstable
On Wed, 2016-09-28 at 22:06 +, Stefan Seifert wrote: > discussed at the Sling Committer Round Table @ adaptTo() 2016 > > currently we have only one launchpad in trunk, which contains a > mixture of release dependencies and snapshot dependencies. this makes > it difficult to release new versions because one has to release all > snapshot projects first, and perhaps the last relese version is fine > as well. > > we came up with a new proposal to fulfill the different needs: > > - the launchpad maintained in trunk should always be the "stable" > launchpad, referencing only released dependencies of the sling > bundles and oft the integration tests project > > - the "unstable" launchpad is created dynamically from this stable > launchpad by using the same bundle list, but for each dependency the > latest snapshot version that is available in the maven repo, > including the latest snapshot version oft he integration tests. > perhaps the slingstart plugin could be extented to generate this. > > - i'm not sure what to do with new bundles which are not released yet > thus are not referenced in the stable launchpad. I'm +1 on this change, it makes it much simpler to perform releases. If we make building an unstable launchpad really simple, e.g. an addition profile or a mojo property, I don't think we lose anything. Robert