Chris, Zeljko, and I actually discussed some of these finer points earlier today. See my answers inline.
* I guess it's a very early stage proposal, but I'd come up with a tag > naming convention that clearly indicates what those tags do, like > @role-echo (which marries the whole thing to Vagrant and its > nomenclature) or @extension-echo. > A clearer naming convention is definitely a good idea. Not only will it avoid name collisions but I think it will promote a greater inference of what the tags mean—people will be less likely to copy, paste, ignore. Consider this might have applications beyond the scope of mw-vagrant, I'd prefer "@extension-" over "@role-". > > * How would puppet alter default cucumber behavior? > We can configure puppet to write a default cucumber config <https://github.com/cucumber/cucumber/wiki/cucumber.yml> in the working directory for each test suite. The config will specify a default command-line invocation with `--tags` options to limit the scope of the run. It's possible that we'll need some sort of "@core" tag for features that don't have any extension dependency, but there may be another way—I'll have to experiment a bit. > > * If we don't want to confuse people why some tests don't run, maybe > we could store the list of available tags, check for available roles > before running cucumber and show a message saying that tests requiring > following (disabled) roles, will not run? > Zeljko brought this up as well. We can probably implement some sort of after or exit hook to inform the user why some tests were skipped—based on the invocation maybe—and output it in yellow or some other alarming but not shit-your-pants alarming color. :) > > -- > Juliusz > > On Tue, Jul 29, 2014 at 5:17 PM, Dan Duvall <[email protected]> wrote: > > There is another alternative that might be worth getting opinions > on—Juliusz > > and I both thought it seemed a bit complicated, but it would have some > > additional benefits, so here goes. > > > > Firstly, we'd add cucumber tags to indicate the additional role > dependencies > > of a given feature (or individual scenario). For instance, > editor_ve.feature > > would include a `@visualeditor` tag and notification.feature would > include > > an `@echo` tag. Secondly, we'd restrict the default behavior of cucumber > > (via puppet) to run only those tests that are tagged with any of the > > currently enabled roles. > > > > The major upside to this approach would be that only those tests suited > to > > the current development environment get run—ostensibly these are the only > > tests the developer would care to run—translating to far fewer false > > negatives and more confidence (and utility) in the test suite. With the > > dual-roles approach, it's highly likely that developers will continue to > run > > into these soft-dependency issues. And, as already stated, a single-role > > approach is less flexible and will bloat the provision and git-update > times. > > > > The biggest downside to this approach is pretty obvious, I think: It > would > > require the author of the test to know and include any role dependencies > in > > the list of tags. Is this a reasonable expectation of developers and/or > > project managers (who might one day be writing features)? Another > possible > > downside is that altering cucumber's default behavior might confuse > people > > as to why some tests are being skipped. > > > > > > On Tue, Jul 22, 2014 at 10:56 AM, Arthur Richards < > [email protected]> > > wrote: > >> > >> I agree with Max - or alternatively (or in addition) setting up a > >> mobilefrontend-production role to set things up as close as possible to > >> prod. > >> > >> > >> On Fri, Jul 18, 2014 at 9:22 PM, Max Semenik <[email protected]> > >> wrote: > >>> > >>> I personally would prefer 2, because buncing everything into one role > >>> would be too inflexible. I don't see much maintenance overhead if > compared > >>> with 1. > >>> > >>> > >>> On Fri, Jul 18, 2014 at 6:19 PM, Juliusz Gonera <[email protected] > > > >>> wrote: > >>>> > >>>> I talked to Dan Duvall today and he said that many mobile browser > >>>> tests fail on default vagrant instance because other extensions that > >>>> are not hard dependencies of MF are not activated by default when > >>>> activating the mobilefrontend role (extensions such as Echo, GeoData, > >>>> VisualEditor). There are two things we can do: > >>>> > >>>> 1. Make other extensions that mobile uses dependencies of > >>>> mobilefrontend role in vagrant > >>>> 2. Create a separate role, e.g. mobilefrontend-browsertests, that will > >>>> list mobilefrontend and all those extensions as its dependencies > >>>> > >>>> I was wondering if there is anyone who uses vagrant and would like to > >>>> have MF enabled, but not all the other extensions. If not, 1. seems > >>>> like a simpler solution. > >>>> > >>>> Thoughts? > >>>> > >>>> -- > >>>> Juliusz > >>> > >>> > >> > >> > >> > >> -- > >> Arthur Richards > >> Team Practices Lead > >> [[User:Awjrichards]] > >> IRC: awjr > >> +1-415-839-6885 x6687 > > > > > > > > > > -- > > Dan Duvall > > Automation Engineer > > Wikimedia Foundation > -- Dan Duvall Automation Engineer Wikimedia Foundation <http://wikimediafoundation.org>
_______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
