I just stumbled on this problem and was wondering if there is a solution now.
It would be great to be able to specify the default annotions (@small, @medium, @large) and/or be able to define some of our own (e.g. @Functional, @Acceptation). It would also be very interesting to have options to include or exclude some annotated tests from a test run which would be configurable in a task. (e.g. run everything but classes having @Large). On Tuesday, August 13, 2013 1:58:09 PM UTC-4, Seth Goldenberg wrote: > > That would actually be great for us. I see that it's available on the > existing instrumentation test runner ( > http://developer.android.com/reference/android/test/InstrumentationTestRunner.html), > > even though there isn't support in the build tools yet. Do you have any > idea where annotation support would fall on the roadmap? > > On Friday, August 9, 2013 7:48:25 AM UTC-7, Xavier Ducrohet wrote: >> >> Technically you can annotate tests with @Small/Medium/Large and then only >> run certain types. >> >> This is not supported yet, but this is something we want to provide. >> Write all your test in the same place (if they are all run the same way >> through the instrumentation runner). Annotate them differently, and then >> create different run configs and run them separately. >> >> We also want to expand the current annotations. >> >> >> >> >> On Fri, Aug 9, 2013 at 6:59 AM, Seth Goldenberg <seth.go...@gmail.com> >> wrote: >> >>> Thanks for the response, Xavier. That is a lot of factors to consider. >>> Our use case is much simpler. We have a set of Robotium automation tests >>> that run like instrumentation tests. We'd just like to keep them separate >>> because 1) they're slow to run and we want to keep our continuous build >>> times short and 2) our QA engineer who writes and maintains the tests likes >>> to use a separate manifest files where he can fiddle with permissions (this >>> one is less important). >>> >>> We used to keep the automation separate in Maven by having a separate >>> project that was included when building with a given profile (ex. mvn clean >>> install -P automation). Given that the automation tests don't need any >>> extra dependencies besides Robotium, I was thinking of writing a workaround >>> that copied the automation test classes to "instrumentTest" folder, ran the >>> tests, and then deleted the classes. I imagine that other developers' use >>> cases won't be as simple though. >>> >>> On Wednesday, August 7, 2013 8:20:52 PM UTC-4, Xavier Ducrohet wrote: >>> >>>> Actually we were having an internal talk about allowing different type >>>> of tests and giving control about how to run them. >>>> >>>> Can you tell me how your automation test are setup compared to your >>>> instrumentation test? >>>> >>>> >>>> On Wed, Aug 7, 2013 at 5:20 PM, Xavier Ducrohet <x...@android.com> >>>> wrote: >>>> >>>>> The issue is that this doesn't support build variants. >>>>> >>>>> It would be interesting to have a helper of sort to create tasks for >>>>> all variants, but when you start running into setting source folders and >>>>> dependency configuration that's a bit complicated. >>>>> >>>>> I think it is useful though to allow more than one type of tests, so >>>>> something specific to that might work. >>>>> >>>>> Something like: >>>>> >>>>> android.createTestTask(baseName, Closure) >>>>> >>>>> >>>>> would create, for each variant: >>>>> >>>>> - a task called "${baseName}${variantName}" or some other >>>>> combination of both. >>>>> - a dependency configuration object called >>>>> "${variantName}${baseName}" >>>>> - a new source set object called "${baseName}${variantName}" in >>>>> the android.sourceSets container >>>>> >>>>> Your closure would do the work. We might also want to automatically >>>>> generate a compile task for it, so that it's separate from the task >>>>> running >>>>> the actual test... >>>>> >>>>> There are other things to figure out, like the actual classpath beyond >>>>> the specific Configuration object (does it extend the app classpath? >>>>> probably, but there might be options). >>>>> >>>>> After that you're probably on your own to run it, but if you want to >>>>> run it on the device, you're going to want to have access to whatever we >>>>> do >>>>> for the extension and it becomes non simple. >>>>> >>>>> There's a lot of details to figure out. >>>>> >>>>> >>>>> >>>>> On Wed, Aug 7, 2013 at 11:35 AM, Seth Goldenberg <seth.go...@gmail.com >>>>> > wrote: >>>>> >>>>>> Right now, it only seems possible to run one set of instrumentation >>>>>> tests for a build. I was hoping to add a new test task to run automation >>>>>> tests in addition to the instrumentation tests. That seems like a tall >>>>>> order given the current state of the build tools. I'd like to propose >>>>>> two >>>>>> options. >>>>>> >>>>>> 1) *The ideal option* - Devs can create a new test task that can >>>>>> test at least one build flavor independent of the supported >>>>>> "instrumentTest" directory. In your build.gradle file, you would define >>>>>> a >>>>>> test task with a name (ex. "automationTest") that points to a directory >>>>>> of >>>>>> the same name in the src folder. These tests would only be run when a >>>>>> given >>>>>> Gradle task is called, separately from the "connnectedDeviceCheck" but >>>>>> maybe folded into the "check" task. >>>>>> 2) *The less-than-ideal option* - The user can create a new task >>>>>> that runs an existing task (probably "connectedDeviceCheck") but pulls >>>>>> in >>>>>> extra classes from another directory before running the tests. This >>>>>> could >>>>>> probably be done with the existing tools. >>>>>> >>>>>> I'm open to other workarounds, but these are the ideas I've been >>>>>> considering. >>>>>> >>>>>> I brought this up on the ADT Google+ page. >>>>>> https://plus.google.com/114026192098014497501/posts/bokGD6aN66n >>>>>> >>>>>> Thanks, >>>>>> Seth >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "adt-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to adt-dev+u...@googlegroups.com. >>>>>> >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Xavier Ducrohet >>>>> Android SDK Tech Lead >>>>> Google Inc. >>>>> http://developer.android.com | http://tools.android.com >>>>> >>>>> Please do not send me questions directly. Thanks! >>>>> >>>> >>>> >>>> >>>> -- >>>> Xavier Ducrohet >>>> Android SDK Tech Lead >>>> Google Inc. >>>> http://developer.android.com | http://tools.android.com >>>> >>>> Please do not send me questions directly. Thanks! >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "adt-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to adt-dev+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> >> >> >> -- >> Xavier Ducrohet >> Android SDK Tech Lead >> Google Inc. >> http://developer.android.com | http://tools.android.com >> >> Please do not send me questions directly. Thanks! >> > -- You received this message because you are subscribed to the Google Groups "adt-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to adt-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
