Re: [android-developers] Re: Replay monkey execution
interesting to hear that it's being used to such a degree. It does give me some comfort just to have something bang at the app in case it hits a silly thing I might not have caught. btw, the monkey seems to be particularly fascinated with the ringer volume, for some reason. At least when I run it. On Fri, May 13, 2011 at 4:21 PM, Dianne Hackborn hack...@android.com wrote: The main thing we use monkey for is automated testing for statistical tracking of stability and performance. That is, for a particular build there is a lab that runs multiple monkeys on the build and collects statistics on how many events they were able to perform before failing and what kinds of failures they had. This is aggregated together to get the overall stability of the platform and the most common kinds of failures that are impacting stability. Performing continuous monkey runs allows this information to be aggregated over time to detect if problems are being introduced or stability is being improved. For application unit test, the instrumentation facility is the intended mechanism. This allows you to have much more control within your own application, and because it is running within that app you can avoid much of the impact of other things going on across the rest of the system. On Fri, May 13, 2011 at 8:08 PM, B Lyon bradfl...@gmail.com wrote: When I first read about the monkey runner, I thought I could use it for all kinds of stuff, but after learning a little more I'm not sure you would want to use it for controlled tests (like regression checks, functional tests). Maybe great to throw a bunch of random stuff at your app (and it's fun to watch it go), and it might do something that hits a bug you might never have found cause of the bizarre sequence it went through, but not sure about how much you can do with it. I kind of hope I'm wrong, so someone correct me if I am not giving the monkey its due. On Fri, May 13, 2011 at 4:01 PM, Dianne Hackborn hack...@android.com wrote: Again, monkeys are NOT deterministic. On Fri, May 13, 2011 at 7:52 PM, avi avinanku...@gmail.com wrote: Hi Diego, Thank you for your post..I tried the exact same steps..Running monkey with verbose option and post processed the file to replay Monkey...but there was slight deviation from the original run..If you have tried it, could you please share your solution with me? Thanks again On May 8, 10:52 pm, Diego Torres Milano dtmil...@gmail.com wrote: I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ?http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as
[android-developers] Re: Replay monkey execution
Hi Diego, Thank you for your post..I tried the exact same steps..Running monkey with verbose option and post processed the file to replay Monkey...but there was slight deviation from the original run..If you have tried it, could you please share your solution with me? Thanks again On May 8, 10:52 pm, Diego Torres Milano dtmil...@gmail.com wrote: I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ?http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
Again, monkeys are NOT deterministic. On Fri, May 13, 2011 at 7:52 PM, avi avinanku...@gmail.com wrote: Hi Diego, Thank you for your post..I tried the exact same steps..Running monkey with verbose option and post processed the file to replay Monkey...but there was slight deviation from the original run..If you have tried it, could you please share your solution with me? Thanks again On May 8, 10:52 pm, Diego Torres Milano dtmil...@gmail.com wrote: I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ?http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
When I first read about the monkey runner, I thought I could use it for all kinds of stuff, but after learning a little more I'm not sure you would want to use it for controlled tests (like regression checks, functional tests). Maybe great to throw a bunch of random stuff at your app (and it's fun to watch it go), and it might do something that hits a bug you might never have found cause of the bizarre sequence it went through, but not sure about how much you can do with it. I kind of hope I'm wrong, so someone correct me if I am not giving the monkey its due. On Fri, May 13, 2011 at 4:01 PM, Dianne Hackborn hack...@android.com wrote: Again, monkeys are NOT deterministic. On Fri, May 13, 2011 at 7:52 PM, avi avinanku...@gmail.com wrote: Hi Diego, Thank you for your post..I tried the exact same steps..Running monkey with verbose option and post processed the file to replay Monkey...but there was slight deviation from the original run..If you have tried it, could you please share your solution with me? Thanks again On May 8, 10:52 pm, Diego Torres Milano dtmil...@gmail.com wrote: I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ?http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
The main thing we use monkey for is automated testing for statistical tracking of stability and performance. That is, for a particular build there is a lab that runs multiple monkeys on the build and collects statistics on how many events they were able to perform before failing and what kinds of failures they had. This is aggregated together to get the overall stability of the platform and the most common kinds of failures that are impacting stability. Performing continuous monkey runs allows this information to be aggregated over time to detect if problems are being introduced or stability is being improved. For application unit test, the instrumentation facility is the intended mechanism. This allows you to have much more control within your own application, and because it is running within that app you can avoid much of the impact of other things going on across the rest of the system. On Fri, May 13, 2011 at 8:08 PM, B Lyon bradfl...@gmail.com wrote: When I first read about the monkey runner, I thought I could use it for all kinds of stuff, but after learning a little more I'm not sure you would want to use it for controlled tests (like regression checks, functional tests). Maybe great to throw a bunch of random stuff at your app (and it's fun to watch it go), and it might do something that hits a bug you might never have found cause of the bizarre sequence it went through, but not sure about how much you can do with it. I kind of hope I'm wrong, so someone correct me if I am not giving the monkey its due. On Fri, May 13, 2011 at 4:01 PM, Dianne Hackborn hack...@android.com wrote: Again, monkeys are NOT deterministic. On Fri, May 13, 2011 at 7:52 PM, avi avinanku...@gmail.com wrote: Hi Diego, Thank you for your post..I tried the exact same steps..Running monkey with verbose option and post processed the file to replay Monkey...but there was slight deviation from the original run..If you have tried it, could you please share your solution with me? Thanks again On May 8, 10:52 pm, Diego Torres Milano dtmil...@gmail.com wrote: I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ?http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email
[android-developers] Re: Replay monkey execution
I haven't tried this but it seems possible. Step #1: Increase monkey verbosity (-v -v -v) Step #2: Run the script and collect the output Step #3: Parse the output to regenerate a monkey script (optionally you can modify monkey to spit a valid monkey script) Step #4: Run the script On May 7, 2:20 pm, Bob Kerns r...@acm.org wrote: True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. -- Have you read my blog ? http://dtmilano.blogspot.com android junit tests ui linux cult thin clients -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
True enough, but the difficulty generally comes between step #1 and step #2, in this sort of scenario of a rare, hard-to-reproduce crash. Often you have to figure out everything you need to know to do step #3 (fix the bug) before you can do step #2. Even so, do it in this order. Write the test case before fixing it, so you can do step #4 -- verify that you fixed it. On Friday, May 6, 2011 10:49:45 PM UTC-7, Mark Murphy (a Commons Guy) wrote: On Fri, May 6, 2011 at 9:43 PM, avi avina...@gmail.com wrote: Thank you Dianne for your response. So, what is Google's recommended process for fixing issues caused by monkey runs? Step #1: Use the monkey to generate a crash Step #2: Write the test case that reproduces the crash Step #3: Fix the bug Step #4: Ensure that the test case you wrote in step #2 now passes -- Mark Murphy (a Commons Guy) http://commonsware.com | http://github.com/commonsguy http://commonsware.com/blog | http://twitter.com/commonsguy _Android Programming Tutorials_ Version 3.4 Available! -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
[android-developers] Re: Replay monkey execution
You are experiencing different crashes because you have fixed the bugs causing the other ones. Right? -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
[android-developers] Re: Replay monkey execution
Sorry, I meant I am getting different crashes on the same build (without any fix)...The crash that I am looking at happens only 1 out of 10 times and it happens after running Monkey for several hours (~8 hours)..So, I would like to replay the monkey run which caused the crash and check if my fix works. On May 6, 4:41 am, lbendlin l...@bendlin.us wrote: You are experiencing different crashes because you have fixed the bugs causing the other ones. Right? -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
If you give the same seed, you will have the same sequence of events executed. However it is pretty unavoidable that the results will be non-deterministic. There is just too much asynchronous stuff, things running in the background that can impact behavior, etc. On Fri, May 6, 2011 at 2:57 PM, avi avinanku...@gmail.com wrote: Sorry, I meant I am getting different crashes on the same build (without any fix)...The crash that I am looking at happens only 1 out of 10 times and it happens after running Monkey for several hours (~8 hours)..So, I would like to replay the monkey run which caused the crash and check if my fix works. On May 6, 4:41 am, lbendlin l...@bendlin.us wrote: You are experiencing different crashes because you have fixed the bugs causing the other ones. Right? -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
[android-developers] Re: Replay monkey execution
Thank you Dianne for your response. So, what is Google's recommended process for fixing issues caused by monkey runs? On May 6, 4:24 pm, Dianne Hackborn hack...@android.com wrote: If you give the same seed, you will have the same sequence of events executed. However it is pretty unavoidable that the results will be non-deterministic. There is just too much asynchronous stuff, things running in the background that can impact behavior, etc. On Fri, May 6, 2011 at 2:57 PM, avi avinanku...@gmail.com wrote: Sorry, I meant I am getting different crashes on the same build (without any fix)...The crash that I am looking at happens only 1 out of 10 times and it happens after running Monkey for several hours (~8 hours)..So, I would like to replay the monkey run which caused the crash and check if my fix works. On May 6, 4:41 am, lbendlin l...@bendlin.us wrote: You are experiencing different crashes because you have fixed the bugs causing the other ones. Right? -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en -- Dianne Hackborn Android framework engineer hack...@android.com Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them. -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
Re: [android-developers] Re: Replay monkey execution
On Fri, May 6, 2011 at 9:43 PM, avi avinanku...@gmail.com wrote: Thank you Dianne for your response. So, what is Google's recommended process for fixing issues caused by monkey runs? Step #1: Use the monkey to generate a crash Step #2: Write the test case that reproduces the crash Step #3: Fix the bug Step #4: Ensure that the test case you wrote in step #2 now passes -- Mark Murphy (a Commons Guy) http://commonsware.com | http://github.com/commonsguy http://commonsware.com/blog | http://twitter.com/commonsguy _Android Programming Tutorials_ Version 3.4 Available! -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
