[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
I just came upon this by accident http://officialandroid.blogspot.co.il/2012/09/the-benefits-importance-of-compatibility.html This seems like the right approach, but my own experience is that the Android reality is very far from this ideal. I've heard about the CTS. The question is - are vendors actually forced to pass the CTS with their customizations? On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Yes. To install Google Play or Google apps, you absolutely are required to pass the CTS. But, from your discussion, the CTS obviously doesn't test all parts of the Android platform: it's just a test suite. Kris On Wed, Jul 31, 2013 at 11:06 AM, Omer Gilad omer.gi...@gmail.com wrote: I just came upon this by accident http://officialandroid.blogspot.co.il/2012/09/the-benefits-importance-of-compatibility.html This seems like the right approach, but my own experience is that the Android reality is very far from this ideal. I've heard about the CTS. The question is - are vendors actually forced to pass the CTS with their customizations? On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
On 07/31/2013 05:17 PM, Kristopher Micinski wrote: Yes. To install Google Play or Google apps, you absolutely are required to pass the CTS. But, from your discussion, the CTS obviously doesn't test all parts of the Android platform: it's just a test suite. Yes, and never will. It may be a good idea to contact them and ask if they can help in setting up a bug database for incompatibilities between different android versions. No idea on how to contact them. -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Just because a test suite doesn't cover all possible behavior doesn't mean stricter enforcement wouldn't help developers. Test suites don't cover all of the codebase by definition, that's why they're called test suites :-) So I'm in favor of setting up a tracker for device bugs, but that doesn't also mean that stricter enforcement of bugs couldn't aid developers. E.g., to mention one of Omer's points: there could have been a CTS test to test the gallery app on that intent. (I believe other intents are tested...) Kris On Wed, Jul 31, 2013 at 12:46 PM, Daniele Segato daniele.seg...@gmail.com wrote: On 07/31/2013 05:17 PM, Kristopher Micinski wrote: Yes. To install Google Play or Google apps, you absolutely are required to pass the CTS. But, from your discussion, the CTS obviously doesn't test all parts of the Android platform: it's just a test suite. Yes, and never will. It may be a good idea to contact them and ask if they can help in setting up a bug database for incompatibilities between different android versions. No idea on how to contact them. -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Since pretty much everyone agrees that the CTS misses things... Wouldn't it be worthwhile to not only track device specific issues, but to also add tests to a fork of CTS? Google might then accept those enhancement, or might not, it would still be helpful in any case. -- K On Wednesday, July 31, 2013 9:11:24 PM UTC+4, Kristopher Micinski wrote: Just because a test suite doesn't cover all possible behavior doesn't mean stricter enforcement wouldn't help developers. Test suites don't cover all of the codebase by definition, that's why they're called test suites :-) So I'm in favor of setting up a tracker for device bugs, but that doesn't also mean that stricter enforcement of bugs couldn't aid developers. E.g., to mention one of Omer's points: there could have been a CTS test to test the gallery app on that intent. (I believe other intents are tested...) Kris On Wed, Jul 31, 2013 at 12:46 PM, Daniele Segato daniele...@gmail.com javascript: wrote: On 07/31/2013 05:17 PM, Kristopher Micinski wrote: Yes. To install Google Play or Google apps, you absolutely are required to pass the CTS. But, from your discussion, the CTS obviously doesn't test all parts of the Android platform: it's just a test suite. Yes, and never will. It may be a good idea to contact them and ask if they can help in setting up a bug database for incompatibilities between different android versions. No idea on how to contact them. -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Hey Mike I still didn't get answers to my questions about this service... What are the remote devices that the code is running on? Do I need to instruct a user to install this on his own device? On Monday, July 29, 2013 8:56:12 PM UTC+3, Mike wrote: Use my App, you can run code on remote device, not just read logs: https://cloudshellapp.appspot.com/ https://play.google.com/store/apps/details?id=com.cloudshell The App will allow you to run code on remote Android phones, so you can fix the bug on the phone you do not have physical access to. On Monday, July 29, 2013 2:57:18 AM UTC-4, Piren wrote: Well, after i started encountering such issues i just LOADED my app with debugging information, like, seriously redonkulous amounts of logging. There's nothing the app didn't log, sorted with tags and extra information to say exactly what is doing on and why. It was easier to fix bugs of even things i did not have in my hand... heck, most of the bugs i solved were of devices i never even held. (all the logs were surrounded by a constant variable that was set during compilation, so that code never made it to release versions, it makes it easier to manage app versions). I guess that will not be as easy to do with a game, but it should still be better than what you have now. You can make it a feature of the app, let the user community engage with you (the developer) and actively participate in the creation of the game and what not... you'll solve bugs, they'll feel as a contributer and get their apps running better. If your game is a paid game, make sure the debug versions are limited in capability and will only be good for debugging. On Monday, July 29, 2013 12:29:25 AM UTC+3, Omer Gilad wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Well, after i started encountering such issues i just LOADED my app with debugging information, like, seriously redonkulous amounts of logging. There's nothing the app didn't log, sorted with tags and extra information to say exactly what is doing on and why. It was easier to fix bugs of even things i did not have in my hand... heck, most of the bugs i solved were of devices i never even held. (all the logs were surrounded by a constant variable that was set during compilation, so that code never made it to release versions, it makes it easier to manage app versions). I guess that will not be as easy to do with a game, but it should still be better than what you have now. You can make it a feature of the app, let the user community engage with you (the developer) and actively participate in the creation of the game and what not... you'll solve bugs, they'll feel as a contributer and get their apps running better. If your game is a paid game, make sure the debug versions are limited in capability and will only be good for debugging. On Monday, July 29, 2013 12:29:25 AM UTC+3, Omer Gilad wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en ---
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
.I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation There isn't any reliable way of dealing with fragmentation - devices and platform bugs has to be workaround (if possible) or device marked as unsupported, no silver bullet here. For games I'd say that best solution is to use high level game engine (unity, project anarchy) since they already incorporates tons of workarounds, uses really well tested features, and are used in QA (both OEMs and google). If you are insisting on rolling your own engine then you have to support it, and boy this is a pain in the ass. -- Bart -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
You can run most of the Android API, you can even hook up and listen to BroadCast events. There are few limitations, one is the callback, you can not use startIntentWithResult . User need to download the App from play store, and install, log in. On Sunday, July 28, 2013 5:30:57 PM UTC-4, Omer Gilad wrote: That looks innovative, I'll definitely dig into this... How versatile is this solution? Can you run only a select part of Android APIs, wrapped by your script? How is the distribution being done (like, where are all those devices that run the test code)? On Sunday, July 28, 2013 10:05:54 PM UTC+3, Mike wrote: I wrote an free App and web site to help myself on the Android fragmentation problem: https://cloudshellapp.appspot.com/ https://play.google.com/store/apps/details?id=com.cloudshell The App will allow you to run code on remote Android phones, so you can fix the bug on the phone you do not have physical access to. I asked my friend in cell phone store to test on the different Android phones, then I will remote read the log and test the code. On Thursday, July 25, 2013 6:39:14 PM UTC-4, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Use my App, you can run code on remote device, not just read logs: https://cloudshellapp.appspot.com/ https://play.google.com/store/apps/details?id=com.cloudshell The App will allow you to run code on remote Android phones, so you can fix the bug on the phone you do not have physical access to. On Monday, July 29, 2013 2:57:18 AM UTC-4, Piren wrote: Well, after i started encountering such issues i just LOADED my app with debugging information, like, seriously redonkulous amounts of logging. There's nothing the app didn't log, sorted with tags and extra information to say exactly what is doing on and why. It was easier to fix bugs of even things i did not have in my hand... heck, most of the bugs i solved were of devices i never even held. (all the logs were surrounded by a constant variable that was set during compilation, so that code never made it to release versions, it makes it easier to manage app versions). I guess that will not be as easy to do with a game, but it should still be better than what you have now. You can make it a feature of the app, let the user community engage with you (the developer) and actively participate in the creation of the game and what not... you'll solve bugs, they'll feel as a contributer and get their apps running better. If your game is a paid game, make sure the debug versions are limited in capability and will only be good for debugging. On Monday, July 29, 2013 12:29:25 AM UTC+3, Omer Gilad wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
I wrote an free App and web site to help myself on the Android fragmentation problem: https://cloudshellapp.appspot.com/ https://play.google.com/store/apps/details?id=com.cloudshell The App will allow you to run code on remote Android phones, so you can fix the bug on the phone you do not have physical access to. I asked my friend in cell phone store to test on the different Android phones, then I will remote read the log and test the code. On Thursday, July 25, 2013 6:39:14 PM UTC-4, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Does anyone have a workaround for one of the bigger problems of this mess: users will blame your app and write bad reviews? That sounds like a joke, but really, has anyone had success just telling users sorry, Samsung's fault :(? Would be a shame to lose sales because of the vendor's problems. On Thursday, July 25, 2013 3:39:14 PM UTC-7, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
I don't have one, can only share more pain :) Most users are very reluctant to accept explanations like that, because the devices are much more expensive than the typical Android app. It's just psychology: I paid $xxx for this, and it's 'top of the line', do you mean to tell me there is something wrong with it? The salesman said it's a 'business' phone. It has the latest Android!, etc. Google's magic, like I said before :) Everyone knows Windows is buggy (although I don't recall having any issues with it, for a long time), but Android is just perfect by definition. I also love but this other app -- in my case, it mostly has to do with buggy mail servers (IMAP, typically). An explanation that goes along the lines of the spec say the server must do this, and it's black English letters on a white background, no way to misread it, and it works with mail services X, Y, Z and a dozen others doesn't always fly either. -- K 2013/7/29 Thomas Jakway thomasjakw...@gmail.com Does anyone have a workaround for one of the bigger problems of this mess: users will blame your app and write bad reviews? That sounds like a joke, but really, has anyone had success just telling users sorry, Samsung's fault :(? Would be a shame to lose sales because of the vendor's problems. On Thursday, July 25, 2013 3:39:14 PM UTC-7, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
That looks innovative, I'll definitely dig into this... How versatile is this solution? Can you run only a select part of Android APIs, wrapped by your script? How is the distribution being done (like, where are all those devices that run the test code)? On Sunday, July 28, 2013 10:05:54 PM UTC+3, Mike wrote: I wrote an free App and web site to help myself on the Android fragmentation problem: https://cloudshellapp.appspot.com/ https://play.google.com/store/apps/details?id=com.cloudshell The App will allow you to run code on remote Android phones, so you can fix the bug on the phone you do not have physical access to. I asked my friend in cell phone store to test on the different Android phones, then I will remote read the log and test the code. On Thursday, July 25, 2013 6:39:14 PM UTC-4, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
The best workaround I know for that is to filter devices on Google Play console. Yes, you lose tons of customers, but at least you don't get 1 star reviews like IT SHOWS BLACK SCREEN ON MY CRAPSUNG S4 CRAP EDITION, DON'T DOWNLOAD On Sunday, July 28, 2013 11:51:32 PM UTC+3, Thomas Jakway wrote: Does anyone have a workaround for one of the bigger problems of this mess: users will blame your app and write bad reviews? That sounds like a joke, but really, has anyone had success just telling users sorry, Samsung's fault :(? Would be a shame to lose sales because of the vendor's problems. On Thursday, July 25, 2013 3:39:14 PM UTC-7, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to android-developers+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer.gi...@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm waiting for an official Google response about this, and what have you been doing in all those years to fix that. I've heard about things like conformance tests for devices and so on, but the reality is far from acceptable in this area. ,Looking forward for helpful responses Omer -- -- You received this message because you are subscribed to the Google Groups Android Developers group. To post to this group, send email to android-developers@googlegroups.com To unsubscribe from this group, send email to android-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-developers?hl=en --- You received this message because you are subscribed to the Google Groups Android Developers group. To unsubscribe from this group and stop receiving emails from it, send an email to
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
With all this talk I still haven't seen an example of a testing service that can actually help me... Got any links? Anything with a reasonable price that can ease the pain (and that you used for yourself)? BTW, I have worked in the past with some of those remote device providers (like http://www.keynotedeviceanywhere.com/). It's very expensive to get a few hours with a device, especially if this is something you have to do constantly with different devices each time there's an issue. And the deal-breaker part - it's so slow, you'll actually have to waste a whole hour just on installing your app and seeing LogCat. On Monday, July 29, 2013 12:48:16 AM UTC+3, Kristopher Micinski wrote: So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer@gmail.comjavascript: wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
I didn't posit that there were concrete things you could go out and buy right now, so apologies if I misphrased. I know a group who works on a research project which does basically this, but there's no word on whether it will be open source any time soon. I think the idea is what I just mentioned, collaborative crowd based testing services that allow you to test on a wide range and variety of devices with very expressive test scripts and possible end user testing when applicable. Kris On Sun, Jul 28, 2013 at 6:05 PM, Omer Gilad omer.gi...@gmail.com wrote: With all this talk I still haven't seen an example of a testing service that can actually help me... Got any links? Anything with a reasonable price that can ease the pain (and that you used for yourself)? BTW, I have worked in the past with some of those remote device providers (like http://www.keynotedeviceanywhere.com/). It's very expensive to get a few hours with a device, especially if this is something you have to do constantly with different devices each time there's an issue. And the deal-breaker part - it's so slow, you'll actually have to waste a whole hour just on installing your app and seeing LogCat. On Monday, July 29, 2013 12:48:16 AM UTC+3, Kristopher Micinski wrote: So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly.
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
A testing service still -- focuses only on specific device models, assuming deterministic failures, ignoring weird shit that can happen on seemingly any of them, even the good ones. Going back to my Youtube video of Google Maps causing a device reboot: Is the Galaxy Nexus a bad device? No. Wasn't Google Maps tested on it? I'm sure it was. And yet, there it was. PS - I'm not saying a testing service (commercial or co-op, test driven or manual) is useless (it's probably very good, I've just never used one), or that limited distribution is not *a* solution (I can see it being *a* solution). Just wanted to point out that even if you do all that, there is still weird stuff that can happen, and will. -- K 2013/7/29 Kristopher Micinski krismicin...@gmail.com So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer.gi...@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big companies like Samsung, HTC, Motorola, LG and so on, contain tons of implementation bugs that prevent apps from working correctly. I'm talking about the fact that you can call a certain simple API, test it on a stock Android ROM (like on Nexus 4), and then have your application crash on some Samsung, that decided to break the implementation because of some customization. How can people stand that? How is it possible to write code, when the machine that executes it is completely broken in unexpected ways? I'm really fed up with it. About 50% of my Android development time is wasted on babysitting broken devices. I'm
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
I'm amused by what came to my mind because of what you wrote - I thought of having a website like that for mutual testing between Android developers. We can call it DroidSurfing (like CouchSurfing), and the point is simple - registered like-minded Android developers let your code surf on their devices, and in exchange you provide the same luxury for other developers. Of course, some minimal reputation monitoring is required, just to prevent malicious developers from spreading false results or from distributing someone's paid app for free... On Monday, July 29, 2013 1:14:48 AM UTC+3, Kristopher Micinski wrote: I didn't posit that there were concrete things you could go out and buy right now, so apologies if I misphrased. I know a group who works on a research project which does basically this, but there's no word on whether it will be open source any time soon. I think the idea is what I just mentioned, collaborative crowd based testing services that allow you to test on a wide range and variety of devices with very expressive test scripts and possible end user testing when applicable. Kris On Sun, Jul 28, 2013 at 6:05 PM, Omer Gilad omer@gmail.comjavascript: wrote: With all this talk I still haven't seen an example of a testing service that can actually help me... Got any links? Anything with a reasonable price that can ease the pain (and that you used for yourself)? BTW, I have worked in the past with some of those remote device providers (like http://www.keynotedeviceanywhere.com/). It's very expensive to get a few hours with a device, especially if this is something you have to do constantly with different devices each time there's an issue. And the deal-breaker part - it's so slow, you'll actually have to waste a whole hour just on installing your app and seeing LogCat. On Monday, July 29, 2013 12:48:16 AM UTC+3, Kristopher Micinski wrote: So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Absolutely, I think I made pretty clear that it wasn't intended as a solution. (E.g., my many previous posts agreeing with you and admitting it was a numbers game.) But that being said, if it's a numbers game, the idea gets you a lot closer to it than testing on the n devices you own. To be fair, in the case of maps you have slightly more to test because you have a bunch of graphics firmware code, etc... I know that you can also find more mundane examples giving evidence of this phenomenon with the regular Android SDK, however. Kris On Sun, Jul 28, 2013 at 6:19 PM, Kostya Vasilyev kmans...@gmail.com wrote: A testing service still -- focuses only on specific device models, assuming deterministic failures, ignoring weird shit that can happen on seemingly any of them, even the good ones. Going back to my Youtube video of Google Maps causing a device reboot: Is the Galaxy Nexus a bad device? No. Wasn't Google Maps tested on it? I'm sure it was. And yet, there it was. PS - I'm not saying a testing service (commercial or co-op, test driven or manual) is useless (it's probably very good, I've just never used one), or that limited distribution is not *a* solution (I can see it being *a* solution). Just wanted to point out that even if you do all that, there is still weird stuff that can happen, and will. -- K 2013/7/29 Kristopher Micinski krismicin...@gmail.com So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer.gi...@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS specific issues that will frustrate you.. especially since you're used to how Android is. On Friday, July 26, 2013 1:39:14 AM UTC+3, Omer Gilad wrote: .I am wondering how developers here are dealing with the fact that there are 1000's of devices out there, some of them running your applications in very broken ways .I keep running into these kind of issues again and again for the past 3 years, and to be honest, I'm fed up with it .I've decided to move to iOS development, and the only way to convince me otherwise is to give me a decent, reliable way of dealing with fragmentation So what do you do when you develop a game, for example, and try to create a high-quality user experience on Google Play? Do you do your QA on 50 different devices? 100? 1000? Or do you just shoot blindly and hope that it works, or wait for users to send you bug reports? To make it clear, I'm not talking about official fragmentation. I don't talk about different screen sizes, densities, features, OS versions and so on. I talk about the unofficial fragmentation. The fact that most devices, even the popular ones from the big
Re: [android-developers] Re: Dealing with 1000's of different devices, each one with its own bugs
Yes, making a system like this work in the real world requires many other things, including proper incentivization (do the developers get back a proportional amount of what they put in), metrics for accounting, adjustment for device ownership stats (people with less common devices will be unduely stressed for testing) and many other things I've sort of thought about but haven't written down. But that's sort of a given when you take research and try to make it really happen :-) Kris On Sun, Jul 28, 2013 at 6:35 PM, Omer Gilad omer.gi...@gmail.com wrote: I'm amused by what came to my mind because of what you wrote - I thought of having a website like that for mutual testing between Android developers. We can call it DroidSurfing (like CouchSurfing), and the point is simple - registered like-minded Android developers let your code surf on their devices, and in exchange you provide the same luxury for other developers. Of course, some minimal reputation monitoring is required, just to prevent malicious developers from spreading false results or from distributing someone's paid app for free... On Monday, July 29, 2013 1:14:48 AM UTC+3, Kristopher Micinski wrote: I didn't posit that there were concrete things you could go out and buy right now, so apologies if I misphrased. I know a group who works on a research project which does basically this, but there's no word on whether it will be open source any time soon. I think the idea is what I just mentioned, collaborative crowd based testing services that allow you to test on a wide range and variety of devices with very expressive test scripts and possible end user testing when applicable. Kris On Sun, Jul 28, 2013 at 6:05 PM, Omer Gilad omer@gmail.com wrote: With all this talk I still haven't seen an example of a testing service that can actually help me... Got any links? Anything with a reasonable price that can ease the pain (and that you used for yourself)? BTW, I have worked in the past with some of those remote device providers (like http://www.keynotedeviceanywhere.com/). It's very expensive to get a few hours with a device, especially if this is something you have to do constantly with different devices each time there's an issue. And the deal-breaker part - it's so slow, you'll actually have to waste a whole hour just on installing your app and seeing LogCat. On Monday, July 29, 2013 12:48:16 AM UTC+3, Kristopher Micinski wrote: So in this case, how does a subscription based test service not help you? I'm not saying that a concrete one exists, but I think this kind of debugging service (or coop, essentially) would be a good tool. You include a time metric, do some tasks to help other developers', and they do some work of doing yours. One of the problems here is the heterogenous distribution of devices, but I don't think that's an inherent limitation. I've thought about starting up one of these services for a while, but don't really have the resources to do so. (I think in my previous posts you thought I was advocating a pushbutton testing service: I wasn't. But the point still stands: if you want to test on greater devices, do it with a service and possibly humans in the loop. Big testing services should integrate this work cycle too, for when pushbutton tests don't work...) Kris On Sun, Jul 28, 2013 at 5:29 PM, Omer Gilad omer@gmail.com wrote: What you wrote is the obvious part of what I do - test with beta users. I agree that this is a must. The problem is, sometimes it's impossible to debug what you find. When the issue is not a simple crash stack trace - but rather some behavior, or display issue, you can't just keep ping-ponging versions with a user without wasting whole days on that... You need the device in your hand. And as an indie developer, it's practically impossible to get a hold of many different devices. On Sunday, July 28, 2013 12:47:30 PM UTC+3, Piren wrote: Wrote a lengthy response but my browser decided not to post it, so here's the short version: - That's a known problem with android development, it was obvious about a couple of months after it came out. when the premise of the system is to be open and as varied as possible, this kind of issues are a given. - Under your limitations, the best approach is to release the app only to a small subset of devices it was tested on and expand that subset as time goes on. Use an open beta group for devices you do not have access to. Even Netflix was released on only 5 devices. - iOS development might not have this issue (it has fragmentation, but it isn't the same as android's), but over all i believe android has a more developer friendly ecosystem... instead of being frustrated with this, you'll find more than enough other iOS
