Most of my code nowadays is ruby, by integration testing I mean testing at the  
controller or API/request level. I also deployed java in data centers and EC2. 
The usual debugger works in production on both platforms but I do not see how 
you could use it without impacting your customers. Staging of course is perfect 
for debugging.

If there is a problem in production I revert to the previous version code and 
database, if needed. It is crucial to be able ro revert fast, the standard ruby 
way is to have separate folders and softlinks. Then you download the logs and 
either fix the bug or add more logging and deploy to just one server if you 
really can't figure it out. Better to not paint yourelf in that corner too 
often :-(

So handling production problems is more about the process than the tools.

Curious to hear other opinions :)

Marius


On Feb 8, 2011, at 6:49 PM, Brian Fromme <[email protected]> wrote:

> Thanks Marius,
> 
> This is excellent advice.  So, on your integration-level testing, for 
> example, what happens when a bug is seen there?  Does any debugging go on at 
> the cloud, or is it all done in a simulated local environment while looking 
> at various log files?
> 
> I know for me, I would want to have some sort of tools in the staging 
> environments that I could use for diagnostics and debugging of the real 
> execution.
> 
> thanks,
> Brian
> 
> On Feb 8, 2011, at 5:23 PM, Marius Seritan wrote:
> 
>> Brian,
>> 
>> I handle the complexity of the cloud as the next level of complexity in the 
>> application. I do as much testing as I can at the unit level, then 
>> integration, then QA on staging. If the code makes it to production with a 
>> real bug all the bets are off. Generally what ever error is left gets 
>> reported by  CPU monitoring on each system (monit) and through centralized 
>> error reporting (hoptoad). Deploying in stages also helps to contain what 
>> ever problem appears.
>> 
>> As the code matures I add custom munin plugins to monitor specific processes 
>> and see trends.
>> 
>> Marius
>> 
>> -- 
>> Marius Seritan                        http://www.de-co-de.com
>> [email protected]      (415) 409-9277
>> 
>> 
>> 
>> 
>> 
>> 
>> On Feb 8, 2011, at 3:43 PM, Brian Fromme wrote:
>> 
>>> Hi,
>>> 
>>> I'm researching tools for cloud development.  Can anyone comment on 
>>> tools/methods for debugging their scalable application in the cloud?
>>> 
>>> thanks,
>>> Brian
>>> 
>>> 
>>> -- 
>>> Ubuntu-cloud mailing list
>>> [email protected]
>>> Modify settings or unsubscribe at: 
>>> https://lists.ubuntu.com/mailman/listinfo/ubuntu-cloud
>> 
> 
> 
> -- 
> Ubuntu-cloud mailing list
> [email protected]
> Modify settings or unsubscribe at: 
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-cloud

-- 
Ubuntu-cloud mailing list
[email protected]
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-cloud

Reply via email to