This is useful for any CouchDB developer :) Thanks Dave! Jan --
Begin forwarded message: > From: Dave Cottlehuber <[email protected]> > Subject: debugging > Date: November 5, 2012 09:11:49 GMT+01:00 > To: [email protected] > Reply-To: [email protected] > > Hi there, > > I wanted to throw in a brief set of notes on debugging stuff I learned > about, some recently & some taken from the internets. > > most of this is for running "couchdb -i" but you can also do this via > remsh if you set a cookie and either sname/name up. Let me know if > that could be covered too. > > # tell a non-networked erlang vm (e.g. started by couchdb -i) to get connected > > net_kernel:start([<newname>, shortnames]). > erlang:set_cookie(node(), 'yumyumyum'). > > now you can remsh or redbug to it. (what's redbug??) …. > > # killing off a process (e.g. a view server) that you don't want > running but there's no API to allow you to do that. The erlang shotgun > is here! > > i(). %% look for the process name. You could do this nicer with > list comprehensions etc below. > exit(list_to_pid("<0.114.0>"), false). %% boom! > > # get a list of all running processes and then do evil stuff > > - in this case we find out who is the memory hog, using v(-1) to refer > to the result of the last computation in the shell and keeping the > info around in Procs just in case we want to check it later. > > %% this trick via Geoff Cant / @archaelus > Procs = [ {Pid, element(2, process_info(Pid, memory))} > || Pid <- erlang:processes() ]. > > hd( lists:reverse( lists:keysort( 2, v(-1) ) ) ). > > # trace all calls going through a particular module > > NB this is pretty savage if you get it wrong, you can *easily* crash > the vm. redbug is usually a better option, but I don't see how to get > it to do this nifty trick: > > open_tracer(Module) -> > dbg:tracer(), > dbg:p(all, [call]), > dbg:tpl(Module, [{'_', [], [{return_trace}]}]). > > # redbug > > If you're using erlang and you don't know about redbug you are doing it > wrong!! > > Install redbug as part of eper kit, build is straightforwards using > rebar, add it into your ebin path. Again I can provide more details on > that if needed. > > https://github.com/massemanet/eper/blob/master/doc/redbug.txt > > Let's do a simple example - tracing all calls to lists:max, from our shell: > > (akai@akai)20> redbug:start("lists:max"). > ok > (akai@akai)21> lists:max([100,900,300,100,720]). > > 08:57:40 <{erlang,apply,2}> {lists,max,[[100,900,300,100,720]]} > 900 > > 08:57:40 <{erlang,apply,2}> {lists,max,[[900,300,100,720],100]} > > 08:57:40 <{erlang,apply,2}> {lists,max,[[300,100,720],900]} > > 08:57:40 <{erlang,apply,2}> {lists,max,[[100,720],900]} > > 08:57:40 <{erlang,apply,2}> {lists,max,[[720],900]} > > 08:57:40 <{erlang,apply,2}> {lists,max,[[],900]} > > cool! you can see your recursion in action. > > Fancy, how about doing that from a remote node? > > switch to your shell & set up redbug to call from a normal prompt/terminal. > > launch redbug from your terminal: > > % redbug akai@akai 5000 10 "lists:max" > > this tetls it to connect to erlang host akai@akai, with a timeout of 5 > seconds (you'll likely need longer the first few times), and collect a > maximum of 10 debug traces before stopping. Trace only lists:max. > > redbug also accepts cookies and a bunch of other self-explanatory options. > > whip back quickly into your erl shell and up arrow to restart the > lists:max function. > > Your terminal will show the same goodies. > > Now imagine doing the same thing on a live system, tracing a specific > call through mochiweb layer, or something in couch docs storage > system. Woot!!! > > redbug can also optionally provide the call stack, arguments and > results but that starts getting hefty. So then write the results out > to a file. > > Why redbug? > > - ease of use compared to writing tracers yourself > - excellent logging > - filtering by time and number of results tends to avoid the "oops my > real-time debugging killed your couches". > > # stats inside the vm > > A number of options - use a fancy tool like vmstats from mononcqc or > for a more hands-on view use entop. > > > # debugging erlang startup > > sometime erlang barfs on startup. how to proceed? > > erl -init_debug -env ERL_LIB > /usr/local/Cellar/couchdb/1.2.0/lib/couchdb/erlang/lib -couch_ini > > And then hand-start each application: > > application:start(crypto). > application:start(couch). > … > > Sometimes this helps find the issue. A working sample is here > https://friendpaste.com/6zett3aAHSafRNZbgfMBoD > > I should try the last step broken down into subsidiary steps rather > than a single application. > > Anybody got other tips/tricks to share? > > A+ > Dave
