We've seen the same behavior when running an lldb command file at startup. Foo contains: b main run
lldb -s foo test.exe (this hangs) Take out the run from foo, and lldb -s foo -o run test.exe (this works) -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project > -----Original Message----- > From: lldb-dev-boun...@cs.uiuc.edu [mailto:lldb-dev-boun...@cs.uiuc.edu] On > Behalf Of Pavel Labath > Sent: Monday, March 23, 2015 4:22 AM > To: Zachary Turner > Cc: lldb-dev@cs.uiuc.edu > Subject: Re: [lldb-dev] Synchronous execution with process plugin > > This looks very much like the issue I was experiencing on linux, where lldb > would just lock up if I executed the process in synchronous mode. > This was fixed with <http://reviews.llvm.org/D8079>, and I suspect you need to > do something similar. The trick here is that you need to set up process event > hijacking in your host launch code. Otherwise, Target::Launch will attempt to > use the global process listener to wait for the process to stop. However, this will > race with the event handler thread, since it uses the same listener to process > events. So, depending on which thread processes the event, it will either work > fine, or you will end up locked in WaitForProcessToStop forever, because the > event will never come. > > Also, note that there is another issue with launching in synchronous mode, > where stdio is not forwarded to the target process > <http://lists.cs.uiuc.edu/pipermail/lldb-dev/2015-March/006853.html>. > I was planning on dealing with this, but I don't think I will get around to it for a > while yet... > > pl > > On 21 March 2015 at 01:35, Zachary Turner <ztur...@google.com> wrote: > > Hmm, on Monday I'll try to see if i can figure out where to put a > > breakpoint to get hit when someone removes something from the queue. > > Thanks On Fri, Mar 20, 2015 at 5:52 PM <jing...@apple.com> wrote: > >> > >> > >> > On Mar 20, 2015, at 5:21 PM, Zachary Turner <ztur...@google.com> > wrote: > >> > > >> > If I do that my process does stop at main, but I think I'm hit with > >> > a race condition. 3 times out of 5 I saw the same results as you, > >> > where it doesn't print the backtrace but if I do bt it works fine. > >> > The other 2 times out of 5 It does print the backtrace, but then > >> > I'm deadlocked. I added a bunch of tracepoints so we can see the > >> > order of events. here's what it looks like when it deadlocks. > >> > > >> > Target::Launch about to call ProcessWindows::Launch > >> > ProcessWindows::DoLaunch entering ProcessWindows::DoLaunch about to > >> > call DebugLaunch ProcessWindows::DoLaunch succeeded, waiting for > >> > initial stop Application > >> > "\??\D:\src\llvm\tools\lldb\test\expression_command\formatters\a.ou > >> > t" found in cache ProcessWindows::OnDebuggerConnected > >> > ProcessWindows::OnDebugException > >> > ProcessWindows got EXCEPTION_BREAKPOINT ProcessWindows got initial > >> > stop, setting initial stop event ProcessWindows::DoLaunch received > >> > initial stop, returning ProcessWindows::DoLaunch finished launching > >> > process, exiting Target::Launch returned from > >> > ProcessWindows::DoLaunch, eLaunchFlagStopAtEntry == false, about to > >> > call WaitForProcessToStop Target::Launch, WaitForProcessToStop > >> > returned, new state = eStateStopped Target::Launch about to call > >> > ProcessWindows::PrivateResume ProcessWindows::DoResume resuming > >> > from active exception ProcessWindows::DoResume couldn't find an > >> > active exception, setting state to eStateRunning > >> > ProcessWindows::OnDebugException ProcessWindows got > >> > EXCEPTION_BREAKPOINT Target::Launch synchronous_execution is true, > >> > calling WaitForProcessToStop again > >> > > >> > The interesting stuff is the last 2 lines. It hits the breakpoint > >> > in main *before* calling WaitForProcessToStop, so > >> > WaitForProcessToStop never sees a "change". > >> > >> > Here's what it looks like when it works. > >> > > >> > Target::Launch about to call ProcessWindows::Launch > >> > ProcessWindows::DoLaunch entering ProcessWindows::DoLaunch about to > >> > call DebugLaunch ProcessWindows::DoLaunch succeeded, waiting for > >> > initial stop Application > >> > "\??\D:\src\llvm\tools\lldb\test\expression_command\formatters\a.ou > >> > t" found in cache ProcessWindows::OnDebuggerConnected > >> > ProcessWindows::OnDebugException > >> > ProcessWindows got EXCEPTION_BREAKPOINT ProcessWindows got initial > >> > stop, setting initial stop event ProcessWindows::DoLaunch received > >> > initial stop, returning ProcessWindows::DoLaunch finished launching > >> > process, exiting ProcessWindows breakpoint handler set private > >> > state to eStateStopped Target::Launch returned from > >> > ProcessWindows::DoLaunch, eLaunchFlagStopAtEntry == false, about to > >> > call WaitForProcessToStop Target::Launch, WaitForProcessToStop > >> > returned, new state = eStateStopped Target::Launch about to call > >> > ProcessWindows::PrivateResume ProcessWindows::DoResume resuming > >> > from active exception ProcessWindows::DoResume couldn't find an > >> > active exception, nothing to resume > >> > ProcessWindows::OnDebugException ProcessWindows got > >> > EXCEPTION_BREAKPOINT Target::Launch synchronous_execution is true, > >> > calling WaitForProcessToStop again ProcessWindows breakpoint > >> > handler set private state to eStateStopped Target::Launch > >> > WaitForProcessToStop returned, yay! > >> > > >> > > >> > The difference here is that "calling WaitForProcessToStop again" happens > >> > before the process plugin sets the private state. Can you try repeating > >> > your test multiple times and seeing if you can get the deadlock to occur? > >> > >> I can't get this to fail on OS X but with timing related things I'm > >> not sure that's all that significant... > >> > >> Anyway, what you are describing doesn't make sense to me yet. After > >> calling PrivateResume, we call WaitForProcessToStop, passing > "wait_always" > >> set to true. When wait_always is true, WaitForProcessToStop won't > >> return till it receives a stop event. It isn't comparing current > >> state against desired state, it is actually waiting for an event. So > >> it should not matter when the breakpoint hit arrives relative to calling > "WaitForProcessToStop" > >> since the breakpoint hit is just going to generate a stop event, > >> which will sit in the event queue till somebody fetches it. It's > >> fine for that to happen before you call WaitForProcessToStop, because > >> we aren't comparing current to desired state, we are waiting for that event. > >> > >> Maybe somebody else is grabbing the breakpoint stop event and taking > >> it off of the queue? That would explain why the second > >> WaitForProcessToStop - the one with wait_always set to true - is missing the > event? > >> > >> Jim > >> > >> > >> > > >> > > >> > On Fri, Mar 20, 2015 at 4:49 PM <jing...@apple.com> wrote: > >> > I'm not sure it is deadlocking the debugger. lldb is just waiting > >> > for a stop. For instance ^C should interrupt it, or sending a > >> > signal externally to the process, or triggering a breakpoint or crash, etc. > >> > > >> > Actually, Greg must have fixed the bug I was remembering, because > >> > this works correctly for me with TOT lldb. > >> > > >> > What happens for you if your .lldbinit has: > >> > > >> > file a.out > >> > break set -n main > >> > run > >> > > >> > For me this stops at the breakpoint at main. We still have a > >> > little clean up to do here, because I don't see the stop notification in this > case. > >> > I see: > >> > > >> > > lldb -S cmds.lldb > >> > (lldb) command source -s 1 'cmds.lldb' > >> > 4 locations added to breakpoint 1 > >> > (lldb) > >> > > >> > but if I then do "bt" I'm sitting at main: > >> > > >> > (lldb) bt > >> > * thread #1: tid = 0x1bf8a4, function: main , stop reason = > >> > breakpoint > >> > 1.1 > >> > * frame #0: 0x0000000100018e87 Sketch`main at SKTMain.m:17 > >> > frame #1: 0x00007fff94f445ad libdyld.dylib`start > >> > frame #2: 0x00007fff94f445ad libdyld.dylib`start > >> > > >> > Not sure what's up with the stutter at start either. But that's a > >> > different rabbit to chase... > >> > > >> > Jim > >> > > >> > > >> > > On Mar 20, 2015, at 4:40 PM, Zachary Turner <ztur...@google.com> > >> > > wrote: > >> > > > >> > > If that's the case, then a .lldbinit file like this: > >> > > > >> > > file a.out > >> > > run > >> > > > >> > > Will deadlock the debugger, because the real stop never comes? > >> > > > >> > > On Fri, Mar 20, 2015 at 4:35 PM <jing...@apple.com> wrote: > >> > > That's the stop at entry stop. The code you quoted is in a block > >> > > that starts with: > >> > > > >> > > if (launch_info.GetFlags().Test(eLaunchFlagStopAtEntry) > >> > > == > >> > > false) > >> > > { > >> > > > >> > > So we've stopped at the entry point, but the user didn't want to > >> > > know about that, so we resume and wait for a "real" stop. > >> > > > >> > > Jim > >> > > > >> > > > >> > > > On Mar 20, 2015, at 4:30 PM, Zachary Turner > >> > > > <ztur...@google.com> > >> > > > wrote: > >> > > > > >> > > > I'm a little confused. You said that in synchronous execution, > >> > > > Launch won't return until the process has stopped. That makes > >> > > > sense, but it already checks that the process has stopped once > >> > > > regardless of whether synchronous execution is set. Then, it > >> > > > calls PrivateResume() (even if synchronous_execution is set), and then > waits for the process to stop again? > >> > > > What would trigger this second stop? Target::Launch already > >> > > > asked it to resume, so now it's happily running while > >> > > > Target::Launch is waiting for it to stop a second time. > >> > > > > >> > > > On Fri, Mar 20, 2015 at 4:23 PM <jing...@apple.com> wrote: > >> > > > In synchronous execution, the "Launch" command won't return > >> > > > till the process has stopped. The point of synchronous execution is that > you can do: > >> > > > > >> > > > break set -n foo > >> > > > run > >> > > > bt > >> > > > > >> > > > So "run" can't return till the breakpoint has been hit. That > >> > > > is why it waits for the process to stop. I'm not quite sure > >> > > > why this is done in Target::Launch, in other cases (e.g. in for > >> > > > "step" and "continue" the command object is the one that takes > >> > > > care of waiting for the stop. Launch is a little funny > >> > > > however, because it can't use the normal process wait mechanism > >> > > > to do its job since the real process isn't alive when it has to start > waiting... > >> > > > > >> > > > I think the reason you are hanging here is that the code that > >> > > > reads in all the init statements runs an event loop temporarily > >> > > > while it is reading them in, and the kills that and hands off > >> > > > the the real command execution loop, and this continuation gets > >> > > > lost in the handoff. I thought Greg had already fixed that, but maybe > it's still sitting in his queue. > >> > > > > >> > > > Jim > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Mar 20, 2015, at 3:57 PM, Zachary Turner > >> > > > > <ztur...@google.com> > >> > > > > wrote: > >> > > > > > >> > > > > I ran into an issue earlier where I tried to make a .lldbinit > >> > > > > file with some lines like this: > >> > > > > > >> > > > > file a.out > >> > > > > run > >> > > > > > >> > > > > > >> > > > > When this happens the process runs, the breakpoint gets hit > >> > > > > and I see the source listing, it returns to the lldb prompt, > >> > > > > but then I can't type anything. It appears LLDB is > >> > > > > deadlocked inside of Target::Launch() at the following location: > >> > > > > > >> > > > > if (!synchronous_execution) > >> > > > > m_process_sp->RestoreProcessEvents (); > >> > > > > > >> > > > > error = m_process_sp->PrivateResume(); > >> > > > > > >> > > > > if (error.Success()) > >> > > > > { > >> > > > > // there is a race condition where this > >> > > > > thread will return up the call stack to the main command > >> > > > > // handler and show an (lldb) prompt > >> > > > > before HandlePrivateEvent (from PrivateStateThread) has > >> > > > > // a chance to call PushProcessIOHandler() > >> > > > > m_process_sp->SyncIOHandler(2000); > >> > > > > > >> > > > > if (synchronous_execution) > >> > > > > { > >> > > > > > >> > > > > state = m_process_sp->WaitForProcessToStop (NULL, NULL, true, > >> > > > > hijack_listener_sp.get(), stream); > >> > > > > const bool must_be_alive = false; // > >> > > > > eStateExited is ok, so this must be false > >> > > > > if (!StateIsStoppedState(state, > >> > > > > must_be_alive)) > >> > > > > { > >> > > > > > >> > > > > error.SetErrorStringWithFormat("process isn't stopped: %s", > >> > > > > StateAsCString(state)); > >> > > > > } > >> > > > > } > >> > > > > } > >> > > > > > >> > > > > Normally when I'm using LLDB and entering the commands > >> > > > > myself, this synchronous_execution value is not set, and > >> > > > > everything works as expected. How is this supposed to work? > >> > > > > What does my plugin need to do differently in order to handle > >> > > > > this case? The process has already stopped once and resumed, > >> > > > > so I'm not sure why it would need to stop again? I see that > >> > > > > it's not restoring process events in the case of synchronous > execution, so maybe it should have never resumed in the first place? > >> > > > > _______________________________________________ > >> > > > > lldb-dev mailing list > >> > > > > lldb-dev@cs.uiuc.edu > >> > > > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev > >> > > > > >> > > > >> > > >> > > > > _______________________________________________ > > lldb-dev mailing list > > lldb-dev@cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev > > > _______________________________________________ > lldb-dev mailing list > lldb-dev@cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev _______________________________________________ lldb-dev mailing list lldb-dev@cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
