Hello Vasily, I like your constructive overview. I agree with you that traversing managed frames is quite VM specific and proves to be a problem for native debuggers.
To my personal point of view, the complexity of using external thread libraries is underestimated in your analysis since VM uses several self-made synchronization primitives which do most of the job. For example, not far from now gdb on IPF crashed in a simple test. Windows threading model may prove some unexpected challenges. For example, Ivan Volosyuk spent a day trying to understand that our VM thread local data should be prefixed with unused field to made debugging possible. I faced a number of gdb crashes on Windows. As for your questions of alternatives, a year ago I investigated cross platform alternatives for native frame enumeration. I considered gdb and a debugger from Intel compiler. I talked to people who developed the debuggers, and then decided to use our internal solution Ilya was finalizing. Thank you for your attention. BTW, I believe we should not mix two things in our discussion. I like if someone will come up with better solution: many places of DRLVM can be improved. From the other side casting a vote BTW2, the file pthread_dbg.h you've provided a reference to doesn't have a pure BSD license, see the third item. On 10/13/07, Vasily Levchenko <[EMAIL PROTECTED]> wrote: > Originally, Java developers using JNI have to debug their application using > two debugger for accessing java and native world( > http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html, > http://developers.sun.com/learning/javaoneonline/2006/tools/TS-1011.pdf). > This concept should be the foundation for "mixed" debugger. Native debugger > (gdb, adb or other) to debug java world need information how to deal with > java stack, other jdb functionality comes almost automatically. Let me start > from the final stage of the good "mixed mode debugger". Ideal mixed mode > debugger should open core or dump file of java application crush restoring > internal state like it do for other application, it should attach and start > application in debugging mode. If look on debuggers' history we can see > similar task that was done before, e.g. thread handling in debugger. How > debugger operates with thread? How it get TSD and TLS from the thread on > remote application or core file? A long-long time ago Solaris or Sun OS > introduced thread debugging interface libc_db > http://docs.sun.com/app/docs/doc/816-5173/6mbb8adsh?l=en&a=view and then it > migrated to Linux world ( > http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/nptl_db/?cvsroot=glibc) > and BSD (http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpthread_dbg/) world > (thread_db). This library inform debugger how to fetch thread information > from core file or debugee process. Similar approach could be done with java > world but before we need several baby steps. > > Short-term: > 1. Using JVMTI possibilities provide information for debugger to operate > with the java stack > 2. Provide debugger operations throw JVMTI (break points, watch points and > so on) > > Result construction could used as a source for a long solution described as > ideal mixed-mode-debugger. > > BTW: here some links about ready to use short-term solutions: > http://www.hp.com/products1/unix/java/pdfs/07_06Tools_NativeDebug.pdf > http://docs.sun.com/source/819-3683/Java_debug.html > > On 10/12/07, Tim Ellison <[EMAIL PROTECTED]> wrote: > > > > It looks like there is some good discussion going on around this topic, > > and possible alternatives being explored. Rather than prematurely > > forcing the issue with a vote, maybe we should stop the vote and > > continue the discussion for a while until we reach a consensus, then > > restart the vote. > > > > What do you think? > > > > Regards, > > Tim > > > > Vasily Levchenko wrote: > > > On 10/12/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > >> Vasily, > > >> > > >> You wrote, > > >>> As I understand my project is single user of NCAI > > >> I believe this is not the only use case for NCAI. Keeping internal > > >> layer of debugging interfaces is important for VM development itself. > > > > > > > > > Why are you so sure that it will help you in internal VM development? > > you > > > can used it for building some kind viewer of state in JNI wrapper, but > > not > > > more. Your "viewer" can't get debugging services provided by kernel. > > You're > > > breaking the model that people build for a long time ago and develop new > > one > > > instead. You haven't (not personally you but harmony community I guess) > > > even try re-use the existing solutions. > > > > > > > > > Let me assure you that I'm far from convincing you to support NCAI in > > >> your stepchild tool if you don't want to. > > >> > > >> Thanks. > > >> > > >> On 10/12/07, Vasily Levchenko < [EMAIL PROTECTED]> wrote: > > >>> On 10/12/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > >>>> Vasily, > > >>>> > > >>>> 1. > > >>>> You use capitalized wordings like "Mixed Mode" debugging or > > >>>> "Integrated Debugger was transferred to me". This sound like names of > > > > >>>> internal projects you are participating. It would be great if you > > >>>> start a new thread and disclose your activity to the community (and > > >>>> your humble servant). Or if they are top secret projects, I have > > never > > >>>> seen these references on the dev list. :-) Thank you in advance. > > >>> > > >>> > > >>> Yes it's public available semi-product that available on > > >>> http://whatif.intel.com, and actually I'm the owner of Integrated > > >> debugger > > >>> for Java/JNI environments. As I understand my project is single user > > of > > >> NCAI > > >>> and will available during Q4. > > >>> > > >>> > > >>> 2. > > >>>> Sorry for being unaware of projects like libthread_db.so. I've tried > > >>>> to google it and the first search item didn't describe this project > > >>>> well: > > >>>>> 'Crashes in /lib/libthread_db.so.1' thread - MARC > > >>>> Could you please support your words with references to these > > >>>> alternative projects and their licenses? > > >>> > > >>> libthread_db.so is usual part of system libraries e.g. for GLIbc is > > LGPL > > >>> in BSD world it's BSDL and so on. > > >>> > > >>> > > >>> 3. > > >>>> In any case I believe we need to take the code now, and than compare > > >>>> it with alternative when it is ready. Technologies quickly change and > > >>>> replace each other, and I don't think we should reject a contribution > > >>>> counting on its possible future obsolescence. > > >>> > > >>> > > >>> I don't really undestand what harry for? > > >>> > > >>> > > >>> Contrary, our mission is to make today's code a good base for the new > > >>>> development. > > >>> > > >>> If you really want good code for development, read NetBSD source tree > > >> and > > >>> polish current code ;) > > >>> > > >>> > > >>> Any code will become obsolete one day. I believe > > >>>> interfaces are the easiest thing to be replaced. Other things in this > > >>>> implementation such as stack frame enumeration will live much longer > > >>>> because there are not too many sane maniacs which are able to > > >>>> understand all our calling conventions. > > >>>> > > >>>> Thanks! > > >>>> > > >>>> > > >>>> On 10/12/07, Vasily Levchenko < [EMAIL PROTECTED]> wrote: > > >>>>> On 10/12/07, Alexei Fedotov <[EMAIL PROTECTED]> wrote: > > >>>>>> Hello Vasily, > > >>>>>> > > >>>>>> Now I see your attitude towards this donation. :-) When I initally > > >>>>>> looked through interface specification, I was not very supportive > > >> and > > >>>>>> replied with the same argument about gdb. > > >>>>>> > > >>>>>> Unfortunately, gdb is not BSD-licensed or Apache compatible. So we > > >>>>>> cannot link with it. It is important if we want to debug > > >> application > > >>>>>> in a JVMTI-like way from the same process. > > >>>>> > > >>>>> It's really doesn't matter you can find the sample of GDB agents > > >> under > > >>>> LGPL > > >>>>> and BSD license, actually good example libthread_db.so that provide > > >>>> access > > >>>>> to libpthread.so internals fo GDB usually it's the same version as > > >> the > > >>>> rest > > >>>>> of the system libraries. > > >>>>> > > >>>>> > > >>>>> If you like solutions with external debuggers, you may come up with > > >>>>>> one as well as Salikh did two years ago, and this proved to be > > >> useful. > > >>>>>> This just does not help in two areas I've initially outlined: > > >>>>>> defensive programming and crash handling. > > >>>>> > > >>>>> > > >>>>> It were actually one of the lack of good crash handling that we > > >>>> discussed > > >>>>> when the Integrated Debugger was transfered to me. > > >>>>> > > >>>>> > > >>>>> Having a level of internal > > >>>>>> reflection from inside VM is very important to express assertions > > >>>>>> about correct VM state for a debug build. > > >>>>> > > >>>>> > > >>>>> Yes, it's true but it should be done in similar way as thread_db > > >>>> interface > > >>>>> which could provide even core file analyzing, but with in-process > > >>>> debugger > > >>>>> it is impossible or very hard to implement. > > >>>>> > > >>>>> > > >>>>> Finally I want to address your concern of a "pure viewing" nature of > > > > >>>>>> NCAI. NCAI allows changes in runtime environment. For example, you > > >> can > > >>>>>> change values of variables, or any memory location. > > >>>>> > > >>>>> > > >>>>> GDB or any other out-of-process debugger provides the same > > >> functionality > > >>>>> without any side-effects. > > >>>>> > > >>>>> > > >>>>> Thanks. > > >>>>>> > > >>>>>> On 10/12/07, Vasily Levchenko <[EMAIL PROTECTED]> > > >> wrote: > > >>>>>>> On 10/12/07, Alexei Fedotov < [EMAIL PROTECTED]> wrote: > > >>>>>>>> +1 > > >>>>>>>> NCAI adds very useful architectural level for VM internal > > >>>> debugging > > >>>>>>>> and defensive programming. The best example is a crash > > >> handler, > > >>>> but > > >>>>>>>> other utilities such us self-detection of deadlocks, is also > > >> quite > > >>>>>>>> interesting. > > >>>>>>> > > >>>>>>> NCAI could be interface to build some kind of viewer, but no > > >> real > > >>>>>> debugger. > > >>>>>>> 1. It couldn't be very useful just because it's single process > > >>>>>> debugging > > >>>>>>> model and if you set breakpoint incorrectly you could lock whole > > >>>> JVM. > > >>>>>>> 2. building debugger from the very beginning you try to invent > > >> wheel > > >>>>>> once > > >>>>>>> again, you have to develop new mechanisms that already exists in > > >>>> native > > >>>>>>> debuggers. > > >>>>>>> 3. building infrastructure for Mixed Mode debugging using > > >> native > > >>>>>> debugger > > >>>>>>> could be implemented in 1-2 month and it'll be really powerful > > >> and > > >>>>>> useful > > >>>>>>> solution especially JNI developers. > > >>>>>>> > > >>>>>>> HP(gdb) and Sun (dbx) already introduced native debugger-centric > > >>>>>> solutions > > >>>>>>> for debugging in mixed environments. So NCAI seams to be > > >> potential > > >>>> dead > > >>>>>> code > > >>>>>>> and I won't be surprised if year or two later you'll be voting > > >> for > > >>>>>> removing > > >>>>>>> this code. > > >>>>>>> > > >>>>>>> On 10/12/07, Vasily Levchenko <[EMAIL PROTECTED]> > > >> wrote: > > >>>>>>>>> Hi Mikhail, > > >>>>>>>>> Have you got any idea how to use it? Except "Integrated > > >> Debugger > > >>>> for > > >>>>>>>>> Java/JNI environment" (former MMD). > > >>>>>>>>> > > >>>>>>>>> On 10/12/07, Mikhail Loenko < [EMAIL PROTECTED]> wrote: > > >>>>>>>>>> We now have all requisite paperwork in place for > > >>>>>>>>>> http://issues.apache.org/jira/browse/HARMONY-4689, so > > >> let's > > >>>> vote > > >>>>>> to > > >>>>>>>>>> accept: > > >>>>>>>>>> > > >>>>>>>>>> [ ] +1 Accept this contribution > > >>>>>>>>>> [ ] -1 Reject because... > > >>>>>>>>>> > > >>>>>>>>>> As usual, 3 days unless someone complains they need more > > >> time. > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> -- > > >>>>>>>>> --vvl > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> With best regards, > > >>>>>>>> Alexei, > > >>>>>>>> ESSD, Intel > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> --vvl > > >>>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> With best regards, > > >>>>>> Alexei, > > >>>>>> ESSD, Intel > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> --vvl > > >>>>> > > >>>> > > >>>> -- > > >>>> With best regards, > > >>>> Alexei, > > >>>> ESSD, Intel > > >>>> > > >>> > > >>> > > >>> -- > > >>> --vvl > > >>> > > >> > > >> -- > > >> With best regards, > > >> Alexei, > > >> ESSD, Intel > > >> > > > > > > > > > > > > > > > -- > --vvl > -- With best regards, Alexei, ESSD, Intel
