Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
Fine! I've started playing with the code. I hope to provide something working in about 2 weeks. I'll probably have more questions meanwhile :) Thanks, Mikhail 2007/11/8, Asaf Yaffe <[EMAIL PROTECTED]>: > Hi Mikhail, > > You point makes sense. I agree that for start we don't have to support > arbitrary instrumentations, and as far as I know neither the TPTP profiler > nor Probekit alters control flow in a significant way (basically, they add > exception handlers). They also make minimal code additions, mainly in the > form of invoking some external static methods for the purpose of recording > information (method enter/leave, object allocations, exceptions, method > arguments, etc). > > Bottom line: we can definitely reuse information from the existing > StackMapTable to create the new one. > > Thanks, > Asaf > > > - Original Message > From: Mikhail Loenko <[EMAIL PROTECTED]> > To: [email protected] > Sent: Wednesday, November 7, 2007 3:28:32 PM > Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute without > loading the classes > > 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > > Hi Mikhail, > > > > Thinking about it, my last statement does not make any sense > (*blush*). Please ignore it. > > > > To summarize this point: I am still not sure whether using the > existing StackMapTable (before instrumentation) will solve the "class load" > problem when verifying the instrumented code. > > > So I'll try to explain my point: > I assume that if original class was broken or instrumentation broke the > class > then it's OK to generate invalid stackmap table so that verifier of > the JVM which > will execute the class wil capture that. (that verifier will be able > to load all the necessary classes) > > If you take original file and stackmap and try to verify that, you'll > have > to check some pairs of classes for assignability. We can assume that > original > class file was correct and all the pair are good in terms of > assignability. (if we are wrong then we'll build incorrect stackmap > and verifier will decline the class) So we can build a set of > 'trusted' pairs. > > > Then you make instrumentation of the class. > Of course you can modify it in a random way: just add refrerences to > random classes there or modify control flow in an arbitrary way. In a > general case it might happen that we won't be able to generate > stackmap until we load the classes. > > But in real life you probably don't hack the class in a random way: > if your instrumented class assumes that some class A is assignable to > another > class B then you made this guess either because original class file > implied that or > because A and B are instrumentor-internal classes and you just know > that. > > If original class implied that A is assignable to B, then we will know > it from > the information captured as I described earlier ('trusted' pairs). > > If there are cases when you add new classes (which instrumentor knows > about): we may either require that they can be obtained by > implementation of cl_get_class() or I could provide interface to > specify instrumentor-specific set of 'trusted' pairs. > > If neither original class implied that assignability nor you just know > that for sure then it's probably an incorrect instrumentation... > > Thanks, > Mikhail > > > > > Thanks, > > Asaf > > > > > > - Original Message > > From: Mikhail Loenko <[EMAIL PROTECTED]> > > To: [email protected] > > Sent: Wednesday, November 7, 2007 12:35:36 PM > > Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute > without loading the classes > > > > > > Hi Asaf, > > > > 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > > > Hi Mikhail, > > > > > > The existing StackMap attribute (before instrumentation) can help > > only for the code that was part of the method *before* is was > > instrumented. It cannot help with code *added* by instrumentation. > > > > I did not quite understand this. Do you mean information about the > > classes references to which are added by instrumentation, or > something > > else? Could you please provide an example? > > > > Thanks, > > Mikhail > > > > > > > > > > > > > > Therefore I see two options here: > > > > > > 1. Find a way for the instrumentor to annotate the the added code > > with enough data so that the verifier can use this info
Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
Hi Mikhail, You point makes sense. I agree that for start we don't have to support arbitrary instrumentations, and as far as I know neither the TPTP profiler nor Probekit alters control flow in a significant way (basically, they add exception handlers). They also make minimal code additions, mainly in the form of invoking some external static methods for the purpose of recording information (method enter/leave, object allocations, exceptions, method arguments, etc). Bottom line: we can definitely reuse information from the existing StackMapTable to create the new one. Thanks, Asaf - Original Message From: Mikhail Loenko <[EMAIL PROTECTED]> To: [email protected] Sent: Wednesday, November 7, 2007 3:28:32 PM Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > Hi Mikhail, > > Thinking about it, my last statement does not make any sense (*blush*). Please ignore it. > > To summarize this point: I am still not sure whether using the existing StackMapTable (before instrumentation) will solve the "class load" problem when verifying the instrumented code. So I'll try to explain my point: I assume that if original class was broken or instrumentation broke the class then it's OK to generate invalid stackmap table so that verifier of the JVM which will execute the class wil capture that. (that verifier will be able to load all the necessary classes) If you take original file and stackmap and try to verify that, you'll have to check some pairs of classes for assignability. We can assume that original class file was correct and all the pair are good in terms of assignability. (if we are wrong then we'll build incorrect stackmap and verifier will decline the class) So we can build a set of 'trusted' pairs. Then you make instrumentation of the class. Of course you can modify it in a random way: just add refrerences to random classes there or modify control flow in an arbitrary way. In a general case it might happen that we won't be able to generate stackmap until we load the classes. But in real life you probably don't hack the class in a random way: if your instrumented class assumes that some class A is assignable to another class B then you made this guess either because original class file implied that or because A and B are instrumentor-internal classes and you just know that. If original class implied that A is assignable to B, then we will know it from the information captured as I described earlier ('trusted' pairs). If there are cases when you add new classes (which instrumentor knows about): we may either require that they can be obtained by implementation of cl_get_class() or I could provide interface to specify instrumentor-specific set of 'trusted' pairs. If neither original class implied that assignability nor you just know that for sure then it's probably an incorrect instrumentation... Thanks, Mikhail > > Thanks, > Asaf > > > - Original Message > From: Mikhail Loenko <[EMAIL PROTECTED]> > To: [email protected] > Sent: Wednesday, November 7, 2007 12:35:36 PM > Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes > > > Hi Asaf, > > 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > > Hi Mikhail, > > > > The existing StackMap attribute (before instrumentation) can help > only for the code that was part of the method *before* is was > instrumented. It cannot help with code *added* by instrumentation. > > I did not quite understand this. Do you mean information about the > classes references to which are added by instrumentation, or something > else? Could you please provide an example? > > Thanks, > Mikhail > > > > > > > Therefore I see two options here: > > > > 1. Find a way for the instrumentor to annotate the the added code > with enough data so that the verifier can use this information without the > need to load classes > > > > 2. Delay the entire StackMap computation process to the point where > JNI is available (i.e., when VM_Start JVMTI event fires) and use > RedefineClasses to instrument the (bootstrap) classes already loaded. > > > > The main issue I see with Option 2 is that RedefineClasses is not a > mature feature in most JVM out there (some cannot redefine certain > bootstrapping classes at all), and relying on this feature will hurt the > robustness of our tools. > > > > Any thoughts about option 1? Maybe other suggestions? > > > > Thanks, > > Asaf > > > > - Original Message > > From: Mikhail Loenko <[EMAIL PROTECTED]> > > To: [email protected]
Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>:
> Hi Mikhail,
>
> Thinking about it, my last statement does not make any sense (*blush*).
> Please ignore it.
>
> To summarize this point: I am still not sure whether using the existing
> StackMapTable (before instrumentation) will solve the "class load" problem
> when verifying the instrumented code.
So I'll try to explain my point:
I assume that if original class was broken or instrumentation broke the class
then it's OK to generate invalid stackmap table so that verifier of
the JVM which
will execute the class wil capture that. (that verifier will be able
to load all the necessary classes)
If you take original file and stackmap and try to verify that, you'll have
to check some pairs of classes for assignability. We can assume that original
class file was correct and all the pair are good in terms of
assignability. (if we are wrong then we'll build incorrect stackmap
and verifier will decline the class) So we can build a set of
'trusted' pairs.
Then you make instrumentation of the class.
Of course you can modify it in a random way: just add refrerences to
random classes there or modify control flow in an arbitrary way. In a
general case it might happen that we won't be able to generate
stackmap until we load the classes.
But in real life you probably don't hack the class in a random way:
if your instrumented class assumes that some class A is assignable to another
class B then you made this guess either because original class file
implied that or
because A and B are instrumentor-internal classes and you just know that.
If original class implied that A is assignable to B, then we will know it from
the information captured as I described earlier ('trusted' pairs).
If there are cases when you add new classes (which instrumentor knows
about): we may either require that they can be obtained by
implementation of cl_get_class() or I could provide interface to
specify instrumentor-specific set of 'trusted' pairs.
If neither original class implied that assignability nor you just know
that for sure then it's probably an incorrect instrumentation...
Thanks,
Mikhail
>
> Thanks,
> Asaf
>
>
> - Original Message
> From: Mikhail Loenko <[EMAIL PROTECTED]>
> To: [email protected]
> Sent: Wednesday, November 7, 2007 12:35:36 PM
> Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute without
> loading the classes
>
>
> Hi Asaf,
>
> 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>:
> > Hi Mikhail,
> >
> > The existing StackMap attribute (before instrumentation) can help
> only for the code that was part of the method *before* is was
> instrumented. It cannot help with code *added* by instrumentation.
>
> I did not quite understand this. Do you mean information about the
> classes references to which are added by instrumentation, or something
> else? Could you please provide an example?
>
> Thanks,
> Mikhail
>
>
>
>
>
>
> Therefore I see two options here:
> >
> > 1. Find a way for the instrumentor to annotate the the added code
> with enough data so that the verifier can use this information without the
> need to load classes
> >
> > 2. Delay the entire StackMap computation process to the point where
> JNI is available (i.e., when VM_Start JVMTI event fires) and use
> RedefineClasses to instrument the (bootstrap) classes already loaded.
> >
> > The main issue I see with Option 2 is that RedefineClasses is not a
> mature feature in most JVM out there (some cannot redefine certain
> bootstrapping classes at all), and relying on this feature will hurt the
> robustness of our tools.
> >
> > Any thoughts about option 1? Maybe other suggestions?
> >
> > Thanks,
> > Asaf
> >
> > - Original Message
> > From: Mikhail Loenko <[EMAIL PROTECTED]>
> > To: [email protected]
> > Sent: Tuesday, November 6, 2007 11:40:09 AM
> > Subject: [drlvm][verifier] rebuilding stackmaptable attribute without
> loading the classes
> >
> >
> > 2007/11/6, Asaf Yaffe <[EMAIL PROTECTED]>:
> > > 2. Class Load issues (as described in my previous post): are there
> > any options for running the verifier when classes cannot be loaded
> as
> > necessary? Please open a separate mail thread for this (prefixed
> with
> > [drlvm][verifier]).
> >
> >
> > If we have a point where e.g. two types are comming like class A and
> > class B
> > and the differrent successor instruction expect to see there two
> other
> > classes
> > like class C and class D, then for the stackmap we should place such
> a
Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
Hi Mikhail, Thinking about it, my last statement does not make any sense (*blush*). Please ignore it. To summarize this point: I am still not sure whether using the existing StackMapTable (before instrumentation) will solve the "class load" problem when verifying the instrumented code. Thanks, Asaf - Original Message From: Mikhail Loenko <[EMAIL PROTECTED]> To: [email protected] Sent: Wednesday, November 7, 2007 12:35:36 PM Subject: Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes Hi Asaf, 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > Hi Mikhail, > > The existing StackMap attribute (before instrumentation) can help only for the code that was part of the method *before* is was instrumented. It cannot help with code *added* by instrumentation. I did not quite understand this. Do you mean information about the classes references to which are added by instrumentation, or something else? Could you please provide an example? Thanks, Mikhail Therefore I see two options here: > > 1. Find a way for the instrumentor to annotate the the added code with enough data so that the verifier can use this information without the need to load classes > > 2. Delay the entire StackMap computation process to the point where JNI is available (i.e., when VM_Start JVMTI event fires) and use RedefineClasses to instrument the (bootstrap) classes already loaded. > > The main issue I see with Option 2 is that RedefineClasses is not a mature feature in most JVM out there (some cannot redefine certain bootstrapping classes at all), and relying on this feature will hurt the robustness of our tools. > > Any thoughts about option 1? Maybe other suggestions? > > Thanks, > Asaf > > - Original Message > From: Mikhail Loenko <[EMAIL PROTECTED]> > To: [email protected] > Sent: Tuesday, November 6, 2007 11:40:09 AM > Subject: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes > > > 2007/11/6, Asaf Yaffe <[EMAIL PROTECTED]>: > > 2. Class Load issues (as described in my previous post): are there > any options for running the verifier when classes cannot be loaded as > necessary? Please open a separate mail thread for this (prefixed with > [drlvm][verifier]). > > > If we have a point where e.g. two types are comming like class A and > class B > and the differrent successor instruction expect to see there two other > classes > like class C and class D, then for the stackmap we should place such a > class > X so that A and B are assignsable to X and X is assignable to C and D. > > If all classes might be loaded (like in case of javac who usually > generates stackmap table attribute) than it's an easy task. > > If nethier of A, B, C, and D might be loaded then the task becomes a > bit challenging :) > > What comes to my mind is if we have code and stackmap of the original > class (before instrumentation), we could extract some info from there. > > For example if original class implies that A is assignable to B, then > we could choose X=B as a solution for the above > > Thanks, > Mikhail > > > > > > __ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
Hi Asaf, 2007/11/7, Asaf Yaffe <[EMAIL PROTECTED]>: > Hi Mikhail, > > The existing StackMap attribute (before instrumentation) can help only for > the code that was part of the method *before* is was instrumented. It cannot > help with code *added* by instrumentation. I did not quite understand this. Do you mean information about the classes references to which are added by instrumentation, or something else? Could you please provide an example? Thanks, Mikhail Therefore I see two options here: > > 1. Find a way for the instrumentor to annotate the the added code with enough > data so that the verifier can use this information without the need to load > classes > > 2. Delay the entire StackMap computation process to the point where JNI is > available (i.e., when VM_Start JVMTI event fires) and use RedefineClasses to > instrument the (bootstrap) classes already loaded. > > The main issue I see with Option 2 is that RedefineClasses is not a mature > feature in most JVM out there (some cannot redefine certain bootstrapping > classes at all), and relying on this feature will hurt the robustness of our > tools. > > Any thoughts about option 1? Maybe other suggestions? > > Thanks, > Asaf > > - Original Message > From: Mikhail Loenko <[EMAIL PROTECTED]> > To: [email protected] > Sent: Tuesday, November 6, 2007 11:40:09 AM > Subject: [drlvm][verifier] rebuilding stackmaptable attribute without loading > the classes > > > 2007/11/6, Asaf Yaffe <[EMAIL PROTECTED]>: > > 2. Class Load issues (as described in my previous post): are there > any options for running the verifier when classes cannot be loaded as > necessary? Please open a separate mail thread for this (prefixed with > [drlvm][verifier]). > > > If we have a point where e.g. two types are comming like class A and > class B > and the differrent successor instruction expect to see there two other > classes > like class C and class D, then for the stackmap we should place such a > class > X so that A and B are assignsable to X and X is assignable to C and D. > > If all classes might be loaded (like in case of javac who usually > generates stackmap table attribute) than it's an easy task. > > If nethier of A, B, C, and D might be loaded then the task becomes a > bit challenging :) > > What comes to my mind is if we have code and stackmap of the original > class (before instrumentation), we could extract some info from there. > > For example if original class implies that A is assignable to B, then > we could choose X=B as a solution for the above > > Thanks, > Mikhail > > > > > > __ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com
Re: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes
Hi Mikhail, The existing StackMap attribute (before instrumentation) can help only for the code that was part of the method *before* is was instrumented. It cannot help with code *added* by instrumentation. Therefore I see two options here: 1. Find a way for the instrumentor to annotate the the added code with enough data so that the verifier can use this information without the need to load classes 2. Delay the entire StackMap computation process to the point where JNI is available (i.e., when VM_Start JVMTI event fires) and use RedefineClasses to instrument the (bootstrap) classes already loaded. The main issue I see with Option 2 is that RedefineClasses is not a mature feature in most JVM out there (some cannot redefine certain bootstrapping classes at all), and relying on this feature will hurt the robustness of our tools. Any thoughts about option 1? Maybe other suggestions? Thanks, Asaf - Original Message From: Mikhail Loenko <[EMAIL PROTECTED]> To: [email protected] Sent: Tuesday, November 6, 2007 11:40:09 AM Subject: [drlvm][verifier] rebuilding stackmaptable attribute without loading the classes 2007/11/6, Asaf Yaffe <[EMAIL PROTECTED]>: > 2. Class Load issues (as described in my previous post): are there any options for running the verifier when classes cannot be loaded as necessary? Please open a separate mail thread for this (prefixed with [drlvm][verifier]). If we have a point where e.g. two types are comming like class A and class B and the differrent successor instruction expect to see there two other classes like class C and class D, then for the stackmap we should place such a class X so that A and B are assignsable to X and X is assignable to C and D. If all classes might be loaded (like in case of javac who usually generates stackmap table attribute) than it's an easy task. If nethier of A, B, C, and D might be loaded then the task becomes a bit challenging :) What comes to my mind is if we have code and stackmap of the original class (before instrumentation), we could extract some info from there. For example if original class implies that A is assignable to B, then we could choose X=B as a solution for the above Thanks, Mikhail __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
