On 10/16/11 8:01 AM, Luc Maisonobe wrote:
> Le 16/10/2011 09:24, Phil Steitz a écrit :
>> On 10/9/11 7:46 AM, Phil Steitz wrote:
>>> On 10/9/11 5:39 AM, Luc Maisonobe wrote:
>>>> Hi Phil,
>>>>
>>>> Le 08/10/2011 23:42, Phil Steitz a écrit :
>>>>> On 10/8/11 2:24 PM, Luc Maisonobe wrote:
>>>>>>
>>>>>> Phil Steitz<phil.ste...@gmail.com> a écrit :
>>>>>>
>>>>>>> I am getting RTE with message above when I try to run the
>>>>>>> example
>>>>>>> under "updating the base and differentiated objects" in the
>>>>>>> docs.
>>>> Digging into the code, here are the bytecode operations that are
>>>> not supported yet:
>>>>
>>>> DALOAD, DASTORE:
>>>> element access in double arrays
>>>> GETSTATIC, PUTSTATIC, GETFIELD, PUTFIELD:
>>>> field access (instance fields and class fields)
>>>> INVOKEVIRTUAL/INVOKESPECIAL/INVOKESTATIC/INVOKEINTERFACE:
>>>> method calls
>>>> NEWARRAY/ANEWARRAY/MULTIANEWARRAY:
>>>> array creation
>>>>
>>>>>>> Is this example supposed to work with the code in trunk? Also,
>>>>>>> I am
>>>>>> I'll look at this tomorrow, but I think for now you need to have
>>>>>> a standalone function, it cannot be split
>>>>>> as a main function calling subfunctions. The only allowed calls
>>>>>> are the static methods from Math/StrictMath.
>>>>>> I did not add our own FastMath, but it is trivial to do.
>>>>>>
>>>>>> Another limitation is that your function cannot store
>>>>>> intermediate results as clas attributes yet.
>>>>> Thanks, Luc! What I was trying to illustrate was partial
>>>>> derivatives, which IIUC you need something like that example
>>>>> to do.
>>>>> The following almost works:
>>>>>
>>>>> public void testPartialDerivatives() throws Exception {
>>>>> PartialFunction function = new PartialFunction(1);
>>>>> final UnivariateDerivative derivative = new
>>>>> ForwardModeAlgorithmicDifferentiator().differentiate(function);
>>>>> DifferentialPair t = DifferentialPair.newVariable(1);
>>>>> Assert.assertEquals(3,
>>>>> derivative.f(t).getFirstDerivative(), 0);
>>>>> Assert.assertEquals(2, derivative.f(t).getValue(), 0);
>>>>> function.setX(2);
>>>>> Assert.assertEquals(4,
>>>>> derivative.f(t).getFirstDerivative(), 0);
>>>>> Assert.assertEquals(3, derivative.f(t).getValue(), 0);
>>>>> }
>>>>>
>>>>> with
>>>>>
>>>>> public class PartialFunction implements
>>>>> UnivariateDifferentiable {
>>>>> private double x;
>>>>> public PartialFunction(double x) {
>>>>> this.x = x;
>>>>> }
>>>>> public void setX(double x) {
>>>>> this.x = x;
>>>>> }
>>>>> public double getX() {
>>>>> return x;
>>>>> }
>>>>> public double f(double y) {
>>>>> return x * y + y * y;
>>>>> }
>>>>> }
>>>>>
>>>>> But I end up with java.lang.VerifyError: (class:
>>>>> ExampleTest$1PartialFunction$NablaForwardModeUnivariateDerivative,
>>>>>
>>>>> method: f signature:
>>>>> (Lorg/apache/commons/nabla/core/DifferentialPair;)Lorg/apache/commons/nabla/core/DifferentialPair;)
>>>>>
>>>>>
>>>>> Incompatible type for getting or setting field
>>>>> at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>>> at
>>>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
>>>>> at java.lang.Class.getDeclaredConstructors(Class.java:1836)
>>>>> at
>>>>> org.apache.commons.nabla.algorithmic.forward.ForwardModeAlgorithmicDifferentiator.differentiate(ForwardModeAlgorithmicDifferentiator.java:107)
>>>>>
>>>>>
>>>>> at ExampleTest.testPartialDerivatives(ExampleTest.java:66)
>>>> This error seems to be due to the lack of support for the GETFIELD
>>>> instruction. As x is an instance field, the f method reads this
>>>> field before multiplying the result.
>>
>> Right. As an exercise to help me understand the internals of the
>> code, I have been trying to figure out how to add this support.
>> Working backwards from the generated bytecode, the GETFIELD and the
>> ALOAD 0 before it seem to get copied unchanged:
>>
>> ALOAD 0
>> GETFIELD PartialFunction.x : D
>>
>> Working backwards to where it should get changed,
>> MethodDifferentiator#getReplacement throws RuntimeException when it
>> sees a GETFIELD; but in my example it does not throw. This means
>> the instruction is not making it into the changes set. The question
>> then is a) how to modify MethodDifferentiator#identifyChanges to
>> identify the need to change the instruction
>
> Changes are deduced from a data flow analysis. We start at method
> entry, knowing that the original method signature was
>
> public double f(double x);
>
> and that the transformed method signature is
>
> public DifferentialPair f(DifferentialPair x);
>
> So on entry, the first (and only) parameter is changed from double
> to DifferentialPair.
>
> From this starting point, we propagate types, using
> differentiation rules, i.e. when we see that an instruction like
> "add" has a DifferentialPair at least in one of its arguments,
> then its result must be a DifferentialPair. At the end of the data
> flow analysis, we have identified all instructions that either
> consume or produce DifferentialPair instances, these instructions
> must be changed as in the original method they did consume or
> produce double number instead.
>
>> and b) how to transform
>> it (and other instructions that depend on it).
>
> Transform may depend on context. For example, if the field is only
> read in the method and never set (which is the case in this
> example, as the method computes x * y + y * y where x is the field
> and y is the method parameter), then this field should remain a
> simple double and the instruction using it should be changed from
> multiplication of two doubles (x * y) to a multiplication of a
> double and a DifferentialPair. If the field were also set (for
> example using setX(y * y)), then it would be the result of an
> instruction producing a DifferentialPair and an additional field
> should be set up in the generated class. We could either choose to
> set a complete "private DifferentialPair xNabla" containing both
> the value and the derivative, with some code to make sure the x
> field from the enclosing class is kept in sync, or we could simply
> use a "private double xNabla" for the derivative and still use the
> x from the enclosing class for the value, hence avoiding sync code.
If possible, I would think it best to avoid the sync code. This
would also eliminate the need to check for changes made by other
clients to the primitive field.
>
>> A reference to the
>> enclosing class also has to be made available. Is that already
>> there somewhere?
>
> Yes, there is a "primitive" field in the generated class that
> points to the original instance. This field is the one returned by
> the automatically generated "getPrimitive" method.
Duh...Should have seen that. Thanks. Quick bytecode question.
Here is the generated init for the derivative class, showing where
the primitive field is set:
public <init>(LPartialFunction;)V
ALOAD 0
INVOKESPECIAL java/lang/Object.<init> ()V
ALOAD 0
ALOAD 1
PUTFIELD
PartialFunction$NablaForwardModeUnivariateDerivative.primitive :
LPartialFunction;
I am a novice when it comes to bytecode analysis. From the above,
it looks like the enclosing class is available using ALOAD 1. Is
this always the case? If that is true, then might the incorrect
field access bytecode above be fixed just by changing 0 to 1 in the
ALOAD?
>
>
> I think there is a problem in the data flow analysis, in the
> TrackingValue class. It's "merge" is never populated with
> anything. I will fix this.
>
> I will also try to improve the debugging prints I have added a few
> days ago (creating a new class for it). It should also display the
> status of local variables and stack, as these are the main drivers
> for the data flow analysis and it is probably where the error lies
> here.
Thanks, Luc. I am starting to understand how things work and I have
to say there are some really nice ideas in here. Regarding the data
flow analysis, at first I thought there was no way that the GETFIELD
could get picked up because the data flow analysis only looks at t;
but I can see now that assuming the field is actually used, the
instruction should get picked up. I will look at this some more.
Regarding debugging, what I have found useful is dumping the
original and derived bytecode. I have started dumping the changes
set too to understand the data flow analysis.
One final sort of philosophical question. As you point out in the
docs, one could actually build up DifferentialPairs using the
primitives defined on them. These are not used internally and you
remark that using this method to construct things might be error
prone. A hybrid model where (lets call them non-degenerate)
DifferentialPairs are provided as arguments to derived functions is
also possible. The derivatives will be correctly evaluated and
applied according to the chain rule. Is this the kind of thing you
had in mind and, if so, what kinds of applications would use this?
Phil
>
> Luc
>
>>
>> Phil
>>>>
>>>> I have added a debug display message (to be removed later on) that
>>>> should print the generated bytecode to standard error when a
>>>> VerifyError exception occurs. It' clearly not targeted towards end
>>>> users, but it could help during development.
>>> Thanks, Luc!
>>>
>>> Phil
>>>> Luc
>>>>
>>>>>>
>>>>>> You can look at the junit tests for what is supported. Simple
>>>>>> expressions, calls to traditional functions like sin, cos,
>>>>>> exp ...,
>>>>>> Simple loops and conditionals, local automatic variables should
>>>>>> all work (I hope ...)
>>>>> Yep, I have gotten all of this to work. Even "knows" the chain
>>>>> rule :)
>>>>>
>>>>>
>>>>> Phil
>>>>>>> assuming
>>>>>>> s/ForwardAlgorithmicDifferentiator/ForwardModeAlgorithmicDifferentiator
>>>>>>>
>>>>>>>
>>>>>>> throughout. Correct?
>>>>>> Yes, the name was changed because a distant goal will be to also
>>>>>> support reverse mode, which is especially
>>>>>> useful when computing gradients (i.e. when one scalar function
>>>>>> depends on many inputs and we want all partial
>>>>>> derivatives).
>>>>>>
>>>>>> Luc
>>>>>>
>>>>>>> Phil
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>
>>>>>
>>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>>
>>>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>>>> For additional commands, e-mail: dev-h...@commons.apache.org
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>>
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
