Hi Gunnar,
>> I would have thought that changing the value and gradient of the
>> target function had the potential to alter the minima?
>
> Indeed, the target function is changed during the search, but
> once a stable minimum is found, the DEN potential needs to
> be zero by definition and the coordinates have to sit in a minimum
> of the original target function.
Yes, I believe both statements are correct - I was referring to the nature of
the function during the procedure, and you refer to the nature after asymptotic
convergence. The nature of the target function and location of the minima are
changed during DEN refinement - at any given time step (before asymptotic
convergence) the minima of the target function may lie in locations different
to the original target function. However, upon convergence, dij ~= Dij
regardless of particular parameter values. With DEN, the target function
changes shape throughout the procedure, but results in the target function
being asymptotically equal to the original, thus refinement converges to a
minima of the original target function.
This behaviour is interesting and notable because it is different to other
terms used in refinement. Generally, prior information (e.g. geometry terms,
external structural information) in the form of restraints is determined
externally and remains static during refinement. These determine the nature of
the target function, but do not change it during refinement - this contrasts
with the DEN approach. Of course, the structure factors are also updated and
thus also alter the nature of the target function during refinement. Just some
interesting observations!
>> Here are my thoughts: since the DEN update formula is recursive, the
>> equilibrium distance can also be written in > terms of the Dij alone (still
>> assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial
>> distance Dij(0) for all times t. …
>
> I hope I do not get you wrong, but with this argument
> aren't you just saying that the path/trajectory (of both the atomic
> coordinates
> and the DEN potential) depends on the starting point?
No, this wasn't quite the point I was trying to make. For sure, we all
trivially know that any path during refinement depends on the starting point of
the parameter values.
However, it is interesting that the DEN restraint target (or DEN potential, or
equilibrium distance) depends on the starting point. Moreover, the DEN
restraint target can be expressed in a form that makes this dependency
explicit. This observation is not trivial, as it differs from other approaches.
As above, it is most interesting to acknowledge that this contrasts with other
terms used in refinement. For example, simple geometry/external restraints
representing prior information always remain static during refinement. At time
t, other restraints do not depend on their value at time t-1, and thus do not
depend on their value at time 0. Rather, they are pre-determined before
refinement begins.
> The important point is that the decision on how to move the DEN
> minimum from one iteration (at time t) to the next (at time t+1)
> depends only on where the atoms are at t+1 and where the DEN minimum was
> at time t.
Or equivalently, the decision on how to move the DEN minimum from one iteration
(at time t) to the next (at time t+1) depends on where the atoms are at t+1,
where they were at time t, where they were at time t-1, … , where they were at
time 0. Of course, the degree of dependency on distant history is controlled by
kappa. Very low values of kappa will result in DEN "remembering" more distant
historical values of the interatomic distance, and thus refinement will take
longer to converge. Very high values of kappa will result in DEN being
dependent only on the immediate history, and thus will have little effect on
refinement.
> If we assume that there is a second starting point which results
> in a minimization path that happens to cross exactly the path from the first
> starting point (same atomic coordinates and same position of DEN minimum)
> at some time t'. Then the new position of the DEN minimum at time t'+1 would
> be
> exactly at the same position that you get from the first path at time t+1.
Of course, if a second minimisation path happens to cross exactly the first
minimisation path, then they would both end up with the same final result.
There would be something wrong if they didn't! The property that two paths
within some neighbourhood of each other both converge to the same final
positions is a simple requirement for refinement robustness. Just to clarify, I
certainly did not make any incorrect/unsupported claims that DEN is not robust.
I was merely investigating the exact nature of the technique.
Interestingly, note that DEN requires both the atomic coordinates to be at the
same position AND the DEN minimum to be at the same position in order for this
argument to hold. This is a much stronger condition than the analogous
requirement without DEN, which would require only that the atomic coordinates
be near the same positions in order to achieve in the same final result.
If two paths from different starting values cross such that their atomic
coordinates are within some neighbourhood of each other, then one would hope
that the DEN potentials would adjust themselves (without taking too many
refinement cycles, which would depend on kappa) so that they too become within
some neighbourhood of each other, hopefully resulting in them both converging
to the same local minima.
> This shows that the DEN update does not depend on the starting point.
No, I believe that the DEN update does depend on the starting point. However,
kappa can be set sufficiently large so that the dependency on the starting
point is very low. The higher dependency on more recent historical distances
does indeed allow the entire conformational space to be accessible, but there
is still a dependency on the starting point, technically speaking. True, upon
convergence, this dependency on the starting point should be negligible.
Logically, this will result in a reduced rate of convergence during refinement,
but hopefully with the implicit benefit of increased robustness from
regularisation.
Thanks for the interesting discussion! It is good to dissect these techniques,
and hopefully it will be useful for some users out there who want to know
exactly what is going on!
Cheers,
Rob
On 2 Sep 2012, at 20:15, Gunnar Schroeder wrote:
> Hi Rob,
>
>>> This also means that the position of the minima of the target function
>>> are not changed by the DEN (gamma=1) restraints.
>
>> I would have thought that changing the value and gradient of the
>> target function had the potential to alter the minima?
>
> Indeed, the target function is changed during the search, but
> once a stable minimum is found, the DEN potential needs to
> be zero by definition and the coordinates have to sit in a minimum
> of the original target function.
>
> First we note that if the DEN potential minimum is at the same position
> as the atomic coordinates, the potential and the first derivative are zero.
>
> Assume the atoms are at a stable minimum of the combined energy function
> (original target function + DEN potential with gamma=1) AND the DEN potential
> minimum is different from the atomic positions. Then the DEN potential
> minimum would move towards the atomic positions, which would change the
> combined energy function and its derivative. The atoms would not be in a
> stable minimum anymore, which contradicts the assumption and proofs that
> the DEN potential is always zero if the atoms are in a stable minimium of the
> combined energy function.
>
>
>> Here are my thoughts: since the DEN update formula is recursive, the
>> equilibrium distance can also be written in > terms of the Dij alone (still
>> assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial
>> distance Dij(0) for all times t. …
>
> I hope I do not get you wrong, but with this argument
> aren't you just saying that the path/trajectory (of both the atomic
> coordinates
> and the DEN potential) depends on the starting point?
> Every simulation/minimization depends on the starting point.
> In a steepest descent minimization the step size determines
> how long it takes to move away from the starting point, just like
> the parameter kappa determines how long it takes for the DEN potential
> and the atomic coordinates to move away from the starting
> model. I do not see the difference? Am I missing something here?
>
> The important point is that the decision on how to move the DEN
> minimum from one iteration (at time t) to the next (at time t+1)
> depends only on where the atoms are at t+1 and where the DEN minimum was
> at time t. If we assume that there is a second starting point which results
> in a minimization path that happens to cross exactly the path from the first
> starting point (same atomic coordinates and same position of DEN minimum)
> at some time t'. Then the new position of the DEN minimum at time t'+1 would
> be
> exactly at the same position that you get from the first path at time t+1.
> This
> shows that the DEN update does not depend on the starting point.
>
> Cheers,
> Gunnar
>
>
>
> PS: Just for the record, here we only discuss DEN refinement for gamma=1.
>
>
> On Aug 31, 2012, at 11:30 AM, Robert Nicholls wrote:
>
>> Hi Gunnar,
>>
>> I generally agree with your comments. However, I'd like to clarify a couple
>> of points:
>>
>>> For gamma=1 the DEN potential can follow anywhere, the entire
>>> conformational
>>> space is accessible and dij(t+1) depends only on Dij(t) and dij(t).
>> ...
>>> But, again, the starting (or reference)
>>> model is completely forgotten and never used after the first iteration.
>>
>>
>> Certainly, the entire conformational space is accessible. However, I'm not
>> so sure about the starting model being completely forgotten and never used
>> after the first iteration. Here are my thoughts: since the DEN update
>> formula is recursive, the equilibrium distance can also be written in terms
>> of the Dij alone (still assuming gamma=1):
>> dij(t+1) = Dij(0)*(1-kappa)^(t+1) + kappa*sum_n=0^t{Dij(t+1-n)*(1-kappa)^n}
>> This means that the equilibrium distance is indeed dependent on the initial
>> distance Dij(0) for all times t. For values of kappa in (0,1), this
>> dependency will diminish with time t, but will always exist. In fact, the
>> equilibrium distance dij(t) is dependent on the whole history of the
>> distance throughout the procedure, i.e. Dij(n) for n=0…t. Of course, the
>> degree of influence of the historical information is controlled by kappa.
>> Values of kappa~=0 would mean that the initial distance has very high weight
>> (equilibrium distance dij(t) = Dij(0) in the limit kappa=0), and kappa~=1
>> would mean that the most recent distances have very high weight (equilibrium
>> distance dij(t) = Dij(t) in the limit kappa=1, as you have already stated).
>> Intermediate values of kappa will give various non-zero weights to the
>> historical values of Dij.
>>
>>> This also means that the position of the minima of the target function
>>> are not changed by the DEN (gamma=1) restraints.
>>
>>
>> I would have thought that changing the value and gradient of the target
>> function had the potential to alter the minima?
>>
>>> It is therefore usually useful to run a final minimization without
>>> restraints to test whether the refinement reached a stable minimum of the
>>> target function.
>>
>> I agree. In the context of REFMAC5, my current favourite strategy at low
>> resolution is to first use external restraints in order to aid the structure
>> to adopt a more sensible conformation, but then subsequently release the
>> external restraints and replace them with jelly-body restraints towards the
>> final refinement stages.
>>
>>> From the user perspective, I think the main difference is that DEN is
>>> designed
>>> to be used in simulated annealing MD refinement, whereas jelly-body is
>>> designed
>>> to be used in minimization (and cannot be used for MD refinement as there
>>> are
>>> no second derivatives).
>>
>> I agree. Since the second derivative is utilised in ML refinement, it is
>> possible to design a regulariser that has the desirable properties X=0 and
>> X'=0 (e.g. jelly-body refinement) in the absence of any externally-derived
>> prior information. Since this is not possible in simulated annealing MD
>> refinement, the analogous solution will undoubtedly have to alter X and/or
>> X'. Either way, all of these 'tricks' are just designed to aid robustness
>> and combat overfitting! Certainly, both approaches can give positive results
>> when refining at low resolution.
>>
>> Cheers
>> Rob
>>
>
> --
> _____________________________________________________________
> Gunnar F. Schröder
> Computational Structural Biology Group
> Institute of Complex Systems (ICS-6)
> Forschungszentrum Jülich
>
> tel +49-2461-61-3259
> http://www.schroderlab.org
