Olivier Delalleau wrote:
> With the current implementation of PRNGs, isn't there a risk to see the
> behavior of some component change because we added or removed some
> component somewhere else in the graph?
> It seems to me this is the case, which is potentially dangerous because
> of the risk of doing e.g. different splits / weight initializations
> without being aware of it.
This "danger" is inherent with any pseudo-random number generator; the
PRNG class just makes it easier to avoid it.  As we have already
discussed, this class was designed to ensure that parallelized code will
behave in a deterministic way and yield the exact same output as a
sequential run.  It was not designed to "do what I mean".

In most cases, the addition or removal of components in the graph of
components will not modify the shape of the tree of PRNGs (those are two
different graphs.)  Also, replacing a component with another one will
have absolutely no effect on other components' PRNGs (would not be true
with a single global generator.)  And, it allows you to test with
different split and weight initializations by simply changing the root
PRNG's seed while avoiding an accidental overlap of generated numbers,
as would be the case with several unrelated manually-initialized
generators; overlap is also avoided when building complex models from
parts of other, already existing models.  In any case, you need to
understand what is happening within your model to avoid bad surprises.

If you know of a better way of dealing with potentially concurrent
deterministic pseudo-random number generators, please tell me.  As I
said in our previous discussion, I believe that the biggest possible
improvement to these PRNGs would be to implement Pierre L'Ecuyer's
jumpahead to calculate a new seed from an existing one instead of using
a Numpy-supplied hashing algorithm; the "danger" you mention would still
be present.


P.S. seat belts don't make anyone invincible nor do they have any effect
on driving skill, but they still do improve safety when worn properly ;)

Mailing list: https://launchpad.net/~piaget-dev
Post to     : piaget-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~piaget-dev
More help   : https://help.launchpad.net/ListHelp

Reply via email to