If save/load actually makes a reliable difference, then it would be useful
to do something like this, and see what you see:
save("X", X)
X2 = load("X.npy")
diff = (X == X2)
# did save/load change anything?
any(diff)
# if so, then what changed?
X[diff]
X2[diff]
# any subtle differences in floating point representation?
X[diff][0].view(np.uint8)
X2[diff][0].view(np.uint8)
(You should still run memtest. It's very easy - just install it with your
package manager, then reboot. Hold down the shift key while booting, and
you'll get a boot menu. Choose memtest, and then leave it to run overnight.)
- Nathaniel
On Dec 2, 2011 10:10 PM, "kneil" <magnetotellur...@gmail.com> wrote:
>
> OK - here is the status update on this problem:
> 1. To test for bad RAM, I saved the code plus data onto a usb drive and
> manually transferred it to a colleague's desktop machine with same specs as
> mine. The NaN values continued to appear at random, so it is unlikely to
> be
> bad RAM - unless its contagious.
>
> 2. Here is how I am temporarily working around the problem: Right before
> performing the multiplication of X*X.H where X=asmatrix(Y), I save X to
> file
> using np.save('X',X). Then I reload it via X=np.load('X.npy'). Cast as
> matrix: X=asmatrix(X); and carry on: S=X*X.H. I have not seen any NaN
> although I have only run it a few times... but it seems to work.
>
> I'd be very curious to hear any ideas on why this problem exists in the
> first place, and why save/load gets me around it.
>
> For the record I am running ubuntu 11.04, have 16GB RAM (not 12) and use
> python2.7.
> cheers,
> Karl
>
> Hi Pierre,
>
> I confirmed with the guy who put together the machine that it is non-ECC
> RAM. You know, now that i think about it, this machine seems to crash a
> fair amount more often than its identical twin which sits on a desk near
> me.
> I researched memtest a bit... downloaded and compiled it, but I do not
> quite
> understand the finer points of using it... it seems that I want to remove
> my
> RAM cards and test them one at a time. Do you know a good reference for
> using it.
>
> I think at this point the best thing to do will be to dump my data/code to
> portable HDD and load it on the other computer with same specs as this one.
> If it runs without generating any NaN then I will proceed to a full
> memtest.
>
> Thanks for the advice.
> -Karl
>
> On Thu, Dec 1, 2011 at 2:47 PM, kneil <magnetotellur...@gmail.com> wrote:
>
> Are you using non-ECC RAM, by chance? (Though if you have >4GB of ram, I
> can't imagine that you wouldn't be using ECC...)
>
> Alternately, have you run memtest lately? That sound suspiciously like bad
> ram...
>
>
>
> --
> View this message in context:
> http://old.nabble.com/Apparently-non-deterministic-behaviour-of-complex-array-multiplication-tp32893004p32906553.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
