I see unclosed/unflushed streams in computeV, computeU, but that doesn't seem to be the reason.
On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: > yes. i would venture to say that U computation (or restoration) is > somehow corrupted starting with 2nd block. at least it looks this way. > > On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: >> A-reconstructed difference looks good up to row 399 but starting at >> row 400 differences do not add up to 0 anymore (although both inputs >> are not 0). >> >> So it doesn't look like trivial case of something is not initialized >> on top of it. It does seem something to do with blocking mechanism >> though since apparently 400th row is a boundary of some blocking >> somewhere, but it is hard for me to see where it fails at this point. >> >> On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov <dlie...@gmail.com> wrote: >>> oh. it's because the synthetic input has only 4 singular values. >>> >>> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov <dlie...@gmail.com> >>> wrote: >>>> But it is not a problem reading U or V files, that's indeed what U and >>>> V contain. >>>> >>>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov <dlie...@gmail.com> >>>> wrote: >>>>> U and V look suspect, degenerate (only 4 first columns are nonzero, >>>>> the rest of matrices are zeros. >>>>> >>>>> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov <dlie...@gmail.com> >>>>> wrote: >>>>>> Yeah, fails for me on ubuntu without any special environment issues. >>>>>> Which makes it easier, i can step thru. >>>>>> >>>>>> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning <ted.dunn...@gmail.com> >>>>>> wrote: >>>>>>> What do checksums look like? >>>>>>> >>>>>>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll >>>>>>> <gsing...@apache.org>wrote: >>>>>>> >>>>>>>> I commented out the deletion of the dir in the tearDown. Not sure if >>>>>>>> that >>>>>>>> looks reasonable or not, but on the surface they look equivalent. >>>>>>>> >>>>>>>> Here's the contents of the dir on Ubuntu: >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 1632612 2011-12-28 21:17 A-000000000 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 1632612 2011-12-28 21:17 A-000000200 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 1632612 2011-12-28 21:17 A-000000400 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 1632612 2011-12-28 21:17 A-000000600 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 1387722 2011-12-28 21:17 A-000000800 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168312 2011-12-28 21:17 B-000000000 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168312 2011-12-28 21:17 B-000000210 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168312 2011-12-28 21:17 B-000000420 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168312 2011-12-28 21:17 B-000000630 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 144312 2011-12-28 21:17 B-000000840 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 160412 2011-12-28 21:17 U-0 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 160412 2011-12-28 21:17 U-200 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 160412 2011-12-28 21:17 U-400 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 160412 2011-12-28 21:17 U-600 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 136352 2011-12-28 21:17 U-800 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168432 2011-12-28 21:17 V-0 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168432 2011-12-28 21:17 V-1 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168432 2011-12-28 21:17 V-2 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 168432 2011-12-28 21:17 V-3 >>>>>>>> -rw-rw-r-- 1 XXXXXX XXXXXX 144372 2011-12-28 21:17 V-4 >>>>>>>> >>>>>>>> Here's what my Mac looks like: >>>>>>>> total 20296 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 1.6M Dec 28 21:28 A-000000000 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 1.6M Dec 28 21:28 A-000000200 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 1.6M Dec 28 21:28 A-000000400 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 1.6M Dec 28 21:28 A-000000600 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 1.3M Dec 28 21:28 A-000000800 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 B-000000000 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 B-000000210 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 B-000000420 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 B-000000630 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 141K Dec 28 21:28 B-000000840 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 157K Dec 28 21:28 U-0 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 157K Dec 28 21:28 U-200 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 157K Dec 28 21:28 U-400 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 157K Dec 28 21:28 U-600 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 133K Dec 28 21:28 U-800 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 V-0 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 V-1 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 V-2 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 164K Dec 28 21:28 V-3 >>>>>>>> -rw-r--r-- 1 XXXXXX staff 141K Dec 28 21:28 V-4 >>>>>>>> >>>>>>>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote: >>>>>>>> >>>>>>>> > Yeah.. but this is a difference from the correct answer. I am >>>>>>>> > moderately >>>>>>>> > sure that this is a problem writing to the temp directory. >>>>>>>> > >>>>>>>> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll <gsing...@apache.org >>>>>>>> >wrote: >>>>>>>> > >>>>>>>> >> It's expecting the answer to be 0, but it's some really large value. >>>>>>>> >> >>>>>>>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest): >>>>>>>> >> expected:<0.0> but was:<4131200.0000000037> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote: >>>>>>>> >> >>>>>>>> >>> I think that the answer is 0 because the model is not being read >>>>>>>> >>> and we >>>>>>>> >> are >>>>>>>> >>> swallowing an exception somewhere. This is what an uninitialized >>>>>>>> matrix >>>>>>>> >>> would give as a result. >>>>>>>> >>> >>>>>>>> >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll >>>>>>>> >>> <gsing...@apache.org >>>>>>>> >>> wrote: >>>>>>>> >>> >>>>>>>> >>>> I can reproduce outside of Jenkins. It really seems odd that the >>>>>>>> answer >>>>>>>> >>>> is off by so much. >>>>>>>> >>>> >>>>>>>> >>>> On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote: >>>>>>>> >>>> >>>>>>>> >>>>> I vaguely remember Jenkins had problems with creating stuff in >>>>>>>> >>>>> Java >>>>>>>> tmp >>>>>>>> >>>>> dir. E.g. I remember that was creating problems for Mr tasks in >>>>>>>> >>>>> local >>>>>>>> >> mr >>>>>>>> >>>>> mode legitimately using boxed task temporary space. >>>>>>>> >>>>> >>>>>>>> >>>>> OK I'll try to scan for the problem tomorrow. >>>>>>>> >>>>> On Dec 27, 2011 10:50 PM, "Ted Dunning" <ted.dunn...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>> >>>>>>>> >>>>>> So I am like everybody else. The test works for me. >>>>>>>> >>>>>> >>>>>>>> >>>>>> My suspicion is that there is something going on with the >>>>>>>> >>>>>> temporary >>>>>>>> >>>>>> directory that I am trying to use and that the environment that >>>>>>>> >> Jenkins >>>>>>>> >>>> is >>>>>>>> >>>>>> using is somehow strange. >>>>>>>> >>>>>> >>>>>>>> >>>>>> The only slightly surprising idiom I am using is to create a >>>>>>>> temporary >>>>>>>> >>>>>> file, delete it and recreate it as a directory. I even check >>>>>>>> >>>>>> the >>>>>>>> >> return >>>>>>>> >>>>>> values from the delete and the mkdir. >>>>>>>> >>>>>> >>>>>>>> >>>>>> I will keep looking. >>>>>>>> >>>>>> >>>>>>>> >>>>>> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning < >>>>>>>> ted.dunn...@gmail.com> >>>>>>>> >>>>>> wrote: >>>>>>>> >>>>>> >>>>>>>> >>>>>>> Indeed it does. Thanks for pointing that out. >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> This error is very strange. >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov < >>>>>>>> >> dlie...@gmail.com >>>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Ted, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> do you have an idea why this test may be failing? I think this >>>>>>>> test >>>>>>>> >>>>>> comes >>>>>>>> >>>>>>>> with M-792 commit. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I can take a look at it, I suspect something in the >>>>>>>> >>>>>>>> environment >>>>>>>> can >>>>>>>> >> be >>>>>>>> >>>>>>>> tripping it. >>>>>>>> >>>>>>>> On Dec 27, 2011 8:54 PM, "Sean Owen" <sro...@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> It's all errors in the Apache infrastructure, rather than a >>>>>>>> >>>>>>>>> real >>>>>>>> >> test >>>>>>>> >>>>>>>>> failure. At least, stuff passes for me locally, and that's >>>>>>>> >>>>>>>>> what's >>>>>>>> >>>>>>>>> important. >>>>>>>> >>>>>>>>> So I'm ignoring these. >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman >>>>>>>> >>>>>>>>> <jeast...@windwardsolutions.com> wrote: >>>>>>>> >>>>>>>>>> I'm getting a lot of these emails yet all the tests run >>>>>>>> >>>>>>>>>> locally >>>>>>>> >> for >>>>>>>> >>>>>>>> me. >>>>>>>> >>>>>>>>> Does >>>>>>>> >>>>>>>>>> anybody have an idea what the problem is? This close to a >>>>>>>> release >>>>>>>> >> it >>>>>>>> >>>>>>>>> would >>>>>>>> >>>>>>>>>> be really nice to have Jenkins on our side. >>>>>>>> >>>>>>>>>> Jeff >>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>> >>>>>>>> >>>> >>>>>>>> >>>> -------------------------------------------- >>>>>>>> >>>> Grant Ingersoll >>>>>>>> >>>> http://www.lucidimagination.com >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >>>> >>>>>>>> >> >>>>>>>> >> -------------------------------------------- >>>>>>>> >> Grant Ingersoll >>>>>>>> >> http://www.lucidimagination.com >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >>>>>>>> -------------------------------------------- >>>>>>>> Grant Ingersoll >>>>>>>> http://www.lucidimagination.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>
