Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

No Joy (no time) yet.

On Fri, Jan 6, 2012 at 6:59 AM, Grant Ingersoll  wrote:

> Any luck on this, as it is still failing for me on Ubuntu?
>
> We are slowing whittling down the 0.6 issues, but this one seems critical.
>
>
> On Dec 31, 2011, at 8:07 PM, Ted Dunning wrote:
>
> > I have replicated this and applied this patch but the test still fails.
> >
> > I will take a longer look when I get a chance.
> >
> > On Sat, Dec 31, 2011 at 3:57 PM, Ted Dunning 
> wrote:
> >
> >> I just took a look and found several files not closed issues as Dmitriy
> >> mentioned.  There might be a buffering difference on Linux versus make.
> >>
> >> I have a patch to fix this and will test right now on EC2 if I have
> enough
> >> time.
> >>
> >>
> >> On Sat, Dec 31, 2011 at 2:25 PM, Grant Ingersoll  >wrote:
> >>
> >>> Any progress on this?  Seems like it is a show stopper for code freeze.
> >>> On Dec 29, 2011, at 3:58 AM, Dmitriy Lyubimov wrote:
> >>>
>  note identical md5 for u200 and u400.
> 
>  On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov  >
> >>> wrote:
> > MD5 sums
> >
> > b8217318a29ef69c58b921013eb019e5
> >>> /tmp/matrix8554072597307396201/A-0
> > 41db088ff74c5efd5b766dba253efc03
> >>> /tmp/matrix8554072597307396201/A-00200
> > b8217318a29ef69c58b921013eb019e5
> >>> /tmp/matrix8554072597307396201/A-00400
> > 41db088ff74c5efd5b766dba253efc03
> >>> /tmp/matrix8554072597307396201/A-00600
> > c8dc2a7df82065b5c1e8284ff23aecc6
> >>> /tmp/matrix8554072597307396201/A-00800
> > 83bccdd2fa191e01d34646e2030f0e77
> >>> /tmp/matrix8554072597307396201/B-0
> > 9d6878fb789d61d5453b994ea1a5c6db
> >>> /tmp/matrix8554072597307396201/B-00210
> > cbdf720b17ce25feb686effd1aa0ebef
> >>> /tmp/matrix8554072597307396201/B-00420
> > 2f71d6ba6891b242575b5cc6ba1c4358
> >>> /tmp/matrix8554072597307396201/B-00630
> > f50b76bb48c8f6a791a6d8206d980492
> >>> /tmp/matrix8554072597307396201/B-00840
> > 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
> > 019c881c1d7c5748a1cacb3e0b3e5899
>  /tmp/matrix8554072597307396201/U-200
> > 019c881c1d7c5748a1cacb3e0b3e5899
>  /tmp/matrix8554072597307396201/U-400
> > b84e4b01ffb9d691c87b496f8b4d84ec
>  /tmp/matrix8554072597307396201/U-600
> > 6bb29ca304889a6c8e6e5c062dc8
>  /tmp/matrix8554072597307396201/U-770
> >
> >
> > On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning  >
> >>> wrote:
> >> Thanks.  Good hints.
> >>
> >> I will take a look on a linux machine in the next few days.
> >>
> >> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov <
> dlie...@gmail.com
>  wrote:
> >>
> >>> yes. i would venture to say that U computation (or restoration) is
> >>> somehow corrupted starting with 2nd block. at least it looks this
> >>> way.
> >>>
> >>> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov <
> >>> dlie...@gmail.com>
> >>> wrote:
>  A-reconstructed difference looks good up to row 399 but starting
> at
>  row 400 differences do not add up to 0 anymore (although both
> inputs
>  are not 0).
> 
>  So it doesn't look like trivial case of something is not
> initialized
>  on top of it. It does seem something to do with blocking mechanism
>  though since apparently 400th row is a boundary of some blocking
>  somewhere, but it is hard for me to see where it fails at this
> >>> point.
> 
>  On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov <
> >>> dlie...@gmail.com>
> >>> wrote:
> > oh. it's because the synthetic input has only 4 singular values.
> >
> > On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov <
> >>> dlie...@gmail.com>
> >>> wrote:
> >> But it is not a problem reading U or V files, that's indeed what
> >>> U and
> >> V contain.
> >>
> >> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov <
> >>> dlie...@gmail.com>
> >>> wrote:
> >>> U and V look suspect, degenerate (only 4 first columns are
> >>> nonzero,
> >>> the rest of matrices are zeros.
> >>>
> >>> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov <
> >>> dlie...@gmail.com>
> >>> wrote:
>  Yeah, fails for me on ubuntu without any special environment
> >>> issues.
>  Which makes it easier, i can step thru.
> 
>  On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning <
> >>> ted.dunn...@gmail.com>
> >>> wrote:
> > What do checksums look like?
> >
> > On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
> >>> gsing...@apache.org>wrote:
> >
> >> I commented out the deletion of the dir in the tearDown.
>  Not
> >>> sure
> >>> if that
> >> looks reasonable or not, but on the surface they look
> >>> equivalent.
> >>
> >> Here's

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

Any luck on this, as it is still failing for me on Ubuntu?  

We are slowing whittling down the 0.6 issues, but this one seems critical.


On Dec 31, 2011, at 8:07 PM, Ted Dunning wrote:

> I have replicated this and applied this patch but the test still fails.
> 
> I will take a longer look when I get a chance.
> 
> On Sat, Dec 31, 2011 at 3:57 PM, Ted Dunning  wrote:
> 
>> I just took a look and found several files not closed issues as Dmitriy
>> mentioned.  There might be a buffering difference on Linux versus make.
>> 
>> I have a patch to fix this and will test right now on EC2 if I have enough
>> time.
>> 
>> 
>> On Sat, Dec 31, 2011 at 2:25 PM, Grant Ingersoll wrote:
>> 
>>> Any progress on this?  Seems like it is a show stopper for code freeze.
>>> On Dec 29, 2011, at 3:58 AM, Dmitriy Lyubimov wrote:
>>> 
 note identical md5 for u200 and u400.
 
 On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov 
>>> wrote:
> MD5 sums
> 
> b8217318a29ef69c58b921013eb019e5
>>> /tmp/matrix8554072597307396201/A-0
> 41db088ff74c5efd5b766dba253efc03
>>> /tmp/matrix8554072597307396201/A-00200
> b8217318a29ef69c58b921013eb019e5
>>> /tmp/matrix8554072597307396201/A-00400
> 41db088ff74c5efd5b766dba253efc03
>>> /tmp/matrix8554072597307396201/A-00600
> c8dc2a7df82065b5c1e8284ff23aecc6
>>> /tmp/matrix8554072597307396201/A-00800
> 83bccdd2fa191e01d34646e2030f0e77
>>> /tmp/matrix8554072597307396201/B-0
> 9d6878fb789d61d5453b994ea1a5c6db
>>> /tmp/matrix8554072597307396201/B-00210
> cbdf720b17ce25feb686effd1aa0ebef
>>> /tmp/matrix8554072597307396201/B-00420
> 2f71d6ba6891b242575b5cc6ba1c4358
>>> /tmp/matrix8554072597307396201/B-00630
> f50b76bb48c8f6a791a6d8206d980492
>>> /tmp/matrix8554072597307396201/B-00840
> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
> 
> 
> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning 
>>> wrote:
>> Thanks.  Good hints.
>> 
>> I will take a look on a linux machine in the next few days.
>> 
>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov >>> wrote:
>> 
>>> yes. i would venture to say that U computation (or restoration) is
>>> somehow corrupted starting with 2nd block. at least it looks this
>>> way.
>>> 
>>> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov <
>>> dlie...@gmail.com>
>>> wrote:
 A-reconstructed difference looks good up to row 399 but starting at
 row 400 differences do not add up to 0 anymore (although both inputs
 are not 0).
 
 So it doesn't look like trivial case of something is not initialized
 on top of it. It does seem something to do with blocking mechanism
 though since apparently 400th row is a boundary of some blocking
 somewhere, but it is hard for me to see where it fails at this
>>> point.
 
 On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov <
>>> dlie...@gmail.com>
>>> wrote:
> oh. it's because the synthetic input has only 4 singular values.
> 
> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov <
>>> dlie...@gmail.com>
>>> wrote:
>> But it is not a problem reading U or V files, that's indeed what
>>> U and
>> V contain.
>> 
>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov <
>>> dlie...@gmail.com>
>>> wrote:
>>> U and V look suspect, degenerate (only 4 first columns are
>>> nonzero,
>>> the rest of matrices are zeros.
>>> 
>>> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov <
>>> dlie...@gmail.com>
>>> wrote:
 Yeah, fails for me on ubuntu without any special environment
>>> issues.
 Which makes it easier, i can step thru.
 
 On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning <
>>> ted.dunn...@gmail.com>
>>> wrote:
> What do checksums look like?
> 
> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
>>> gsing...@apache.org>wrote:
> 
>> I commented out the deletion of the dir in the tearDown.  Not
>>> sure
>>> if that
>> looks reasonable or not, but on the surface they look
>>> equivalent.
>> 
>> Here's the contents of the dir on Ubuntu:
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>>> A-0
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>>> A-00200
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>>> A-00400
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

I have replicated this and applied this patch but the test still fails.

I will take a longer look when I get a chance.

On Sat, Dec 31, 2011 at 3:57 PM, Ted Dunning  wrote:

> I just took a look and found several files not closed issues as Dmitriy
> mentioned.  There might be a buffering difference on Linux versus make.
>
> I have a patch to fix this and will test right now on EC2 if I have enough
> time.
>
>
> On Sat, Dec 31, 2011 at 2:25 PM, Grant Ingersoll wrote:
>
>> Any progress on this?  Seems like it is a show stopper for code freeze.
>> On Dec 29, 2011, at 3:58 AM, Dmitriy Lyubimov wrote:
>>
>> > note identical md5 for u200 and u400.
>> >
>> > On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov 
>> wrote:
>> >> MD5 sums
>> >>
>> >> b8217318a29ef69c58b921013eb019e5
>>  /tmp/matrix8554072597307396201/A-0
>> >> 41db088ff74c5efd5b766dba253efc03
>>  /tmp/matrix8554072597307396201/A-00200
>> >> b8217318a29ef69c58b921013eb019e5
>>  /tmp/matrix8554072597307396201/A-00400
>> >> 41db088ff74c5efd5b766dba253efc03
>>  /tmp/matrix8554072597307396201/A-00600
>> >> c8dc2a7df82065b5c1e8284ff23aecc6
>>  /tmp/matrix8554072597307396201/A-00800
>> >> 83bccdd2fa191e01d34646e2030f0e77
>>  /tmp/matrix8554072597307396201/B-0
>> >> 9d6878fb789d61d5453b994ea1a5c6db
>>  /tmp/matrix8554072597307396201/B-00210
>> >> cbdf720b17ce25feb686effd1aa0ebef
>>  /tmp/matrix8554072597307396201/B-00420
>> >> 2f71d6ba6891b242575b5cc6ba1c4358
>>  /tmp/matrix8554072597307396201/B-00630
>> >> f50b76bb48c8f6a791a6d8206d980492
>>  /tmp/matrix8554072597307396201/B-00840
>> >> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
>> >> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
>> >> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
>> >> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
>> >> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
>> >>
>> >>
>> >> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning 
>> wrote:
>> >>> Thanks.  Good hints.
>> >>>
>> >>> I will take a look on a linux machine in the next few days.
>> >>>
>> >>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov > >wrote:
>> >>>
>>  yes. i would venture to say that U computation (or restoration) is
>>  somehow corrupted starting with 2nd block. at least it looks this
>> way.
>> 
>>  On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov <
>> dlie...@gmail.com>
>>  wrote:
>> > A-reconstructed difference looks good up to row 399 but starting at
>> > row 400 differences do not add up to 0 anymore (although both inputs
>> > are not 0).
>> >
>> > So it doesn't look like trivial case of something is not initialized
>> > on top of it. It does seem something to do with blocking mechanism
>> > though since apparently 400th row is a boundary of some blocking
>> > somewhere, but it is hard for me to see where it fails at this
>> point.
>> >
>> > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov <
>> dlie...@gmail.com>
>>  wrote:
>> >> oh. it's because the synthetic input has only 4 singular values.
>> >>
>> >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov <
>> dlie...@gmail.com>
>>  wrote:
>> >>> But it is not a problem reading U or V files, that's indeed what
>> U and
>> >>> V contain.
>> >>>
>> >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov <
>> dlie...@gmail.com>
>>  wrote:
>>  U and V look suspect, degenerate (only 4 first columns are
>> nonzero,
>>  the rest of matrices are zeros.
>> 
>>  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov <
>> dlie...@gmail.com>
>>  wrote:
>> > Yeah, fails for me on ubuntu without any special environment
>> issues.
>> > Which makes it easier, i can step thru.
>> >
>> > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning <
>> ted.dunn...@gmail.com>
>>  wrote:
>> >> What do checksums look like?
>> >>
>> >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
>>  gsing...@apache.org>wrote:
>> >>
>> >>> I commented out the deletion of the dir in the tearDown.  Not
>> sure
>>  if that
>> >>> looks reasonable or not, but on the surface they look
>> equivalent.
>> >>>
>> >>> Here's the contents of the dir on Ubuntu:
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>> A-0
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>> A-00200
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>> A-00400
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17
>> A-00600
>> >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17
>> A-00800
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17
>> B-0
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17
>> 

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

I just took a look and found several files not closed issues as Dmitriy
mentioned.  There might be a buffering difference on Linux versus make.

I have a patch to fix this and will test right now on EC2 if I have enough
time.

On Sat, Dec 31, 2011 at 2:25 PM, Grant Ingersoll wrote:

> Any progress on this?  Seems like it is a show stopper for code freeze.
> On Dec 29, 2011, at 3:58 AM, Dmitriy Lyubimov wrote:
>
> > note identical md5 for u200 and u400.
> >
> > On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov 
> wrote:
> >> MD5 sums
> >>
> >> b8217318a29ef69c58b921013eb019e5
>  /tmp/matrix8554072597307396201/A-0
> >> 41db088ff74c5efd5b766dba253efc03
>  /tmp/matrix8554072597307396201/A-00200
> >> b8217318a29ef69c58b921013eb019e5
>  /tmp/matrix8554072597307396201/A-00400
> >> 41db088ff74c5efd5b766dba253efc03
>  /tmp/matrix8554072597307396201/A-00600
> >> c8dc2a7df82065b5c1e8284ff23aecc6
>  /tmp/matrix8554072597307396201/A-00800
> >> 83bccdd2fa191e01d34646e2030f0e77
>  /tmp/matrix8554072597307396201/B-0
> >> 9d6878fb789d61d5453b994ea1a5c6db
>  /tmp/matrix8554072597307396201/B-00210
> >> cbdf720b17ce25feb686effd1aa0ebef
>  /tmp/matrix8554072597307396201/B-00420
> >> 2f71d6ba6891b242575b5cc6ba1c4358
>  /tmp/matrix8554072597307396201/B-00630
> >> f50b76bb48c8f6a791a6d8206d980492
>  /tmp/matrix8554072597307396201/B-00840
> >> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
> >> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
> >> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
> >> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
> >> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
> >>
> >>
> >> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning 
> wrote:
> >>> Thanks.  Good hints.
> >>>
> >>> I will take a look on a linux machine in the next few days.
> >>>
> >>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov  >wrote:
> >>>
>  yes. i would venture to say that U computation (or restoration) is
>  somehow corrupted starting with 2nd block. at least it looks this way.
> 
>  On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov  >
>  wrote:
> > A-reconstructed difference looks good up to row 399 but starting at
> > row 400 differences do not add up to 0 anymore (although both inputs
> > are not 0).
> >
> > So it doesn't look like trivial case of something is not initialized
> > on top of it. It does seem something to do with blocking mechanism
> > though since apparently 400th row is a boundary of some blocking
> > somewhere, but it is hard for me to see where it fails at this point.
> >
> > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov <
> dlie...@gmail.com>
>  wrote:
> >> oh. it's because the synthetic input has only 4 singular values.
> >>
> >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov <
> dlie...@gmail.com>
>  wrote:
> >>> But it is not a problem reading U or V files, that's indeed what U
> and
> >>> V contain.
> >>>
> >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov <
> dlie...@gmail.com>
>  wrote:
>  U and V look suspect, degenerate (only 4 first columns are
> nonzero,
>  the rest of matrices are zeros.
> 
>  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov <
> dlie...@gmail.com>
>  wrote:
> > Yeah, fails for me on ubuntu without any special environment
> issues.
> > Which makes it easier, i can step thru.
> >
> > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning <
> ted.dunn...@gmail.com>
>  wrote:
> >> What do checksums look like?
> >>
> >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
>  gsing...@apache.org>wrote:
> >>
> >>> I commented out the deletion of the dir in the tearDown.  Not
> sure
>  if that
> >>> looks reasonable or not, but on the surface they look
> equivalent.
> >>>
> >>> Here's the contents of the dir on Ubuntu:
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
> >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
> >>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
> >>> -rw-rw-r-- 1 XX

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

Any progress on this?  Seems like it is a show stopper for code freeze.
On Dec 29, 2011, at 3:58 AM, Dmitriy Lyubimov wrote:

> note identical md5 for u200 and u400.
> 
> On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov  wrote:
>> MD5 sums
>> 
>> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-0
>> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00200
>> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-00400
>> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00600
>> c8dc2a7df82065b5c1e8284ff23aecc6  /tmp/matrix8554072597307396201/A-00800
>> 83bccdd2fa191e01d34646e2030f0e77  /tmp/matrix8554072597307396201/B-0
>> 9d6878fb789d61d5453b994ea1a5c6db  /tmp/matrix8554072597307396201/B-00210
>> cbdf720b17ce25feb686effd1aa0ebef  /tmp/matrix8554072597307396201/B-00420
>> 2f71d6ba6891b242575b5cc6ba1c4358  /tmp/matrix8554072597307396201/B-00630
>> f50b76bb48c8f6a791a6d8206d980492  /tmp/matrix8554072597307396201/B-00840
>> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
>> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
>> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
>> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
>> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
>> 
>> 
>> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning  wrote:
>>> Thanks.  Good hints.
>>> 
>>> I will take a look on a linux machine in the next few days.
>>> 
>>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov wrote:
>>> 
 yes. i would venture to say that U computation (or restoration) is
 somehow corrupted starting with 2nd block. at least it looks this way.
 
 On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov 
 wrote:
> A-reconstructed difference looks good up to row 399 but starting at
> row 400 differences do not add up to 0 anymore (although both inputs
> are not 0).
> 
> So it doesn't look like trivial case of something is not initialized
> on top of it. It does seem something to do with blocking mechanism
> though since apparently 400th row is a boundary of some blocking
> somewhere, but it is hard for me to see where it fails at this point.
> 
> On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov 
 wrote:
>> oh. it's because the synthetic input has only 4 singular values.
>> 
>> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov 
 wrote:
>>> But it is not a problem reading U or V files, that's indeed what U and
>>> V contain.
>>> 
>>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov 
 wrote:
 U and V look suspect, degenerate (only 4 first columns are nonzero,
 the rest of matrices are zeros.
 
 On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov 
 wrote:
> Yeah, fails for me on ubuntu without any special environment issues.
> Which makes it easier, i can step thru.
> 
> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning 
 wrote:
>> What do checksums look like?
>> 
>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
 gsing...@apache.org>wrote:
>> 
>>> I commented out the deletion of the dir in the tearDown.  Not sure
 if that
>>> looks reasonable or not, but on the surface they look equivalent.
>>> 
>>> Here's the contents of the dir on Ubuntu:
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>>

Re: Build failed in Jenkins: mahout-nightly #737

[Lance Norskog]

A-0 and A-400 match, and A-200 and A-600 match.
Bs are all unique.
U-200 matches U-400

On Thu, Dec 29, 2011 at 12:58 AM, Dmitriy Lyubimov  wrote:
> note identical md5 for u200 and u400.
>
> On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov  wrote:
>> MD5 sums
>>
>> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-0
>> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00200
>> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-00400
>> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00600
>> c8dc2a7df82065b5c1e8284ff23aecc6  /tmp/matrix8554072597307396201/A-00800
>> 83bccdd2fa191e01d34646e2030f0e77  /tmp/matrix8554072597307396201/B-0
>> 9d6878fb789d61d5453b994ea1a5c6db  /tmp/matrix8554072597307396201/B-00210
>> cbdf720b17ce25feb686effd1aa0ebef  /tmp/matrix8554072597307396201/B-00420
>> 2f71d6ba6891b242575b5cc6ba1c4358  /tmp/matrix8554072597307396201/B-00630
>> f50b76bb48c8f6a791a6d8206d980492  /tmp/matrix8554072597307396201/B-00840
>> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
>> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
>> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
>> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
>> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
>>
>>
>> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning  wrote:
>>> Thanks.  Good hints.
>>>
>>> I will take a look on a linux machine in the next few days.
>>>
>>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov wrote:
>>>
 yes. i would venture to say that U computation (or restoration) is
 somehow corrupted starting with 2nd block. at least it looks this way.

 On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov 
 wrote:
 > A-reconstructed difference looks good up to row 399 but starting at
 > row 400 differences do not add up to 0 anymore (although both inputs
 > are not 0).
 >
 > So it doesn't look like trivial case of something is not initialized
 > on top of it. It does seem something to do with blocking mechanism
 > though since apparently 400th row is a boundary of some blocking
 > somewhere, but it is hard for me to see where it fails at this point.
 >
 > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov 
 wrote:
 >> oh. it's because the synthetic input has only 4 singular values.
 >>
 >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov 
 wrote:
 >>> But it is not a problem reading U or V files, that's indeed what U and
 >>> V contain.
 >>>
 >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov 
 wrote:
  U and V look suspect, degenerate (only 4 first columns are nonzero,
  the rest of matrices are zeros.
 
  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov 
 wrote:
 > Yeah, fails for me on ubuntu without any special environment issues.
 > Which makes it easier, i can step thru.
 >
 > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning 
 wrote:
 >> What do checksums look like?
 >>
 >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
 gsing...@apache.org>wrote:
 >>
 >>> I commented out the deletion of the dir in the tearDown.  Not sure
 if that
 >>> looks reasonable or not, but on the surface they look equivalent.
 >>>
 >>> Here's the contents of the dir on Ubuntu:
 >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
 >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
 >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
 >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
 >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
 >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
 >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
 >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
 >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
 >>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
 >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
 >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
 >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
 >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
 >>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
 >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
 >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
 >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
 >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
 >>> -r

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

note identical md5 for u200 and u400.

On Thu, Dec 29, 2011 at 12:57 AM, Dmitriy Lyubimov  wrote:
> MD5 sums
>
> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-0
> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00200
> b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-00400
> 41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00600
> c8dc2a7df82065b5c1e8284ff23aecc6  /tmp/matrix8554072597307396201/A-00800
> 83bccdd2fa191e01d34646e2030f0e77  /tmp/matrix8554072597307396201/B-0
> 9d6878fb789d61d5453b994ea1a5c6db  /tmp/matrix8554072597307396201/B-00210
> cbdf720b17ce25feb686effd1aa0ebef  /tmp/matrix8554072597307396201/B-00420
> 2f71d6ba6891b242575b5cc6ba1c4358  /tmp/matrix8554072597307396201/B-00630
> f50b76bb48c8f6a791a6d8206d980492  /tmp/matrix8554072597307396201/B-00840
> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
> 019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
> b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
> 6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770
>
>
> On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning  wrote:
>> Thanks.  Good hints.
>>
>> I will take a look on a linux machine in the next few days.
>>
>> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov wrote:
>>
>>> yes. i would venture to say that U computation (or restoration) is
>>> somehow corrupted starting with 2nd block. at least it looks this way.
>>>
>>> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov 
>>> wrote:
>>> > A-reconstructed difference looks good up to row 399 but starting at
>>> > row 400 differences do not add up to 0 anymore (although both inputs
>>> > are not 0).
>>> >
>>> > So it doesn't look like trivial case of something is not initialized
>>> > on top of it. It does seem something to do with blocking mechanism
>>> > though since apparently 400th row is a boundary of some blocking
>>> > somewhere, but it is hard for me to see where it fails at this point.
>>> >
>>> > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov 
>>> wrote:
>>> >> oh. it's because the synthetic input has only 4 singular values.
>>> >>
>>> >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov 
>>> wrote:
>>> >>> But it is not a problem reading U or V files, that's indeed what U and
>>> >>> V contain.
>>> >>>
>>> >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov 
>>> wrote:
>>>  U and V look suspect, degenerate (only 4 first columns are nonzero,
>>>  the rest of matrices are zeros.
>>> 
>>>  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov 
>>> wrote:
>>> > Yeah, fails for me on ubuntu without any special environment issues.
>>> > Which makes it easier, i can step thru.
>>> >
>>> > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning 
>>> wrote:
>>> >> What do checksums look like?
>>> >>
>>> >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
>>> gsing...@apache.org>wrote:
>>> >>
>>> >>> I commented out the deletion of the dir in the tearDown.  Not sure
>>> if that
>>> >>> looks reasonable or not, but on the surface they look equivalent.
>>> >>>
>>> >>> Here's the contents of the dir on Ubuntu:
>>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>>> >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>>> >>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>>> >>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>>> >>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>> >>>
>>> >>> Here's what my Mac looks like:
>>> >>> total 20296
>>> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>>> >>> -rw-r--r--  1 XX  staf

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

MD5 sums

b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-0
41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00200
b8217318a29ef69c58b921013eb019e5  /tmp/matrix8554072597307396201/A-00400
41db088ff74c5efd5b766dba253efc03  /tmp/matrix8554072597307396201/A-00600
c8dc2a7df82065b5c1e8284ff23aecc6  /tmp/matrix8554072597307396201/A-00800
83bccdd2fa191e01d34646e2030f0e77  /tmp/matrix8554072597307396201/B-0
9d6878fb789d61d5453b994ea1a5c6db  /tmp/matrix8554072597307396201/B-00210
cbdf720b17ce25feb686effd1aa0ebef  /tmp/matrix8554072597307396201/B-00420
2f71d6ba6891b242575b5cc6ba1c4358  /tmp/matrix8554072597307396201/B-00630
f50b76bb48c8f6a791a6d8206d980492  /tmp/matrix8554072597307396201/B-00840
6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-0
019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-200
019c881c1d7c5748a1cacb3e0b3e5899  /tmp/matrix8554072597307396201/U-400
b84e4b01ffb9d691c87b496f8b4d84ec  /tmp/matrix8554072597307396201/U-600
6bb29ca304889a6c8e6e5c062dc8  /tmp/matrix8554072597307396201/U-770


On Thu, Dec 29, 2011 at 12:52 AM, Ted Dunning  wrote:
> Thanks.  Good hints.
>
> I will take a look on a linux machine in the next few days.
>
> On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov wrote:
>
>> yes. i would venture to say that U computation (or restoration) is
>> somehow corrupted starting with 2nd block. at least it looks this way.
>>
>> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov 
>> wrote:
>> > A-reconstructed difference looks good up to row 399 but starting at
>> > row 400 differences do not add up to 0 anymore (although both inputs
>> > are not 0).
>> >
>> > So it doesn't look like trivial case of something is not initialized
>> > on top of it. It does seem something to do with blocking mechanism
>> > though since apparently 400th row is a boundary of some blocking
>> > somewhere, but it is hard for me to see where it fails at this point.
>> >
>> > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov 
>> wrote:
>> >> oh. it's because the synthetic input has only 4 singular values.
>> >>
>> >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov 
>> wrote:
>> >>> But it is not a problem reading U or V files, that's indeed what U and
>> >>> V contain.
>> >>>
>> >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov 
>> wrote:
>>  U and V look suspect, degenerate (only 4 first columns are nonzero,
>>  the rest of matrices are zeros.
>> 
>>  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov 
>> wrote:
>> > Yeah, fails for me on ubuntu without any special environment issues.
>> > Which makes it easier, i can step thru.
>> >
>> > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning 
>> wrote:
>> >> What do checksums look like?
>> >>
>> >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
>> gsing...@apache.org>wrote:
>> >>
>> >>> I commented out the deletion of the dir in the tearDown.  Not sure
>> if that
>> >>> looks reasonable or not, but on the surface they look equivalent.
>> >>>
>> >>> Here's the contents of the dir on Ubuntu:
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>> >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>> >>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>> >>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>> >>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>> >>>
>> >>> Here's what my Mac looks like:
>> >>> total 20296
>> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
>> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
>> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
>> >>> -rw-r--r--  1 XX  st

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

Thanks.  Good hints.

I will take a look on a linux machine in the next few days.

On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov wrote:

> yes. i would venture to say that U computation (or restoration) is
> somehow corrupted starting with 2nd block. at least it looks this way.
>
> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov 
> wrote:
> > A-reconstructed difference looks good up to row 399 but starting at
> > row 400 differences do not add up to 0 anymore (although both inputs
> > are not 0).
> >
> > So it doesn't look like trivial case of something is not initialized
> > on top of it. It does seem something to do with blocking mechanism
> > though since apparently 400th row is a boundary of some blocking
> > somewhere, but it is hard for me to see where it fails at this point.
> >
> > On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov 
> wrote:
> >> oh. it's because the synthetic input has only 4 singular values.
> >>
> >> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov 
> wrote:
> >>> But it is not a problem reading U or V files, that's indeed what U and
> >>> V contain.
> >>>
> >>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov 
> wrote:
>  U and V look suspect, degenerate (only 4 first columns are nonzero,
>  the rest of matrices are zeros.
> 
>  On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov 
> wrote:
> > Yeah, fails for me on ubuntu without any special environment issues.
> > Which makes it easier, i can step thru.
> >
> > On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning 
> wrote:
> >> What do checksums look like?
> >>
> >> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll <
> gsing...@apache.org>wrote:
> >>
> >>> I commented out the deletion of the dir in the tearDown.  Not sure
> if that
> >>> looks reasonable or not, but on the surface they look equivalent.
> >>>
> >>> Here's the contents of the dir on Ubuntu:
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
> >>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
> >>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
> >>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
> >>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
> >>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
> >>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
> >>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
> >>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
> >>>
> >>> Here's what my Mac looks like:
> >>> total 20296
> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
> >>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
> >>> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
> >>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
> >>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
> >>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
> >>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
> >>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
> >>> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
> >>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
> >>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
> >>>
> >>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
> >>>
> >>> > Yeah.. but this is a difference from the correct answer.  I am
> moderately
> >>> > sure that this is a problem writing to the temp directory.
> >>> >
> >>

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

I see unclosed/unflushed streams in computeV, computeU, but that
doesn't seem to be the reason.

On Thu, Dec 29, 2011 at 12:42 AM, Dmitriy Lyubimov  wrote:
> yes. i would venture to say that U computation (or restoration) is
> somehow corrupted starting with 2nd block. at least it looks this way.
>
> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov  wrote:
>> A-reconstructed difference looks good up to row 399 but starting at
>> row 400 differences do not add up to 0 anymore (although both inputs
>> are not 0).
>>
>> So it doesn't look like trivial case of something is not initialized
>> on top of it. It does seem something to do with blocking mechanism
>> though since apparently 400th row is a boundary of some blocking
>> somewhere, but it is hard for me to see where it fails at this point.
>>
>> On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov  wrote:
>>> oh. it's because the synthetic input has only 4 singular values.
>>>
>>> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov  
>>> wrote:
 But it is not a problem reading U or V files, that's indeed what U and
 V contain.

 On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  
 wrote:
> U and V look suspect, degenerate (only 4 first columns are nonzero,
> the rest of matrices are zeros.
>
> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  
> wrote:
>> Yeah, fails for me on ubuntu without any special environment issues.
>> Which makes it easier, i can step thru.
>>
>> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  
>> wrote:
>>> What do checksums look like?
>>>
>>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll 
>>> wrote:
>>>
 I commented out the deletion of the dir in the tearDown.  Not sure if 
 that
 looks reasonable or not, but on the surface they look equivalent.

 Here's the contents of the dir on Ubuntu:
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
 -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
 -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
 -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
 -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4

 Here's what my Mac looks like:
 total 20296
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
 -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
 -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
 -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
 -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4

 On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:

 > Yeah.. but this is a difference from the correct answer.  I am 
 > moderately
 > sure that this is a problem writing to the temp directory.
 >
 > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll

Re: Build failed in Jenkins: mahout-nightly #737

[Dawid Weiss]

Does the same problem occur with another JVM, Dmitriy? JRockit or
IBM's J9? Or even a different version of HotSpot? I remember we were
once trying to figure out a similar bug (lots of computations,
different result) in our code and it turned out that the problem was
due to different native floating point precision roundings and code
JITting at different points in time (this was a very hard stuff to
debug).

The result in our case did not propagate to such large differences,
but I would still try to reproduce with a different JVM just to see if
this makes any difference.

My next step would then be to instrument the code with AspectJ
(statically) and run on a machine (jvm) where the test passes and on
the one where it fails and dump, dump, dump execution progress (all
method entry/exit points, with arguments). These are usually very
large files, but you can easily diff them against each other and see
where _exactly_ the execution started to diverge. If you add
-XX:PrintCompilation to hotspot you may even be able to tell if this
is a JIT problem or something else.

Shooting at the dark here, but maybe you'll find it helpful. I can
help you with aspectj instrumentation if you find this interesting.

Dawid

On Thu, Dec 29, 2011 at 9:42 AM, Dmitriy Lyubimov  wrote:
> yes. i would venture to say that U computation (or restoration) is
> somehow corrupted starting with 2nd block. at least it looks this way.
>
> On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov  wrote:
>> A-reconstructed difference looks good up to row 399 but starting at
>> row 400 differences do not add up to 0 anymore (although both inputs
>> are not 0).
>>
>> So it doesn't look like trivial case of something is not initialized
>> on top of it. It does seem something to do with blocking mechanism
>> though since apparently 400th row is a boundary of some blocking
>> somewhere, but it is hard for me to see where it fails at this point.
>>
>> On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov  wrote:
>>> oh. it's because the synthetic input has only 4 singular values.
>>>
>>> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov  
>>> wrote:
 But it is not a problem reading U or V files, that's indeed what U and
 V contain.

 On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  
 wrote:
> U and V look suspect, degenerate (only 4 first columns are nonzero,
> the rest of matrices are zeros.
>
> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  
> wrote:
>> Yeah, fails for me on ubuntu without any special environment issues.
>> Which makes it easier, i can step thru.
>>
>> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  
>> wrote:
>>> What do checksums look like?
>>>
>>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll 
>>> wrote:
>>>
 I commented out the deletion of the dir in the tearDown.  Not sure if 
 that
 looks reasonable or not, but on the surface they look equivalent.

 Here's the contents of the dir on Ubuntu:
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
 -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
 -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
 -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
 -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4

 Here's what my Mac looks like:
 total 20296
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
 -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
 -rw

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

yes. i would venture to say that U computation (or restoration) is
somehow corrupted starting with 2nd block. at least it looks this way.

On Thu, Dec 29, 2011 at 12:31 AM, Dmitriy Lyubimov  wrote:
> A-reconstructed difference looks good up to row 399 but starting at
> row 400 differences do not add up to 0 anymore (although both inputs
> are not 0).
>
> So it doesn't look like trivial case of something is not initialized
> on top of it. It does seem something to do with blocking mechanism
> though since apparently 400th row is a boundary of some blocking
> somewhere, but it is hard for me to see where it fails at this point.
>
> On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov  wrote:
>> oh. it's because the synthetic input has only 4 singular values.
>>
>> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov  wrote:
>>> But it is not a problem reading U or V files, that's indeed what U and
>>> V contain.
>>>
>>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  
>>> wrote:
 U and V look suspect, degenerate (only 4 first columns are nonzero,
 the rest of matrices are zeros.

 On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  
 wrote:
> Yeah, fails for me on ubuntu without any special environment issues.
> Which makes it easier, i can step thru.
>
> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  
> wrote:
>> What do checksums look like?
>>
>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll 
>> wrote:
>>
>>> I commented out the deletion of the dir in the tearDown.  Not sure if 
>>> that
>>> looks reasonable or not, but on the surface they look equivalent.
>>>
>>> Here's the contents of the dir on Ubuntu:
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>>
>>> Here's what my Mac looks like:
>>> total 20296
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
>>> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
>>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
>>> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
>>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>>>
>>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>>>
>>> > Yeah.. but this is a difference from the correct answer.  I am 
>>> > moderately
>>> > sure that this is a problem writing to the temp directory.
>>> >
>>> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll >> >wrote:
>>> >
>>> >> It's expecting the answer to be 0, but it's some really large value.
>>> >>
>>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
>>> >> expected:<0.0> but was:<4131200.37>
>

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

A-reconstructed difference looks good up to row 399 but starting at
row 400 differences do not add up to 0 anymore (although both inputs
are not 0).

So it doesn't look like trivial case of something is not initialized
on top of it. It does seem something to do with blocking mechanism
though since apparently 400th row is a boundary of some blocking
somewhere, but it is hard for me to see where it fails at this point.

On Thu, Dec 29, 2011 at 12:13 AM, Dmitriy Lyubimov  wrote:
> oh. it's because the synthetic input has only 4 singular values.
>
> On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov  wrote:
>> But it is not a problem reading U or V files, that's indeed what U and
>> V contain.
>>
>> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  wrote:
>>> U and V look suspect, degenerate (only 4 first columns are nonzero,
>>> the rest of matrices are zeros.
>>>
>>> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  
>>> wrote:
 Yeah, fails for me on ubuntu without any special environment issues.
 Which makes it easier, i can step thru.

 On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  wrote:
> What do checksums look like?
>
> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll 
> wrote:
>
>> I commented out the deletion of the dir in the tearDown.  Not sure if 
>> that
>> looks reasonable or not, but on the surface they look equivalent.
>>
>> Here's the contents of the dir on Ubuntu:
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>
>> Here's what my Mac looks like:
>> total 20296
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
>> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
>> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>>
>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>>
>> > Yeah.. but this is a difference from the correct answer.  I am 
>> > moderately
>> > sure that this is a problem writing to the temp directory.
>> >
>> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll > >wrote:
>> >
>> >> It's expecting the answer to be 0, but it's some really large value.
>> >>
>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
>> >> expected:<0.0> but was:<4131200.37>
>> >>
>> >>
>> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
>> >>
>> >>> I think that the answer is 0 because the model is not being read and 
>> >>> we
>> >> are
>> >>> swallowing an exception somewhere.  This is what an uninitialized
>> matrix
>> >>> would give as

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

oh. it's because the synthetic input has only 4 singular values.

On Wed, Dec 28, 2011 at 11:56 PM, Dmitriy Lyubimov  wrote:
> But it is not a problem reading U or V files, that's indeed what U and
> V contain.
>
> On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  wrote:
>> U and V look suspect, degenerate (only 4 first columns are nonzero,
>> the rest of matrices are zeros.
>>
>> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  wrote:
>>> Yeah, fails for me on ubuntu without any special environment issues.
>>> Which makes it easier, i can step thru.
>>>
>>> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  wrote:
 What do checksums look like?

 On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll 
 wrote:

> I commented out the deletion of the dir in the tearDown.  Not sure if that
> looks reasonable or not, but on the surface they look equivalent.
>
> Here's the contents of the dir on Ubuntu:
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>
> Here's what my Mac looks like:
> total 20296
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>
> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>
> > Yeah.. but this is a difference from the correct answer.  I am 
> > moderately
> > sure that this is a problem writing to the temp directory.
> >
> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll  >wrote:
> >
> >> It's expecting the answer to be 0, but it's some really large value.
> >>
> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
> >> expected:<0.0> but was:<4131200.37>
> >>
> >>
> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
> >>
> >>> I think that the answer is 0 because the model is not being read and 
> >>> we
> >> are
> >>> swallowing an exception somewhere.  This is what an uninitialized
> matrix
> >>> would give as a result.
> >>>
> >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll  >>> wrote:
> >>>
>  I can reproduce outside of Jenkins.  It really seems odd that the
> answer
>  is off by so much.
> 
>  On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
> 
> > I vaguely remember Jenkins had problems with creating stuff in Java
> tmp
> > dir. E.g. I remember that was creating problems for Mr tasks in 
> > local
> >> mr
> > mode legitimately using boxed task temporary space.
> >
>>

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

But it is not a problem reading U or V files, that's indeed what U and
V contain.

On Wed, Dec 28, 2011 at 11:49 PM, Dmitriy Lyubimov  wrote:
> U and V look suspect, degenerate (only 4 first columns are nonzero,
> the rest of matrices are zeros.
>
> On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  wrote:
>> Yeah, fails for me on ubuntu without any special environment issues.
>> Which makes it easier, i can step thru.
>>
>> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  wrote:
>>> What do checksums look like?
>>>
>>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll wrote:
>>>
 I commented out the deletion of the dir in the tearDown.  Not sure if that
 looks reasonable or not, but on the surface they look equivalent.

 Here's the contents of the dir on Ubuntu:
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
 -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
 -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
 -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
 -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
 -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
 -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
 -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
 -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4

 Here's what my Mac looks like:
 total 20296
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
 -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
 -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
 -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
 -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
 -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
 -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
 -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4

 On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:

 > Yeah.. but this is a difference from the correct answer.  I am moderately
 > sure that this is a problem writing to the temp directory.
 >
 > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll >>> >wrote:
 >
 >> It's expecting the answer to be 0, but it's some really large value.
 >>
 testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
 >> expected:<0.0> but was:<4131200.37>
 >>
 >>
 >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
 >>
 >>> I think that the answer is 0 because the model is not being read and we
 >> are
 >>> swallowing an exception somewhere.  This is what an uninitialized
 matrix
 >>> would give as a result.
 >>>
 >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll >>> >>> wrote:
 >>>
  I can reproduce outside of Jenkins.  It really seems odd that the
 answer
  is off by so much.
 
  On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
 
 > I vaguely remember Jenkins had problems with creating stuff in Java
 tmp
 > dir. E.g. I remember that was creating problems for Mr tasks in local
 >> mr
 > mode legitimately using boxed task temporary space.
 >
 > OK I'll try to scan for the problem tomorrow.
 > On Dec 27, 2011 10:50 PM, "Ted Dunning" 
 wrote:
 >
 >> So I am like everybody else.  The test works for me.
 >>
 >> My suspicion is that there is something going on 

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

U and V look suspect, degenerate (only 4 first columns are nonzero,
the rest of matrices are zeros.

On Wed, Dec 28, 2011 at 11:44 PM, Dmitriy Lyubimov  wrote:
> Yeah, fails for me on ubuntu without any special environment issues.
> Which makes it easier, i can step thru.
>
> On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  wrote:
>> What do checksums look like?
>>
>> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll wrote:
>>
>>> I commented out the deletion of the dir in the tearDown.  Not sure if that
>>> looks reasonable or not, but on the surface they look equivalent.
>>>
>>> Here's the contents of the dir on Ubuntu:
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>>
>>> Here's what my Mac looks like:
>>> total 20296
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
>>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
>>> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
>>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
>>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
>>> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
>>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
>>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>>>
>>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>>>
>>> > Yeah.. but this is a difference from the correct answer.  I am moderately
>>> > sure that this is a problem writing to the temp directory.
>>> >
>>> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll >> >wrote:
>>> >
>>> >> It's expecting the answer to be 0, but it's some really large value.
>>> >>
>>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
>>> >> expected:<0.0> but was:<4131200.37>
>>> >>
>>> >>
>>> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
>>> >>
>>> >>> I think that the answer is 0 because the model is not being read and we
>>> >> are
>>> >>> swallowing an exception somewhere.  This is what an uninitialized
>>> matrix
>>> >>> would give as a result.
>>> >>>
>>> >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll >> >>> wrote:
>>> >>>
>>>  I can reproduce outside of Jenkins.  It really seems odd that the
>>> answer
>>>  is off by so much.
>>> 
>>>  On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
>>> 
>>> > I vaguely remember Jenkins had problems with creating stuff in Java
>>> tmp
>>> > dir. E.g. I remember that was creating problems for Mr tasks in local
>>> >> mr
>>> > mode legitimately using boxed task temporary space.
>>> >
>>> > OK I'll try to scan for the problem tomorrow.
>>> > On Dec 27, 2011 10:50 PM, "Ted Dunning" 
>>> wrote:
>>> >
>>> >> So I am like everybody else.  The test works for me.
>>> >>
>>> >> My suspicion is that there is something going on with the temporary
>>> >> directory that I am trying to use and that the environment that
>>> >> Jenkins
>>>  is
>>> >> using is somehow strange.
>>> >>
>>> >> The only slightly surprising idiom I am using is to create a
>>> temp

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

Yeah, fails for me on ubuntu without any special environment issues.
Which makes it easier, i can step thru.

On Wed, Dec 28, 2011 at 9:01 PM, Ted Dunning  wrote:
> What do checksums look like?
>
> On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll wrote:
>
>> I commented out the deletion of the dir in the tearDown.  Not sure if that
>> looks reasonable or not, but on the surface they look equivalent.
>>
>> Here's the contents of the dir on Ubuntu:
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
>> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
>> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
>> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
>> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
>> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
>> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
>> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
>> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>>
>> Here's what my Mac looks like:
>> total 20296
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
>> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
>> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
>> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
>> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
>> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
>> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>>
>> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>>
>> > Yeah.. but this is a difference from the correct answer.  I am moderately
>> > sure that this is a problem writing to the temp directory.
>> >
>> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll > >wrote:
>> >
>> >> It's expecting the answer to be 0, but it's some really large value.
>> >>
>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
>> >> expected:<0.0> but was:<4131200.37>
>> >>
>> >>
>> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
>> >>
>> >>> I think that the answer is 0 because the model is not being read and we
>> >> are
>> >>> swallowing an exception somewhere.  This is what an uninitialized
>> matrix
>> >>> would give as a result.
>> >>>
>> >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll > >>> wrote:
>> >>>
>>  I can reproduce outside of Jenkins.  It really seems odd that the
>> answer
>>  is off by so much.
>> 
>>  On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
>> 
>> > I vaguely remember Jenkins had problems with creating stuff in Java
>> tmp
>> > dir. E.g. I remember that was creating problems for Mr tasks in local
>> >> mr
>> > mode legitimately using boxed task temporary space.
>> >
>> > OK I'll try to scan for the problem tomorrow.
>> > On Dec 27, 2011 10:50 PM, "Ted Dunning" 
>> wrote:
>> >
>> >> So I am like everybody else.  The test works for me.
>> >>
>> >> My suspicion is that there is something going on with the temporary
>> >> directory that I am trying to use and that the environment that
>> >> Jenkins
>>  is
>> >> using is somehow strange.
>> >>
>> >> The only slightly surprising idiom I am using is to create a
>> temporary
>> >> file, delete it and recreate it as a directory.  I even check the
>> >> return
>> >> values from the delete and the mkdir.
>> >>
>> >> I will keep looking.
>> >>
>> >> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning <
>> ted.dunn...@gma

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

What do checksums look like?

On Wed, Dec 28, 2011 at 6:33 PM, Grant Ingersoll wrote:

> I commented out the deletion of the dir in the tearDown.  Not sure if that
> looks reasonable or not, but on the surface they look equivalent.
>
> Here's the contents of the dir on Ubuntu:
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
> -rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
> -rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
> -rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
> -rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
> -rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
> -rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
> -rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
> -rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4
>
> Here's what my Mac looks like:
> total 20296
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
> -rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
> -rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
> -rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
> -rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
> -rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
> -rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4
>
> On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:
>
> > Yeah.. but this is a difference from the correct answer.  I am moderately
> > sure that this is a problem writing to the temp directory.
> >
> > On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll  >wrote:
> >
> >> It's expecting the answer to be 0, but it's some really large value.
> >>
> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
> >> expected:<0.0> but was:<4131200.37>
> >>
> >>
> >> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
> >>
> >>> I think that the answer is 0 because the model is not being read and we
> >> are
> >>> swallowing an exception somewhere.  This is what an uninitialized
> matrix
> >>> would give as a result.
> >>>
> >>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll  >>> wrote:
> >>>
>  I can reproduce outside of Jenkins.  It really seems odd that the
> answer
>  is off by so much.
> 
>  On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
> 
> > I vaguely remember Jenkins had problems with creating stuff in Java
> tmp
> > dir. E.g. I remember that was creating problems for Mr tasks in local
> >> mr
> > mode legitimately using boxed task temporary space.
> >
> > OK I'll try to scan for the problem tomorrow.
> > On Dec 27, 2011 10:50 PM, "Ted Dunning" 
> wrote:
> >
> >> So I am like everybody else.  The test works for me.
> >>
> >> My suspicion is that there is something going on with the temporary
> >> directory that I am trying to use and that the environment that
> >> Jenkins
>  is
> >> using is somehow strange.
> >>
> >> The only slightly surprising idiom I am using is to create a
> temporary
> >> file, delete it and recreate it as a directory.  I even check the
> >> return
> >> values from the delete and the mkdir.
> >>
> >> I will keep looking.
> >>
> >> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning <
> ted.dunn...@gmail.com>
> >> wrote:
> >>
> >>> Indeed it does.  Thanks for pointing that out.
> >>>
> >>> This error is very strange.
> >>>
> >>>
> >>> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov <
> >> dlie...@gmail.com
> >>> wrote:
> >>>
> >>>

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

I commented out the deletion of the dir in the tearDown.  Not sure if that 
looks reasonable or not, but on the surface they look equivalent.

Here's the contents of the dir on Ubuntu:
-rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-0
-rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00200
-rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00400
-rw-rw-r-- 1 XX XX 1632612 2011-12-28 21:17 A-00600
-rw-rw-r-- 1 XX XX 1387722 2011-12-28 21:17 A-00800
-rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-0
-rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00210
-rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00420
-rw-rw-r-- 1 XX XX  168312 2011-12-28 21:17 B-00630
-rw-rw-r-- 1 XX XX  144312 2011-12-28 21:17 B-00840
-rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-0
-rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-200
-rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-400
-rw-rw-r-- 1 XX XX  160412 2011-12-28 21:17 U-600
-rw-rw-r-- 1 XX XX  136352 2011-12-28 21:17 U-800
-rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-0
-rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-1
-rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-2
-rw-rw-r-- 1 XX XX  168432 2011-12-28 21:17 V-3
-rw-rw-r-- 1 XX XX  144372 2011-12-28 21:17 V-4

Here's what my Mac looks like:
total 20296
-rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-0
-rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00200
-rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00400
-rw-r--r--  1 XX  staff   1.6M Dec 28 21:28 A-00600
-rw-r--r--  1 XX  staff   1.3M Dec 28 21:28 A-00800
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-0
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00210
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00420
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 B-00630
-rw-r--r--  1 XX  staff   141K Dec 28 21:28 B-00840
-rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-0
-rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-200
-rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-400
-rw-r--r--  1 XX  staff   157K Dec 28 21:28 U-600
-rw-r--r--  1 XX  staff   133K Dec 28 21:28 U-800
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-0
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-1
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-2
-rw-r--r--  1 XX  staff   164K Dec 28 21:28 V-3
-rw-r--r--  1 XX  staff   141K Dec 28 21:28 V-4

On Dec 28, 2011, at 7:15 PM, Ted Dunning wrote:

> Yeah.. but this is a difference from the correct answer.  I am moderately
> sure that this is a problem writing to the temp directory.
> 
> On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll wrote:
> 
>> It's expecting the answer to be 0, but it's some really large value.
>> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
>> expected:<0.0> but was:<4131200.37>
>> 
>> 
>> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
>> 
>>> I think that the answer is 0 because the model is not being read and we
>> are
>>> swallowing an exception somewhere.  This is what an uninitialized matrix
>>> would give as a result.
>>> 
>>> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll >> wrote:
>>> 
 I can reproduce outside of Jenkins.  It really seems odd that the answer
 is off by so much.
 
 On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
 
> I vaguely remember Jenkins had problems with creating stuff in Java tmp
> dir. E.g. I remember that was creating problems for Mr tasks in local
>> mr
> mode legitimately using boxed task temporary space.
> 
> OK I'll try to scan for the problem tomorrow.
> On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
> 
>> So I am like everybody else.  The test works for me.
>> 
>> My suspicion is that there is something going on with the temporary
>> directory that I am trying to use and that the environment that
>> Jenkins
 is
>> using is somehow strange.
>> 
>> The only slightly surprising idiom I am using is to create a temporary
>> file, delete it and recreate it as a directory.  I even check the
>> return
>> values from the delete and the mkdir.
>> 
>> I will keep looking.
>> 
>> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
>> wrote:
>> 
>>> Indeed it does.  Thanks for pointing that out.
>>> 
>>> This error is very strange.
>>> 
>>> 
>>> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov <
>> dlie...@gmail.com
>>> wrote:
>>> 
 Ted,
 
 do you have an idea why this test may be failing? I think this test
>> comes
 with M-792 commit.
 
 I can take a look at it, I suspect something in the environment can
>> be
 tripping it.
 On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
 
>>

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

Yeah.. but this is a difference from the correct answer.  I am moderately
sure that this is a problem writing to the temp directory.

On Wed, Dec 28, 2011 at 3:45 PM, Grant Ingersoll wrote:

> It's expecting the answer to be 0, but it's some really large value.
> testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest):
> expected:<0.0> but was:<4131200.37>
>
>
> On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:
>
> > I think that the answer is 0 because the model is not being read and we
> are
> > swallowing an exception somewhere.  This is what an uninitialized matrix
> > would give as a result.
> >
> > On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll  >wrote:
> >
> >> I can reproduce outside of Jenkins.  It really seems odd that the answer
> >> is off by so much.
> >>
> >> On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
> >>
> >>> I vaguely remember Jenkins had problems with creating stuff in Java tmp
> >>> dir. E.g. I remember that was creating problems for Mr tasks in local
> mr
> >>> mode legitimately using boxed task temporary space.
> >>>
> >>> OK I'll try to scan for the problem tomorrow.
> >>> On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
> >>>
>  So I am like everybody else.  The test works for me.
> 
>  My suspicion is that there is something going on with the temporary
>  directory that I am trying to use and that the environment that
> Jenkins
> >> is
>  using is somehow strange.
> 
>  The only slightly surprising idiom I am using is to create a temporary
>  file, delete it and recreate it as a directory.  I even check the
> return
>  values from the delete and the mkdir.
> 
>  I will keep looking.
> 
>  On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
>  wrote:
> 
> > Indeed it does.  Thanks for pointing that out.
> >
> > This error is very strange.
> >
> >
> > On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov <
> dlie...@gmail.com
> > wrote:
> >
> >> Ted,
> >>
> >> do you have an idea why this test may be failing? I think this test
>  comes
> >> with M-792 commit.
> >>
> >> I can take a look at it, I suspect something in the environment can
> be
> >> tripping it.
> >> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
> >>
> >>> It's all errors in the Apache infrastructure, rather than a real
> test
> >>> failure. At least, stuff passes for me locally, and that's what's
> >>> important.
> >>> So I'm ignoring these.
> >>>
> >>> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
> >>>  wrote:
>  I'm getting a lot of these emails yet all the tests run locally
> for
> >> me.
> >>> Does
>  anybody have an idea what the problem is? This close to a release
> it
> >>> would
>  be really nice to have Jenkins on our side.
>  Jeff
> 
> >>>
> >>
> >
> >
> 
> >>
> >> 
> >> Grant Ingersoll
> >> http://www.lucidimagination.com
> >>
> >>
> >>
> >>
>
> 
> Grant Ingersoll
> http://www.lucidimagination.com
>
>
>
>

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

It's expecting the answer to be 0, but it's some really large value.
testSingularValues(org.apache.mahout.math.ssvd.SequentialOutOfCoreSvdTest): 
expected:<0.0> but was:<4131200.37>


On Dec 28, 2011, at 6:30 PM, Ted Dunning wrote:

> I think that the answer is 0 because the model is not being read and we are
> swallowing an exception somewhere.  This is what an uninitialized matrix
> would give as a result.
> 
> On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll wrote:
> 
>> I can reproduce outside of Jenkins.  It really seems odd that the answer
>> is off by so much.
>> 
>> On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
>> 
>>> I vaguely remember Jenkins had problems with creating stuff in Java tmp
>>> dir. E.g. I remember that was creating problems for Mr tasks in local mr
>>> mode legitimately using boxed task temporary space.
>>> 
>>> OK I'll try to scan for the problem tomorrow.
>>> On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
>>> 
 So I am like everybody else.  The test works for me.
 
 My suspicion is that there is something going on with the temporary
 directory that I am trying to use and that the environment that Jenkins
>> is
 using is somehow strange.
 
 The only slightly surprising idiom I am using is to create a temporary
 file, delete it and recreate it as a directory.  I even check the return
 values from the delete and the mkdir.
 
 I will keep looking.
 
 On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
 wrote:
 
> Indeed it does.  Thanks for pointing that out.
> 
> This error is very strange.
> 
> 
> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov  wrote:
> 
>> Ted,
>> 
>> do you have an idea why this test may be failing? I think this test
 comes
>> with M-792 commit.
>> 
>> I can take a look at it, I suspect something in the environment can be
>> tripping it.
>> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
>> 
>>> It's all errors in the Apache infrastructure, rather than a real test
>>> failure. At least, stuff passes for me locally, and that's what's
>>> important.
>>> So I'm ignoring these.
>>> 
>>> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
>>>  wrote:
 I'm getting a lot of these emails yet all the tests run locally for
>> me.
>>> Does
 anybody have an idea what the problem is? This close to a release it
>>> would
 be really nice to have Jenkins on our side.
 Jeff
 
>>> 
>> 
> 
> 
 
>> 
>> 
>> Grant Ingersoll
>> http://www.lucidimagination.com
>> 
>> 
>> 
>> 


Grant Ingersoll
http://www.lucidimagination.com

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

Fails on Ubuntu, but passes on my Mac.

On Dec 28, 2011, at 6:21 PM, Grant Ingersoll wrote:

> I can reproduce outside of Jenkins.  It really seems odd that the answer is 
> off by so much.
> 
> On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
> 
>> I vaguely remember Jenkins had problems with creating stuff in Java tmp
>> dir. E.g. I remember that was creating problems for Mr tasks in local mr
>> mode legitimately using boxed task temporary space.
>> 
>> OK I'll try to scan for the problem tomorrow.
>> On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
>> 
>>> So I am like everybody else.  The test works for me.
>>> 
>>> My suspicion is that there is something going on with the temporary
>>> directory that I am trying to use and that the environment that Jenkins is
>>> using is somehow strange.
>>> 
>>> The only slightly surprising idiom I am using is to create a temporary
>>> file, delete it and recreate it as a directory.  I even check the return
>>> values from the delete and the mkdir.
>>> 
>>> I will keep looking.
>>> 
>>> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
>>> wrote:
>>> 
 Indeed it does.  Thanks for pointing that out.
 
 This error is very strange.
 
 
 On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov >>> wrote:
 
> Ted,
> 
> do you have an idea why this test may be failing? I think this test
>>> comes
> with M-792 commit.
> 
> I can take a look at it, I suspect something in the environment can be
> tripping it.
> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
> 
>> It's all errors in the Apache infrastructure, rather than a real test
>> failure. At least, stuff passes for me locally, and that's what's
>> important.
>> So I'm ignoring these.
>> 
>> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
>>  wrote:
>>> I'm getting a lot of these emails yet all the tests run locally for
> me.
>> Does
>>> anybody have an idea what the problem is? This close to a release it
>> would
>>> be really nice to have Jenkins on our side.
>>> Jeff
>>> 
>> 
> 
 
 
>>> 
> 
> 
> Grant Ingersoll
> http://www.lucidimagination.com
> 
> 
> 


Grant Ingersoll
http://www.lucidimagination.com

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

I think that the answer is 0 because the model is not being read and we are
swallowing an exception somewhere.  This is what an uninitialized matrix
would give as a result.

On Wed, Dec 28, 2011 at 3:21 PM, Grant Ingersoll wrote:

> I can reproduce outside of Jenkins.  It really seems odd that the answer
> is off by so much.
>
> On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:
>
> > I vaguely remember Jenkins had problems with creating stuff in Java tmp
> > dir. E.g. I remember that was creating problems for Mr tasks in local mr
> > mode legitimately using boxed task temporary space.
> >
> > OK I'll try to scan for the problem tomorrow.
> > On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
> >
> >> So I am like everybody else.  The test works for me.
> >>
> >> My suspicion is that there is something going on with the temporary
> >> directory that I am trying to use and that the environment that Jenkins
> is
> >> using is somehow strange.
> >>
> >> The only slightly surprising idiom I am using is to create a temporary
> >> file, delete it and recreate it as a directory.  I even check the return
> >> values from the delete and the mkdir.
> >>
> >> I will keep looking.
> >>
> >> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
> >> wrote:
> >>
> >>> Indeed it does.  Thanks for pointing that out.
> >>>
> >>> This error is very strange.
> >>>
> >>>
> >>> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov  >>> wrote:
> >>>
>  Ted,
> 
>  do you have an idea why this test may be failing? I think this test
> >> comes
>  with M-792 commit.
> 
>  I can take a look at it, I suspect something in the environment can be
>  tripping it.
>  On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
> 
> > It's all errors in the Apache infrastructure, rather than a real test
> > failure. At least, stuff passes for me locally, and that's what's
> > important.
> > So I'm ignoring these.
> >
> > On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
> >  wrote:
> >> I'm getting a lot of these emails yet all the tests run locally for
>  me.
> > Does
> >> anybody have an idea what the problem is? This close to a release it
> > would
> >> be really nice to have Jenkins on our side.
> >> Jeff
> >>
> >
> 
> >>>
> >>>
> >>
>
> 
> Grant Ingersoll
> http://www.lucidimagination.com
>
>
>
>

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

I can reproduce outside of Jenkins.  It really seems odd that the answer is off 
by so much.

On Dec 28, 2011, at 2:15 AM, Dmitriy Lyubimov wrote:

> I vaguely remember Jenkins had problems with creating stuff in Java tmp
> dir. E.g. I remember that was creating problems for Mr tasks in local mr
> mode legitimately using boxed task temporary space.
> 
> OK I'll try to scan for the problem tomorrow.
> On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:
> 
>> So I am like everybody else.  The test works for me.
>> 
>> My suspicion is that there is something going on with the temporary
>> directory that I am trying to use and that the environment that Jenkins is
>> using is somehow strange.
>> 
>> The only slightly surprising idiom I am using is to create a temporary
>> file, delete it and recreate it as a directory.  I even check the return
>> values from the delete and the mkdir.
>> 
>> I will keep looking.
>> 
>> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
>> wrote:
>> 
>>> Indeed it does.  Thanks for pointing that out.
>>> 
>>> This error is very strange.
>>> 
>>> 
>>> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov >> wrote:
>>> 
 Ted,
 
 do you have an idea why this test may be failing? I think this test
>> comes
 with M-792 commit.
 
 I can take a look at it, I suspect something in the environment can be
 tripping it.
 On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
 
> It's all errors in the Apache infrastructure, rather than a real test
> failure. At least, stuff passes for me locally, and that's what's
> important.
> So I'm ignoring these.
> 
> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
>  wrote:
>> I'm getting a lot of these emails yet all the tests run locally for
 me.
> Does
>> anybody have an idea what the problem is? This close to a release it
> would
>> be really nice to have Jenkins on our side.
>> Jeff
>> 
> 
 
>>> 
>>> 
>> 


Grant Ingersoll
http://www.lucidimagination.com

RE: Build failed in Jenkins: mahout-nightly #737

[Gavin McDonald]

> -Original Message-
> From: Jeff Eastman [mailto:j...@windwardsolutions.com]
> Sent: Wednesday, 28 December 2011 3:31 PM
> To: dev@mahout.apache.org; infrastruct...@apache.org
> Subject: Re: Build failed in Jenkins: mahout-nightly #737
> 
> +infra@

All build related questions should go to the bui...@apache.org list.


> 
> Well, yes, I guess everybody is ignoring them. But that kinda begs the
> question of why we even bother to have Jenkins if we are going to always
> ignore its warnings? I'm a big fan of continuous integration and I've managed
> its administration before so I understand the challenges. But this seems
> kinda goofy to me.

It is not up to infra to check projects failed builds! It is up to the project.
It is also up to the project to configure their Jenkins configs.

Now, if the project needs any pre-reqs installed on any slaves to help make the
build pass, by all means ask the builds@ list, otherwise what are we infra 
expected to
do?

If Jenkins isn’t working out for you, maybe try Buildbot or Continuum , see
ci.apache.org for more.

Gav...

> 
> 
> On 12/27/11 9:53 PM, Sean Owen wrote:
> > It's all errors in the Apache infrastructure, rather than a real test
> > failure. At least, stuff passes for me locally, and that's what's
> > important.
> > So I'm ignoring these.
> >
> > On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
> >   wrote:
> >> I'm getting a lot of these emails yet all the tests run locally for
> >> me. Does anybody have an idea what the problem is? This close to a
> >> release it would be really nice to have Jenkins on our side.
> >> Jeff
> >>
> >

Re: Build failed in Jenkins: mahout-nightly #737

[Grant Ingersoll]

I'm seeing this pretty consistently on my Jenkins locally, but then when I run 
on my Mac, it passes.

On Dec 27, 2011, at 10:34 PM, Jeff Eastman wrote:

> I'm getting a lot of these emails yet all the tests run locally for me. Does 
> anybody have an idea what the problem is? This close to a release it would be 
> really nice to have Jenkins on our side.
> Jeff
> 
> 
> On 12/27/11 4:54 PM, Apache Jenkins Server wrote:
>> See
>> 
>> --
>> [...truncated 2536 lines...]
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
>> Running 
>> org.apache.mahout.classifier.naivebayes.training.IndexInstancesMapperTest
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
>> Running org.apache.mahout.classifier.naivebayes.training.ThetaMapperTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
>> Running org.apache.mahout.classifier.naivebayes.training.WeightsMapperTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec
>> Running org.apache.mahout.classifier.bayes.BayesFeatureMapperTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 sec
>> Running org.apache.mahout.classifier.bayes.BayesClassifierSelfTest
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.86 sec
>> Running org.apache.mahout.classifier.bayes.BayesFeatureMapReduceTest
>> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
>> Running org.apache.mahout.classifier.bayes.BayesFileFormatterTest
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
>> Running org.apache.mahout.classifier.bayes.BayesClassifierTest
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
>> Running org.apache.mahout.classifier.bayes.CBayesClassifierTest
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
>> Running org.apache.mahout.classifier.evaluation.AucTest
>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.185 sec
>> Running org.apache.mahout.classifier.discriminative.WinnowTrainerTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
>> Running org.apache.mahout.classifier.discriminative.PerceptronTrainerTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
>> Running org.apache.mahout.classifier.discriminative.LinearModelTest
>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
>> Running org.apache.mahout.clustering.kmeans.TestKmeansClustering
>> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 244.039 sec
>> Running org.apache.mahout.clustering.kmeans.TestRandomSeedGenerator
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec
>> Running org.apache.mahout.clustering.TestClusterInterface
>> Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
>> Running org.apache.mahout.clustering.TestClusterClassifier
>> Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.077 sec
>> Running org.apache.mahout.clustering.lda.TestLDAInference
>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.166 sec
>> Running org.apache.mahout.clustering.lda.TestMapReduce
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.059 sec
>> Running org.apache.mahout.clustering.lda.cvb.TestCVBModelTrainer
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 233.081 sec
>> Running org.apache.mahout.clustering.TestGaussianAccumulators
>> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.912 sec
>> Running 
>> org.apache.mahout.clustering.spectral.common.TestAffinityMatrixInputJob
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
>> Running org.apache.mahout.clustering.spectral.common.TestVectorCache
>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec
>> Running org.apache.mahout.clustering.spectral.common.TestUnitVectorizerJob
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
>> Running org.apache.mahout.clustering.spectral.common.TestMatrixDiagonalizeJob
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
>> Running 
>> org.apache.mahout.clustering.spectral.common.TestVectorMatrixMultiplicationJob
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec
>> Running 
>> org.apache.mahout.clustering.spectral.eigencuts.TestEigencutsAffinityCutsJob
>> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec
>> Running 
>> org.apache.mahout.clustering.spectral.eigencuts.TestEigencutsSensitivityJob
>> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec
>> Running org.apache.mahout.clustering.minhash.TestMinHashClustering
>> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.182 sec
>> Runn

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

I vaguely remember Jenkins had problems with creating stuff in Java tmp
dir. E.g. I remember that was creating problems for Mr tasks in local mr
mode legitimately using boxed task temporary space.

OK I'll try to scan for the problem tomorrow.
On Dec 27, 2011 10:50 PM, "Ted Dunning"  wrote:

> So I am like everybody else.  The test works for me.
>
> My suspicion is that there is something going on with the temporary
> directory that I am trying to use and that the environment that Jenkins is
> using is somehow strange.
>
> The only slightly surprising idiom I am using is to create a temporary
> file, delete it and recreate it as a directory.  I even check the return
> values from the delete and the mkdir.
>
> I will keep looking.
>
> On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning 
> wrote:
>
> > Indeed it does.  Thanks for pointing that out.
> >
> > This error is very strange.
> >
> >
> > On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov  >wrote:
> >
> >> Ted,
> >>
> >> do you have an idea why this test may be failing? I think this test
> comes
> >> with M-792 commit.
> >>
> >> I can take a look at it, I suspect something in the environment can be
> >> tripping it.
> >> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
> >>
> >> > It's all errors in the Apache infrastructure, rather than a real test
> >> > failure. At least, stuff passes for me locally, and that's what's
> >> > important.
> >> > So I'm ignoring these.
> >> >
> >> > On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
> >> >  wrote:
> >> > > I'm getting a lot of these emails yet all the tests run locally for
> >> me.
> >> > Does
> >> > > anybody have an idea what the problem is? This close to a release it
> >> > would
> >> > > be really nice to have Jenkins on our side.
> >> > > Jeff
> >> > >
> >> >
> >>
> >
> >
>

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

So I am like everybody else.  The test works for me.

My suspicion is that there is something going on with the temporary
directory that I am trying to use and that the environment that Jenkins is
using is somehow strange.

The only slightly surprising idiom I am using is to create a temporary
file, delete it and recreate it as a directory.  I even check the return
values from the delete and the mkdir.

I will keep looking.

On Tue, Dec 27, 2011 at 10:37 PM, Ted Dunning  wrote:

> Indeed it does.  Thanks for pointing that out.
>
> This error is very strange.
>
>
> On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov wrote:
>
>> Ted,
>>
>> do you have an idea why this test may be failing? I think this test comes
>> with M-792 commit.
>>
>> I can take a look at it, I suspect something in the environment can be
>> tripping it.
>> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
>>
>> > It's all errors in the Apache infrastructure, rather than a real test
>> > failure. At least, stuff passes for me locally, and that's what's
>> > important.
>> > So I'm ignoring these.
>> >
>> > On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
>> >  wrote:
>> > > I'm getting a lot of these emails yet all the tests run locally for
>> me.
>> > Does
>> > > anybody have an idea what the problem is? This close to a release it
>> > would
>> > > be really nice to have Jenkins on our side.
>> > > Jeff
>> > >
>> >
>>
>
>

Re: Build failed in Jenkins: mahout-nightly #737

[Ted Dunning]

Indeed it does.  Thanks for pointing that out.

This error is very strange.


On Tue, Dec 27, 2011 at 10:06 PM, Dmitriy Lyubimov wrote:

> Ted,
>
> do you have an idea why this test may be failing? I think this test comes
> with M-792 commit.
>
> I can take a look at it, I suspect something in the environment can be
> tripping it.
> On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:
>
> > It's all errors in the Apache infrastructure, rather than a real test
> > failure. At least, stuff passes for me locally, and that's what's
> > important.
> > So I'm ignoring these.
> >
> > On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
> >  wrote:
> > > I'm getting a lot of these emails yet all the tests run locally for me.
> > Does
> > > anybody have an idea what the problem is? This close to a release it
> > would
> > > be really nice to have Jenkins on our side.
> > > Jeff
> > >
> >
>

Re: Build failed in Jenkins: mahout-nightly #737

[Dmitriy Lyubimov]

Ted,

do you have an idea why this test may be failing? I think this test comes
with M-792 commit.

I can take a look at it, I suspect something in the environment can be
tripping it.
On Dec 27, 2011 8:54 PM, "Sean Owen"  wrote:

> It's all errors in the Apache infrastructure, rather than a real test
> failure. At least, stuff passes for me locally, and that's what's
> important.
> So I'm ignoring these.
>
> On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
>  wrote:
> > I'm getting a lot of these emails yet all the tests run locally for me.
> Does
> > anybody have an idea what the problem is? This close to a release it
> would
> > be really nice to have Jenkins on our side.
> > Jeff
> >
>

Re: Build failed in Jenkins: mahout-nightly #737

[Jeff Eastman]

+infra@

Well, yes, I guess everybody is ignoring them. But that kinda begs the 
question of why we even bother to have Jenkins if we are going to always 
ignore its warnings? I'm a big fan of continuous integration and I've 
managed its administration before so I understand the challenges. But 
this seems kinda goofy to me.



On 12/27/11 9:53 PM, Sean Owen wrote:

It's all errors in the Apache infrastructure, rather than a real test
failure. At least, stuff passes for me locally, and that's what's
important.
So I'm ignoring these.

On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
  wrote:

I'm getting a lot of these emails yet all the tests run locally for me. Does
anybody have an idea what the problem is? This close to a release it would
be really nice to have Jenkins on our side.
Jeff

Re: Build failed in Jenkins: mahout-nightly #737

[Sean Owen]

It's all errors in the Apache infrastructure, rather than a real test
failure. At least, stuff passes for me locally, and that's what's
important.
So I'm ignoring these.

On Tue, Dec 27, 2011 at 9:34 PM, Jeff Eastman
 wrote:
> I'm getting a lot of these emails yet all the tests run locally for me. Does
> anybody have an idea what the problem is? This close to a release it would
> be really nice to have Jenkins on our side.
> Jeff
>

Re: Build failed in Jenkins: mahout-nightly #737

[Jeff Eastman]

I'm getting a lot of these emails yet all the tests run locally for me. 
Does anybody have an idea what the problem is? This close to a release 
it would be really nice to have Jenkins on our side.

Jeff


On 12/27/11 4:54 PM, Apache Jenkins Server wrote:

See

--
[...truncated 2536 lines...]
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running 
org.apache.mahout.classifier.naivebayes.training.IndexInstancesMapperTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
Running org.apache.mahout.classifier.naivebayes.training.ThetaMapperTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
Running org.apache.mahout.classifier.naivebayes.training.WeightsMapperTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec
Running org.apache.mahout.classifier.bayes.BayesFeatureMapperTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 sec
Running org.apache.mahout.classifier.bayes.BayesClassifierSelfTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.86 sec
Running org.apache.mahout.classifier.bayes.BayesFeatureMapReduceTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running org.apache.mahout.classifier.bayes.BayesFileFormatterTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
Running org.apache.mahout.classifier.bayes.BayesClassifierTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Running org.apache.mahout.classifier.bayes.CBayesClassifierTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Running org.apache.mahout.classifier.evaluation.AucTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.185 sec
Running org.apache.mahout.classifier.discriminative.WinnowTrainerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running org.apache.mahout.classifier.discriminative.PerceptronTrainerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Running org.apache.mahout.classifier.discriminative.LinearModelTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running org.apache.mahout.clustering.kmeans.TestKmeansClustering
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 244.039 sec
Running org.apache.mahout.clustering.kmeans.TestRandomSeedGenerator
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec
Running org.apache.mahout.clustering.TestClusterInterface
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running org.apache.mahout.clustering.TestClusterClassifier
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.077 sec
Running org.apache.mahout.clustering.lda.TestLDAInference
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.166 sec
Running org.apache.mahout.clustering.lda.TestMapReduce
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.059 sec
Running org.apache.mahout.clustering.lda.cvb.TestCVBModelTrainer
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 233.081 sec
Running org.apache.mahout.clustering.TestGaussianAccumulators
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.912 sec
Running org.apache.mahout.clustering.spectral.common.TestAffinityMatrixInputJob
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec
Running org.apache.mahout.clustering.spectral.common.TestVectorCache
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec
Running org.apache.mahout.clustering.spectral.common.TestUnitVectorizerJob
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
Running org.apache.mahout.clustering.spectral.common.TestMatrixDiagonalizeJob
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running 
org.apache.mahout.clustering.spectral.common.TestVectorMatrixMultiplicationJob
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec
Running 
org.apache.mahout.clustering.spectral.eigencuts.TestEigencutsAffinityCutsJob
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec
Running 
org.apache.mahout.clustering.spectral.eigencuts.TestEigencutsSensitivityJob
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec
Running org.apache.mahout.clustering.minhash.TestMinHashClustering
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.182 sec
Running org.apache.mahout.clustering.topdown.PathDirectoryTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Running 
org.apache.mahout.clustering.topdown.postprocessor.ClusterOutputPostProcessorTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.048 sec
Running 
org.apache.mahout.clustering.topdown.postprocessor.ClusterCountRead