Re: Issue in test suite for armel (Was: Problems upgrading r-cran-fastcluster due to failed test in Python module)

[Gard Spreemann]

On Saturday, 19 November 2016 11:46:29 CET Gard Spreemann wrote:
> I guess we can just skip the test on armel?

Oh wait, scratch that, that's stupid: Now that I think about it, it's
certainly possible that fastcluster's code itself relies on correctly
identifying NaNs. I can have a closer look in the next few days.


--
Best,
Gard

Re: Issue in test suite for armel (Was: Problems upgrading r-cran-fastcluster due to failed test in Python module)

[Gard Spreemann]

On Friday, 18 November 2016 14:57:29 CET Andreas Tille wrote:
> Hi,
> 
> python-fastcluster seems to have an issue in the test suite for armel[1]:
>

Hi,

Interesting! Since the test passed on all other architectures, could
it be that certain Python floating point exceptions are simply not
raised on architectures without an FPU? I came across an old NumPy bug
[2] that seems very related. I don't have access to a real armel
device to test this, but I'll probably have time to play in qemu in a
few days.

I guess we can just skip the test on armel? If we decide to go that
route, I can get in touch with upstream and work on getting the change
to happen there too.

[2] https://github.com/numpy/numpy/pull/2879

--
Best,
Gard

Issue in test suite for armel (Was: Problems upgrading r-cran-fastcluster due to failed test in Python module)

[Andreas Tille]

Hi,

python-fastcluster seems to have an issue in the test suite for armel[1]:

...
Method: average...OK.
Meok
test_nan (tests.fastcluster_test) ... FAIL
test_vector (tests.fastcluster_test) ... thod: weighted...OK.
...
Metric: mahalanobis...Method: single...OK.
Metric: user...Mok

==
FAIL: test_nan (tests.fastcluster_test)
--
Traceback (most recent call last):
  File "/«PKGBUILDDIR»/src/python/tests/__init__.py", line 10, in test_nan
self.assertTrue(test())
  File "/«PKGBUILDDIR»/src/python/tests/nantest.py", line 48, in test
raise AssertionError('fastcluster did not detect a NaN value!')
AssertionError: fastcluster did not detect a NaN value!

--
Ran 3 tests in 406.754s

FAILED (failures=1)
ethod: single...OK.
...


Any hint how to proceed from here would be welcome.

Kind regards

  Andreas.


[1] 
https://buildd.debian.org/status/fetch.php?pkg=r-cran-fastcluster&arch=armel&ver=1.1.21-1&stamp=1479436093
 

-- 
http://fam-tille.de

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Andreas Tille]

On Thu, Nov 17, 2016 at 12:10:03PM +0100, Gard Spreemann wrote:
> ...
> I think it would be a good idea for python-fastcluster to recommend
> python-scipy and python3-fastcluster to recommend python3-scipy.

Uploaded (unfortunately without the additional Recommends which I
learned to late since I just prepared the package after the first hint
to scipy. 

I commited the Recommends in SVN to make sure it will not be forgotten
for the next upload.

Thanks to all for the hints

Andreas.

-- 
http://fam-tille.de

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

On Thursday, 17 November 2016 11:24:36 CET Gard Spreemann wrote:
> It's pretty obvious now: The tests start with
> 
>  from scipy.spatial.distance import pdist, squareform
> 
> so they require scipy. I suggest that python-scipy and python3-scipy
> are added as build-deps.
>

I now see that the code itself (i.e. not just the tests) refer
optionally to SciPy. For example:

 try:
 from scipy.spatial.distance import pdist
 except ImportError:
 def pdist(*args, **kwargs):
 raise ImportError('The fastcluster.linkage function cannot process '
   'vector data since the function '
   'scipy.partial.distance.pdist could not be  '
   'imported.')

I think it would be a good idea for python-fastcluster to recommend
python-scipy and python3-fastcluster to recommend python3-scipy.


--
 Best,
 Gard
 

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Tristan Seligmann]

On Thu, 17 Nov 2016 at 12:25 Gard Spreemann  wrote:

> The weird thing is that this error does not occur when building on my
> ordinary system (outside of the pbuilder chroot). Does pbuilder not
> allow non-ASCII output or something?
>

Pbuilder is probably defaulting the locale to C, whereas your system/user
locale is probably a UTF-8 locale (eg. en_US.UTF-8). You can probably
resolve this issue with an `export LC_ALL=C.UTF-8` in debian/rules.

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

On Thursday, 17 November 2016 10:50:06 CET Tristan Seligmann wrote:
> On Thu, 17 Nov 2016 at 12:25 Gard Spreemann  wrote:
> > The weird thing is that this error does not occur when building on my
> > ordinary system (outside of the pbuilder chroot). Does pbuilder not
> > allow non-ASCII output or something?
> 
> Pbuilder is probably defaulting the locale to C, whereas your system/user
> locale is probably a UTF-8 locale (eg. en_US.UTF-8). You can probably
> resolve this issue with an `export LC_ALL=C.UTF-8` in debian/rules.

Thanks!

With "export LC_ALL=C.UTF-8" in debian/rules and the added
python-scipy and python3-scipy build-deps, the new version of the
package builds and tests fine in my pbuilder.


--
 Best,
 Gard
 

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

On Thursday, 17 November 2016 11:24:36 CET Gard Spreemann wrote:
> However, the python3 version now fails with:
> 
>   File "/build/r-cran-fastcluster-1.1.21/src/python/tests/nantest.py", line
> 7, in 
> Copyright (c) 2011 Daniel M\xfcllner, ''')
>  UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position
> 72: ordinal not in range(128)
>  E: pybuild pybuild:276: test: plugin distutils failed with: exit code=1:
> python3.5 setup.py test
>

Sorry, I somehow skipped a line in that output. For completeness, the
corresponding lines in src/python/tests/nantest.py (and test.py and
vectortest.py) are:

 print('''
 Test program for the 'fastcluster' package.
 Copyright (c) 2011 Daniel Müllner, ''')


--
 Best,
 Gard
 

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

Hi again, and sorry for taking so long.

It's pretty obvious now: The tests start with

 from scipy.spatial.distance import pdist, squareform

so they require scipy. I suggest that python-scipy and python3-scipy
are added as build-deps. That makes the python2 version build and test
fine also in my pbuilder chroot.

However, the python3 version now fails with:

  File "/build/r-cran-fastcluster-1.1.21/src/python/tests/nantest.py", line 7, 
in 
Copyright (c) 2011 Daniel M\xfcllner, ''')
 UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 
72: ordinal not in range(128)
 E: pybuild pybuild:276: test: plugin distutils failed with: exit code=1: 
python3.5 setup.py test

The weird thing is that this error does not occur when building on my
ordinary system (outside of the pbuilder chroot). Does pbuilder not
allow non-ASCII output or something?

I guess it's not exactly critical that the upstream author's name is
output correctly during the build, but still. Any ideas?


--
 Best,
  Gard
  

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

Hi Andreas,

On Wednesday, 9 November 2016 17:08:01 CET Andreas Tille wrote:
> I do not think so.  Could you possibly try in a pbuilder sid chroot
> whether you might be able to reproduce the issue?
>

In fact, it fails in the same way both in a Sid pbuilder and in a
Stretch pbuilder, but not on my ordinary Stretch system.

> > Let me know if I can be of assistance.
> 
> I think it makes a difference whether it is sid or testing or may be the
> difference is if its on the running system or in a chroot environment.
> The latter might be a hint that some Build-Dependency is missing which
> is by chance installed on the local machine ... but which one?
>

The latter definitely seems to be the case. Hopefully I'll have more
time to investigate on Friday.


 Best,
  Gard

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Andreas Tille]

Hi Gard,

thanks for trying to track down the issue.

On Wed, Nov 09, 2016 at 03:26:04PM +0100, Gard Spreemann wrote:
> On Tuesday, 8 November 2016 18:38:22 CET Andreas Tille wrote:
> > Hi,
> > 
> > I tried to upgrade r-cran-fastcluster[1] which also created Python
> > modules thanks to Gard Spreemann).  Unfortunately the unit test of the
> > new version fails with […]
> Hi all,
> 
> I tried building on Stretch with the latest upstream (1.1.21) and the
> Debian changes from your SVN [1]. Weirdly enough, I don't see any
> errors, and the package builds nicely with all tests running and
> passing.

I can confirm when running *debuild* on a testing system the package
builds nicely also at my side.
 
> Could there be some weirdly set environment variable or something that
> causes the vectortest module (shipped by upstream) not to be found?

I do not think so.  Could you possibly try in a pbuilder sid chroot
whether you might be able to reproduce the issue?
 
> Let me know if I can be of assistance.

I think it makes a difference whether it is sid or testing or may be the
difference is if its on the running system or in a chroot environment.
The latter might be a hint that some Build-Dependency is missing which
is by chance installed on the local machine ... but which one?

Kind regards

 Andreas.
 
> [1] svn://anonscm.debian.org/debian-med/trunk/packages/R/r-cran-fastcluster/
> trunk/

-- 
http://fam-tille.de

Re: Problems upgrading r-cran-fastcluster due to failed test in Python module

[Gard Spreemann]

On Tuesday, 8 November 2016 18:38:22 CET Andreas Tille wrote:
> Hi,
> 
> I tried to upgrade r-cran-fastcluster[1] which also created Python
> modules thanks to Gard Spreemann).  Unfortunately the unit test of the
> new version fails with […]
> 


Hi all,

I tried building on Stretch with the latest upstream (1.1.21) and the
Debian changes from your SVN [1]. Weirdly enough, I don't see any
errors, and the package builds nicely with all tests running and
passing.

Could there be some weirdly set environment variable or something that
causes the vectortest module (shipped by upstream) not to be found?

Let me know if I can be of assistance.

[1] svn://anonscm.debian.org/debian-med/trunk/packages/R/r-cran-fastcluster/
trunk/


 Best,
  Gard