Re: [help] porting brian on MIPS and POWER, or dropping support
On Sat, 30 Jan 2021 at 17:18, Étienne Mollier wrote: > Hi Michael, > > Michael Crusoe, on 2021-01-30 16:51:27 +0100: > > On Sat, 30 Jan 2021 at 14:33, Étienne Mollier < > etienne.moll...@mailoo.org> > > wrote: > > > > [1] https://salsa.debian.org/med-team/brian/-/tree/master > > > > > https://salsa.debian.org/med-team/brian/-/blob/master/debian/patches/gsl-compiler-arg.patch#L29 > > > > -march=native is not allowed as it violates the architecture baseline, > > unless there is an alternative binary or library built without it. > > I would generally agree, but in this case I understood the code > was compiled (in the background) on the end user's machine[3]. > I don't exclude that I could have misunderstood the mechanism, > so am triple checking at the moment. I probably should have put > the link in the patch header, by the way. > > [3] https://brian2.readthedocs.io/en/stable/user/computation.html If it is truly for local use, then -march=native is great, but not every architecture supports "-march=native", so you may need to add work arounds for archs that don't have that gcc option. Here is the result of some reading of the gcc manual page -march=native & -mtune=native is valid in gcc for the following Debian archs: arm* mips* s390x i386/amd64/x32 but not these HPPA M68k riscv64 power*/ppc* (no -march at all, but does have -mcpu=native and -mtune=native) alpha (no -march at all, but does have -mcpu=native and -mtune=native) sh4 (no -march at all, no -mcpu either) sparc64 (no -march at all, but does have -mcpu=native and -mtune=native) ia64 (no -march, has -mtune but not -mtune=native) > > In any way, I guess putting more regular compiler flags at first > should give a far safer outcome than current architecture > dependent selection. > > Thanks for your review! :) > You are welcome -- Michael R. Crusoe
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi Nilesh, Nilesh Patra, on 2021-01-30 22:57:55 +0530: > Unfortunately, ssh'ing on "mips64el" machines is resulting in timeouts > somehow - no idea there. I can however ssh to mips(32)el machines. > > That being said, I built it on a ppc64el porter box, and the build works > fine there. Attaching the log for reference. It probably didn't cross build > for you, as per earlier mail. > I suggest that you upload this then, and we simply file removal bugs in > case it still doesn't work on mips64 (although I smell another false > positive here) > > Let me know what you think about this. It looks like I'm getting bitten by the virtualisation layers, wouldn't be the first time nor the last. I finish the check that I triggered to remove risks of baseline violations, run a routine-update to wrap up the package and will upload; it looks like it might be worth it. Many thanks for your time and your additional checks, :) -- Étienne Mollier Fingerprint: 8f91 b227 c7d6 f2b1 948c 8236 793c f67e 8f0d 11da Sent from /dev/pts/2, please excuse my verbosity. signature.asc Description: PGP signature
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi Michael, Michael Crusoe, on 2021-01-30 16:51:27 +0100: > On Sat, 30 Jan 2021 at 14:33, Étienne Mollier > wrote: > The package brian[1] has a mechanism based on the GSL which does > > some sort of compilation just-in-time. The default set of build > > flags works rather well on all flavors of amd64 architecture > > with or without extensions, however not all of the specified > > options by default are supported on all architectures. > > > > Hmm.. this sounds like a recipe for a tough time :-D > > Personally I put no extra effort for mips :-) I thought so. Surprizingly, the mipsel 32 bits variant turned out to work normally, contrary to the 64 bits ISA (at least in Qemu context). I opened Bug#981397[2] to proceed to the removal to buy us some time. [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981397 > > [1] https://salsa.debian.org/med-team/brian/-/tree/master > > https://salsa.debian.org/med-team/brian/-/blob/master/debian/patches/gsl-compiler-arg.patch#L29 > > -march=native is not allowed as it violates the architecture baseline, > unless there is an alternative binary or library built without it. I would generally agree, but in this case I understood the code was compiled (in the background) on the end user's machine[3]. I don't exclude that I could have misunderstood the mechanism, so am triple checking at the moment. I probably should have put the link in the patch header, by the way. [3] https://brian2.readthedocs.io/en/stable/user/computation.html In any way, I guess putting more regular compiler flags at first should give a far safer outcome than current architecture dependent selection. Thanks for your review! :) -- Étienne Mollier Fingerprint: 8f91 b227 c7d6 f2b1 948c 8236 793c f67e 8f0d 11da Sent from /dev/pts/2, please excuse my verbosity. signature.asc Description: PGP signature
Re: [help] porting brian on MIPS and POWER, or dropping support
On Sat, 30 Jan 2021 at 14:33, Étienne Mollier wrote: > Hi Michael, and people at ease with architecture ports, > Hello! The package brian[1] has a mechanism based on the GSL which does > some sort of compilation just-in-time. The default set of build > flags works rather well on all flavors of amd64 architecture > with or without extensions, however not all of the specified > options by default are supported on all architectures. > Hmm.. this sounds like a recipe for a tough time :-D Personally I put no extra effort for mips :-) > [1] https://salsa.debian.org/med-team/brian/-/tree/master https://salsa.debian.org/med-team/brian/-/blob/master/debian/patches/gsl-compiler-arg.patch#L29 -march=native is not allowed as it violates the architecture baseline, unless there is an alternative binary or library built without it. -- Michael R. Crusoe
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi Nilesh, Nilesh Patra, on 2021-01-30 20:44:38 +0530: > On Sat, 30 Jan, 2021, 8:14 pm Étienne Mollier, > wrote: > > Étienne Mollier, on 2021-01-30 14:33:41 +0100: > > > * I curiously couldn't locate mipsel builds in my backups, but > > > from what I recall I got results very similar to mips64el. > > > > The build goes through on mipsel (but not mips64el)! I did not > > recall correctly it seems; it might be a clue about what could > > be wrong. > > > > That sounds unusual. It's usually the other way round - can it be a false > positive? > Should I try this on a mips64 porter box? Thank you for your offer of testing it on real hardware. :) I agree this is very unusual, but that is what I am witnessing with qemu-user-static at the time indeed. If you are curious and have the energy to investigate this, I suppose it would be at least worth an attempt on real hardware, sure! Have a nice day, :) -- Étienne Mollier Fingerprint: 8f91 b227 c7d6 f2b1 948c 8236 793c f67e 8f0d 11da Sent from /dev/pts/2, please excuse my verbosity. signature.asc Description: PGP signature
Re: [help] porting brian on MIPS and POWER, or dropping support
On Sat, 30 Jan, 2021, 8:14 pm Étienne Mollier, wrote: > Hi again, > > Nevermind that: > > Étienne Mollier, on 2021-01-30 14:33:41 +0100: > > * I curiously couldn't locate mipsel builds in my backups, but > > from what I recall I got results very similar to mips64el. > > The build goes through on mipsel (but not mips64el)! I did not > recall correctly it seems; it might be a clue about what could > be wrong. > That sounds unusual. It's usually the other way round - can it be a false positive? Should I try this on a mips64 porter box? Nilesh >
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi Andreas, Andreas Tille, on 2021-01-30 15:31:57 +0100: > IMHO we should focus on getting the package into testing for those > architectures that are frequently used (which means temporary removal of > existing older package versions on architectures where the current > version might fail to build). > > Once this is done and we have time we can try to care for the rest. True. I'm aware of a couple of packages which are currently out of Testing due to missing architecture build, and I believe I delayed request for removal for long enough. Will start with brian to buy us some time. Thanks for insisting on that aspect, -- Étienne Mollier Fingerprint: 8f91 b227 c7d6 f2b1 948c 8236 793c f67e 8f0d 11da Sent from /dev/pts/2, please excuse my verbosity. signature.asc Description: PGP signature
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi again, Nevermind that: Étienne Mollier, on 2021-01-30 14:33:41 +0100: > * I curiously couldn't locate mipsel builds in my backups, but > from what I recall I got results very similar to mips64el. The build goes through on mipsel (but not mips64el)! I did not recall correctly it seems; it might be a clue about what could be wrong. Cheers, :) -- Étienne Mollier Fingerprint: 8f91 b227 c7d6 f2b1 948c 8236 793c f67e 8f0d 11da Sent from /dev/pts/2, please excuse my verbosity. signature.asc Description: PGP signature
Re: [help] porting brian on MIPS and POWER, or dropping support
Hi Étienne, On Sat, Jan 30, 2021 at 02:33:39PM +0100, Étienne Mollier wrote: > ... Thanks a lot for your detailed analysis. > I feel it might be possible to get the software working as > intended on at least ppc64el, but time is running out with the > freeze, and brian is still out of Testing to day. I'm pinging > in case you might have an idea, but if not, I consider filing a > request for removal on these architectures by tomorrow. IMHO we should focus on getting the package into testing for those architectures that are frequently used (which means temporary removal of existing older package versions on architectures where the current version might fail to build). Once this is done and we have time we can try to care for the rest. As always thanks a lot for your work on this Andreas. -- http://fam-tille.de
[help] porting brian on MIPS and POWER, or dropping support
Hi Michael, and people at ease with architecture ports,
The package brian[1] has a mechanism based on the GSL which does
some sort of compilation just-in-time. The default set of build
flags works rather well on all flavors of amd64 architecture
with or without extensions, however not all of the specified
options by default are supported on all architectures.
[1] https://salsa.debian.org/med-team/brian/-/tree/master
The buildd logs[2] give a rather precise picture of the
situation with the existing package fortunately. Architectures
affected by unrecognized build options are showing the following
symptom:
Skipped: GSL support for numpy has not been implemented yet
[2] https://buildd.debian.org/status/package.php?p=brian
I introduced a patch to try addressing the issue on Debian's
official architectures and on riscv64; build issues different
symptoms on mips, with missing symbols, but reducing
optimization levels seemed to help moving forward. With these
changes, I got the following results so far:
* still FTBFS on ppc64el, I see the following build time test
step; the issue looks like it might just be a matter of
doing a small adjustment, but I don't know which one:
==
ERROR: brian2.tests.test_GSL.test_GSL_stateupdater_basic
--
Traceback (most recent call last):
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/core/network.py",
line 897, in before_run
obj.before_run(run_namespace)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/input/spikegeneratorgroup.py",
line 200, in before_run
raise ValueError('Using a dt of %s, some neurons of '
ValueError: Using a dt of ,
some neurons of SpikeGeneratorGroup "spikegeneratorgroup" spike more than once
during a time step.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/nose/case.py", line 197, in
runTest
self.test(*self.arg)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/tests/test_GSL.py",
line 19, in wrapped
func()
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/tests/test_GSL.py",
line 57, in test_GSL_stateupdater_basic
net.run(100*ms)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/core/base.py",
line 278, in device_override_decorated_function
return func(*args, **kwds)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/units/fundamentalunits.py",
line 2434, in new_f
result = f(*args, **kwds)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/core/network.py",
line 1008, in run
self.before_run(namespace)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/core/base.py",
line 278, in device_override_decorated_function
return func(*args, **kwds)
File
"/tmp/autopkgtest.t0d9nq/build.nMH/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/core/network.py",
line 899, in before_run
raise BrianObjectException("An error occurred when preparing an
object.", obj) from ex
brian2.core.base.BrianObjectException: Error encountered with object
named "spikegeneratorgroup".
Object was created here (most recent call only, full details in debug
log):
File "/usr/lib/python3/dist-packages/nose/case.py", line 197, in
runTest
self.test(*self.arg)
* the test_GSL_stateupdater_basic works on on mips64el, but
rereading the build, there are so many warnings about run
time errors such as below, that I would be wary introducing
the package on that architecture, so it'd be probably better
to not lose much more time here:
[...]
WARNING
/tmp/autopkgtest.Z9EqJI/build.QFw/src/debian/tmp/usr/lib/python3.9/dist-packages/brian2/units/fundamentalunits.py:2186:
RuntimeWarning: invalid value encountered in square
deviations = np.nansum((np.log10(floatreps) - 1)**2, axis=0)
[py.warnings]
WARNING
/usr/lib/python3/dist-packages/numpy/lib/nanfunctions.py:102: RuntimeWarning:
invalid value encountered in isnan
mask = np.isnan(a)
[py.warnings]
[...]
* I curiously couldn't locate mipsel builds in my backups, but
from what I recall I got results very similar to mips64el.
* the riscv64 build goes through! (yay! :)
I feel it might be possible to get
