Re: opencv WITH_IPP=ON

2024-02-20 Thread Mo Zhou
Intel IPP is not available in archive and it is not free. This request won't be fixed. On 2/20/24 05:14, Stefan Felkel wrote: Hi, According to the debian rules for the opencv package IPP is currently not enabled. For many algorithms, e.g. cv::matchTemplate() IPP increases the performance

Re: Removing ATLAS?

2023-11-05 Thread Mo Zhou
The following line is what I use for the PyTorch package (ignore the fact that I forgot to bump the BLIS abi from 3 to 4): Recommends: libopenblas0 | libblis3 | libatlas3-base | libmkl-rt | libblas3 So the latest Recommends line for performance critical packages relying on BLAS/LAPACK should

Re: jblas update to 1.2.5?

2021-12-16 Thread Mo Zhou
Hi tony, On 2021-12-11 16:14, tony mancill wrote: > My first question is, do we have users? Is the software useful to > maintain in Debian? The popcon for the package is low (around 15) and > there haven't been recent bug reports. If there are users, I will > gladly prepare an update. No

opencv: help wanted due to overload

2020-11-04 Thread Mo Zhou
Hi Science Team, I'd like to ask for some help since I find myself quite busy and overloaded these days, for the upcoming CVPR submission deadline. And recently my energy for debian has been fully occupied by the PyTorch and its build dependencies. As a result opencv has been left behind

ownership transfer of some pckages from science team to deep learning team

2020-10-18 Thread Mo Zhou
Hi science team, I want to transfer the salsa repository ownership of the following packages from science team to the debian learning team [1]: 1. caffe (educational deep learning framework) 2. onednn (intel's CPU/SYCL-based deep learning acceleration library) Maybe src:tensorflow should be

New Mail list debian-ai available

2020-09-13 Thread Mo Zhou
Hi folks, Just to inform you that I've applied for a new mailing list named debian-ai, focusing on AI and some related acceleration libraries: https://lists.debian.org/debian-ai/ Which is expected to be the main discussion channel for the deep learning team

Re: [COVID-19 Hackathon] Bazel build system pending NEW

2020-08-31 Thread Mo Zhou
Hi Michael, I went through the list of missing dependencies. aws-*: I'm not interested in them. In the past I just (manually) filtered out all the code associated with aws functionalities. Most of them are possibly not packaged yet. abseil-cpp: this is a tricky one. Similar to

Re: joining the science team to package spaCy & gensim

2020-08-28 Thread Mo Zhou
Hi Paul, On Fri, Aug 28, 2020 at 10:35:41AM +0800, Paul Wise wrote: > > I think you can simply work on the existing repositories. New repos > > can be created under the deep learning team if you like. > > I will do that, are there any guidelines for which team to use for > specific packages or

Re: joining the science team to package spaCy & gensim

2020-08-27 Thread Mo Zhou
Hi Paul, In the past Andreas Tille and I tried to get spaCy into the archive since it is really useful (in my research projects), and is sometimes more convenient than NLTK (I'm the uploader). Both NLTK and spaCy suffer from a problem -- they cannot be fully functional without pretrained models.

Re: [COVID-19 Hackathon] Bazel build system pending NEW

2020-08-18 Thread Mo Zhou
Hi Olek, What a good news! Your hardword is much appreciated. There are some preliminary packaging changes (using bazel) for tensorflow. Is it in shape so that I can build TF from the git repo? Or what remains to be done? On Tue, Aug 18, 2020 at 05:03:23PM -0400, Olek Wojnar wrote: > Dear FTP

Re: RFC: Changing the debian system default BLAS/LAPACK implementation

2020-06-16 Thread Mo Zhou
these packages to Depends: or Suggests: when appropriate) The package order is the same as that in the update-alternative mechanism. On Tue, Jun 16, 2020 at 11:58:36AM +0200, Andreas Tille wrote: > On Tue, Jun 09, 2020 at 11:11:33AM +0000, Mo Zhou wrote: > > https://salsa.debian.org/sci

[GSoC] fortnightly: blas/lapack enhancements

2020-06-14 Thread Mo Zhou
Hi team, My gsoc project involves several tracks of works. And the following is a summary about the current progress: (The list is tracked at https://salsa.debian.org/lumin/gsoc2020-debian-blaslapack/-/blob/master/README.md and it will grow over time) Threading-aware virtual blas (status:

RFC: Changing the debian system default BLAS/LAPACK implementation

2020-06-09 Thread Mo Zhou
Hi Science team, It's needless to introduce the importance of BLAS/LAPACK to you. Let me directly put forward the question: Are we satisfied with the performance of default BLAS/LAPACK provider? (which is exactly netlib, without optimization) Should we change the system default? I can

Re: OpenBlas Pthread issue with R

2020-05-28 Thread Mo Zhou
Hi Dirk, After browsing some of the related discussions, I think we can try rebuilding openblas with one of of these options (MAKE_OPTIONS) USE_SIMPLE_THREADED_LEVEL3=1 USE_TLS=0 On Wed, May 27, 2020 at 11:22:32PM -0500, Dirk Eddelbuettel wrote: > There is hopefully a programmatic fix

Re: OpenBlas Pthread issue with R

2020-05-27 Thread Mo Zhou
> 1) We provide extra shared objects, libopenblasp, libopenblaso, etc >and let R link against libopenblaso (o=openmp) > >Drawback: too ugly and makes the blas ecosystem overly complex. >Also breaks the alternative mechanism > > 2) Add RPATH=/usr/lib/<..>/blas-pthread/ to all the

Re: OpenBlas Pthread issue with R

2020-05-27 Thread Mo Zhou
uettel wrote: > > Hi Seb, > > On 1 May 2020 at 14:18, Sébastien Villemot wrote: > | Le vendredi 01 mai 2020 à 07:05 -0500, Dirk Eddelbuettel a écrit : > | > On 1 May 2020 at 05:16, Mo Zhou wrote: > | > > On Thu, Apr 30, 2020 at 11:26:23PM -0500, Dirk Edd

Re: RFC: threading-aware virtual BLAS/LAPACK

2020-05-20 Thread Mo Zhou
(forwarding my mail from -devel to -science to RFC) Hi fellow devs, I've suddenly got some inspiration on this problem, which resulted in a much better solution for the problem the original proposal confronts. I like this overhauled solution. No extra shared libs, no extra SONAMEs. No extra

Re: Bug#961108: openmpi: providing 64-bit MPI

2020-05-20 Thread Mo Zhou
Hi Alastair, On Wed, May 20, 2020 at 01:39:43PM +0100, Alastair McKinstry wrote: > I'd be in favour of 64-bit computational stacks; some of the packages > (pnetcdf, etc) already don't work with 32-bit; It should make is > _possible_ to run on 32-bit, > > but accept that performance on 64-bit

RFR: new lintian check: linkage-against-deprecated-libcblas

2020-05-20 Thread Mo Zhou
Hi Science team, I'd like to request for review on a new lintian warning check. https://salsa.debian.org/lintian/lintian/-/merge_requests/309 In the past we've confirmed that programs should not be linked against libcblas.so when they can use libblas.so . The detailed explanations can be found

RFR: build libjulia-openblas64 from src:openblas

2020-05-09 Thread Mo Zhou
Hi Sébastien, Julia needs a openblas library with some special configs (symbol mangling) I'm getting rid of the embedded code copies from src:julia, so just submitted my changes as a pull request: https://salsa.debian.org/science-team/openblas/-/merge_requests/2 Next I'll have to build a

Re: license problem about nvidia's neural network library (cuDNN)

2020-05-07 Thread Mo Zhou
Hi Rebecca, I have completely forgot the threads I opened in the past. However, the cuDNN eula had been revised many times since the last discussion. The latest revision was made on Nov 2019. That means a re-evaluation is needed. On Thu, May 07, 2020 at 05:19:03PM +0100, Rebecca N. Palmer

license problem about nvidia's neural network library (cuDNN)

2020-05-07 Thread Mo Zhou
Hello guys, I mentioned cudnn in the other mail about my "plans for deep learning frameworks". In this separate mail I'm talking about the licensing issue about cudnn, the most important CUDA library for neural networks. https://docs.nvidia.com/deeplearning/sdk/cudnn-sla/index.html Do you have

my plans about deep learning framework

2020-05-07 Thread Mo Zhou
Hi Science and Med Team, I maintain lots of packages related to deep learning and machine learning, but I think I'm reaching an upperbound. So I'm clearly stating my future plans about deep learning packages for Debian, lest potential contributors hesitate to step in and help. Med team has been

RFC: threadding-aware virtual BLAS/LAPACK?

2020-05-06 Thread Mo Zhou
Hi, Through some previous discussions I realized that the issue of threadding implementation of the BLAS can be sometimes cumbersome. For example, sometimes we can observe severe performance regression from pthread program + BLAS with gomp, or even observe severe calculation error from gomp

Re: Welcoming Mo Zhou as GSoC student

2020-05-06 Thread Mo Zhou
Needless to introduce myself here. I'll start working shortly after finishing my reviewer jobs for a deep learning and computer vision conference :-) On Tue, May 05, 2020 at 07:07:30AM +0200, Andreas Tille wrote: > Hi, > > I've got confirmation that Mo Zhou is accepted as GSo

Re: OpenBlas Pthread issue with R

2020-04-30 Thread Mo Zhou
On Thu, Apr 30, 2020 at 11:26:23PM -0500, Dirk Eddelbuettel wrote: > Switching to libopenblas0-openmp works but one needs to uninstall > libopenblas0-pthread (or else fiddle with the alternatives priority). Does that mean the update-alternatives mechanism is malfunctioning? > Dirk > > -- >

Re: [covid-19] Reviving tensorflow packaging effort (Was: Missing dependancies for streamlit)

2020-04-26 Thread Mo Zhou
On Sat, Apr 25, 2020 at 01:42:28PM +0200, Andreas Tille wrote: > > Is there any COVID-19 package using pytorch blocked due to its absense? > > I admit I can not say without doing detailed research on the set of > relevant packages[3] I quickly went through all these projects and none of them

Re: [covid-19] Reviving tensorflow packaging effort (Was: Missing dependancies for streamlit)

2020-04-25 Thread Mo Zhou
deep learning framework in the archive. It's better than having nothing even if I'm working on the cpu-only (free) version. On Wed, Apr 08, 2020 at 10:06:12AM +0200, Andreas Tille wrote: > Hi Mo, > > On Wed, Apr 08, 2020 at 02:46:06AM +, Mo Zhou wrote: > > On Tue, Apr 07

Re: Missing dependancies for streamlit

2020-04-07 Thread Mo Zhou
On Tue, Apr 07, 2020 at 11:49:07AM +0200, Andreas Tille wrote: > On Tue, Apr 07, 2020 at 08:56:45AM +0100, Rebecca N. Palmer wrote: > > tensorflow 1.10 was packaged in experimental, but with reduced performance, > > and was removed because this was considered not worth it: > >

[GSoC] BLAS/LAPACK Ecosystem Enhancement for Debian

2020-03-26 Thread Mo Zhou
Hi science team, As discussed before, thanks to google, I'm participating GSoC this year and I'll work to improve our BLAS/LAPACK ecosystem. Details can be found in my GSoC proposal document: https://people.debian.org/~lumin/debian-gsoc.pdf The confirmed mentors of this project are Andreas

Re: Possibly help from Debian Science team for streamlit needed (Was: COVID-19 Biohackathon April 5-11 2020)

2020-03-20 Thread Mo Zhou
Hi, I'm recently busy with my thesis, but I can provide review and feedbacks should anyone works on it. On Fri, Mar 20, 2020 at 09:40:40AM +0100, Andreas Tille wrote: > Hi folks, > > On Tue, Mar 17, 2020 at 06:42:26AM +0100, Andreas Tille wrote: > > > You know, I am after bcbio for two years

Re: cblas / gsl hint needed (Was: Bug#936609: Ported ghmm to Python3 but issues with clapack)

2020-01-05 Thread Mo Zhou
GSL provides a set of CBLAS API/ABI, delivered with shared object "libgslcblas.so" and header "gsl_cblas.h". That subset becomes redundant once you include the headers of any standard/compatible (C)BLAS library. That's what the compilation error means. Make sure that the code only use one CBLAS

Re: Moving libsvm to Debian Science team

2019-12-21 Thread Mo Zhou
I second this proposal, and the same for src:liblinear. These are high popcon packages, dependencies for a number of other packages. They should be team maintained to unblock important fixes. On Sat, Dec 21, 2019 at 08:35:28AM +0100, Andreas Tille wrote: > Hi Chen-Tse, > > I'm maintaining a

[need mentor] GSoC Proposal: Enhancement to Debian's BLAS/LAPACK Ecosys

2019-12-11 Thread Mo Zhou
Hi science team, I'm proposing the following project for GSoC 2020: "Enhancement to Debian's BLAS/LAPACK Ecosys" To some extent this is basically making some portion of my todo-list well-funded by Google :-) I cannot "mentor" myself as I'm the participating student... So I think I need a GSoC

Re: priority score of libflame (LAPACK alternative)?

2019-12-05 Thread Mo Zhou
unit tests failed with libflame::lapack.so backend. On Mon, Dec 02, 2019 at 01:13:04PM +0000, Mo Zhou wrote: > Hi science team, > > As usual, I'd like to inform the team before registering a new lapack > implementation into our blas/lapack ecosystem. The new implementation > i

priority score of libflame (LAPACK alternative)?

2019-12-02 Thread Mo Zhou
Hi science team, As usual, I'd like to inform the team before registering a new lapack implementation into our blas/lapack ecosystem. The new implementation is called "libflame", from the upstream of BLIS: https://github.com/flame/libflame Similar to BLIS, it is a lapack-like object-based

Re: Is Alink something interesting for us?

2019-11-29 Thread Mo Zhou
Hi Andreas, quote: "Alink is ... based on Flink" (Flink = Apache Flink) Alibaba builds everything with Java. Java is the core language of this company. I don't think it's easy to maintain anything on top of Apache's Java XXX (e.g. Flink) for Debian, especially given that Debian don't have too

science application + opencl/rocm + AMD GPU?

2019-11-23 Thread Mo Zhou
Hi science team, I'd like to request the team for sharing some experience on this topic: "scientific application + opencl/rocm + AMD GPU". Any experience will be very helpful to me in terms of the being-investigated ROCm integration[1] to Debian. I'd like to ask you the following questions: 1.

uploading lapack 3.8.0-8 and openblas to unstable?

2019-10-30 Thread Mo Zhou
Hi Sébastien, Shall we upload lapack 3.8.0-8 + [ Sébastien Villemot ] + * Migrate to Python 3 ++ python3.patch: new patch ++ d/control, d/tests/control: replace python by python3 (Closes: #943142) + * Bump to debhelper compat level 12. +On mipsel, disable dh_dwz. + + [ Mo Zhou

Re: MBF: don't build against libatlas3-base if possible

2019-10-30 Thread Mo Zhou
Hi Ian, On Wed, Oct 30, 2019 at 01:17:20PM +, Ian Jackson wrote: > Hi. I would like to congratulate you on your work so far. I am no > expert on this area but from what I read, I want to encourage you :-). :-) > I think it's important to give the MBF recipients a clear set of >

Summary: BLAS/LAPACK Ecosys Massive Update

2019-10-28 Thread Mo Zhou
Hi fellow developers, A good news especially for Debian scientific computing users. I shall call it a massive update, even if the whole update was decomposed into many tiny steps where some of them had already been finished 1 year ago. BLAS/LAPACK are two typical and classical dense linear

theano: remove python2

2019-10-28 Thread Mo Zhou
Hi Rebecca, Theano is a leaf package blocking python2 removal. What's the blocker for the pending commits on the master branch? It can be successfully built in a clean chroot. May I team upload it on behalf of py2 removal?

Re: pandas: Python 2 removal, 0.23 -> 0.25 transition

2019-10-27 Thread Mo Zhou
Hi Rebecca, Personally I fully support the option (a). Afterall nobody likes taking extra burden especially when there is no upstream support anymore. (b) is for good wish but impractical. (c) means we just let pandas package rot and die. Even if some portion of users still need the python2

MBF: don't build against libatlas3-base if possible

2019-10-26 Thread Mo Zhou
Type: performance improvement, integration Affected: `apt rdepends libatlas3-base` Severity: important BLAS is a basic dense linear algebra library, so important like an mathematical "libc". The standard fortran implementation is included in src:lapack, which is often *hundreds* times slower than

Re: RFR: src:openblas Multi-Flavour updates

2019-10-25 Thread Mo Zhou
min, > > Le samedi 21 septembre 2019 à 22:59 -0700, Mo Zhou a écrit : > >> I've updated the git branch for this feature: >> >> https://salsa.debian.org/science-team/openblas/commits/lumin >> >> >> It can be built against the git master version of s

Re: Bug#941569: RFS: sentencepiece/0.1.83+dfsg-1 [ITP] -- Unsupervised text tokenizer for Neural Network-based text generation

2019-10-03 Thread Mo Zhou
FYI, Science team policy described the standard workflow inside team: https://science-team.pages.debian.net/policy/ When you can create new repos, you should already have the "Maintainer" permission on salsa, that means nothing would be blocked. Please follow the science team policy and see if

Re: RFR: src:lapack's 64bit-indexing variant

2019-09-30 Thread Mo Zhou
Hi Sébastien, On 2019-09-30 11:59, Sébastien Villemot wrote: > > I have uploaded 3.8.0-5 to experimental, that fixes this issue and > several other ones. The package now builds everywhere, and passes both > piuparts and autopkgtests. saw that. > Lumin: Do you have other pending changes? Or

RFR: src:openblas Multi-Flavour updates

2019-09-22 Thread Mo Zhou
Hi Sébastien, I've updated the git branch for this feature: https://salsa.debian.org/science-team/openblas/commits/lumin It can be built against the git master version of src:lapack (i.e. 3.8.0-4 + 1 commit that fixes installation path). Problems pointed out in the previous round of review have

Re: RFR: src:lapack's 64bit-indexing variant

2019-09-20 Thread Mo Zhou
reason. Should we disable the BLAS64/LAPACK64 builds for the 4 architectures and head towards openblas64? mips64el, s390x, ppc64, sparc64, they are not typical architectures used for intensive scientific computing. On 2019-09-04 02:21, Mo Zhou wrote: > Finally the 3.8.0-3 build result on

Re: RFR: src:lapack's 64bit-indexing variant

2019-09-03 Thread Mo Zhou
a few seconds. On 2019-08-20 17:40, Sébastien Villemot wrote: > Le mardi 20 août 2019 à 19:29 +0200, Gilles Filippini a écrit : >> Mo Zhou a écrit le 20/08/2019 à 18:21 : >> > On 2019-08-20 16:15, Sébastien Villemot wrote: >> > > I realize that I don’t know what we

Re: RFR: src:lapack's 64bit-indexing variant

2019-08-20 Thread Mo Zhou
On 2019-08-20 16:15, Sébastien Villemot wrote: > Le mardi 20 août 2019 à 18:11 +0200, Sébastien Villemot a écrit : >> Le mardi 20 août 2019 à 09:00 -0700, Mo Zhou a écrit : >> > Got it. But keep in mind that we haven't adapted the rules/control >> > for 32-bit architec

Re: RFR: src:lapack's 64bit-indexing variant

2019-08-20 Thread Mo Zhou
Hi Sébastien, On 2019-08-20 15:26, Sébastien Villemot wrote: > I pushed a few further commits from myself to address some remaining > issues. The main one concerns the override to dh_makeshlibs that you > had removed, but which is really needed so that packages have the > correct DEBIAN/shlibs

Re: h5py and hdf5-mpi

2019-08-13 Thread Mo Zhou
On 2019-08-14 03:59, Drew Parsons wrote: > Shall we go ahead with Alastair's "minimal" change now, or should we > discuss further? If the MPI build can correctly work for the serial use case, I vote for the "minimal" change. I roughly went through the h5py documentation and found nothing special

Re: h5py and hdf5-mpi

2019-08-12 Thread Mo Zhou
Hi Drew, thanks for the commits to h5py. On 2019-08-12 03:10, Drew Parsons wrote: > We need to change h5py to support hdf5-mpi. h5py is somewhat crippled > as serial-only. I didn't even notice that since my use case for hdf5 is light-weight. (training data is fully loaded from hdf5 into

Cross-distro Call for BLAS64/LAPACK64 Convention Alignment

2019-08-02 Thread Mo Zhou
Dear BLAS/LAPACK maintainers[0], (Please invite more distribution maintainers to join the discussion if you know the right contact, thanks. Please keep the debian-science public mailing list in CC to keep the discussion public.) Mordern BLAS/LAPACK implementations, such as OpenBLAS, BLIS, etc

RFR: src:lapack's 64bit-indexing variant

2019-07-27 Thread Mo Zhou
Hi science team, My 64bit-indexing updates for src:lapack are ready for review. Please have a look at my git commits on the "lumin" branch: g...@salsa.debian.org:science-team/lapack.git -b lumin Several points to note: 1. I merged the content of liblapack-pic into liblapack-dev. 2.

Re: dwz: .debug_line reference above end of section [now in mpi4py]

2019-07-18 Thread Mo Zhou
Hi Drew, FYI I got no update at this point... still noop'ing dh_dwz once it went wrong. On 2019-07-18 15:17, Drew Parsons wrote: > On 15 Jan 2019 Andreas Tille wrote: >> On Tue, Jan 15, 2019 at 02:19:40PM +, Mo Zhou wrote: >> > Temporarily overriding dh_dwz to noop it is an

Re: scientific python stack transitions

2019-07-07 Thread Mo Zhou
Hi science team, By the way, when do we start dropping python2 support? The upstreams of the whole python scientific computing stack had already started dropping it. On 2019-07-07 14:40, Drew Parsons wrote: > On 2019-07-07 21:11, Rebecca N. Palmer wrote: >> matplotlib and scikit-learn have also

Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

2019-06-13 Thread Mo Zhou
Hello guys, On 2019-06-10 13:14, Sam Hartman wrote: > I really like the term toxic candy. > In two words it explains both that the model is appealing and > problematic. So let's keep this name :-) > If there are subdivisions of toxic candy that we decide are free, we > should come back and

Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

2019-06-13 Thread Mo Zhou
Hi Charles, On 2019-06-13 13:11, Charles Plessy wrote: >> 1. Free datasets used to train FreeModel are not required to upload >>to our main section, for example those Osamu mentioned and wikipedia >>dump. We are not scientific data archiving organization and these >>data will blow up

Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

2019-06-09 Thread Mo Zhou
Hi Osamu, On 2019-06-09 08:28, Osamu Aoki wrote: > Although I understand the intent of "SemiFree" or "Tainted" (by Yao), I > don't think these are a good choice. We need to draw a line between > FREE(=main) and NON-FREE(non-free) as a organization. I think there are There is no such a line as

Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

2019-06-08 Thread Mo Zhou
Hi Osamu, On 2019-06-08 18:43, Osamu Aoki wrote: >> This draft is conservative and overkilling, and currently >> only focus on software freedom. That's exactly where we >> start, right? > > OK but it can't be where we end-up-with. That's why I said the two words "conservative" and

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-29 Thread Mo Zhou
Hi, On 2019-05-21 23:52, Paul Wise wrote: > Has anyone repeated the training of Mozilla DeepSpeech for example? By chance I found a paper from a pile of papers (that attacks AI models) that Berkeley researchers have successfully attacked DeepSpeech: https://arxiv.org/pdf/1801.01944.pdf IHMO

Re: is it safe to "try" gcc 9 from experimental in Debian/testing

2019-05-25 Thread Mo Zhou
Hi Christophe, I personally recommend you to try it within a docker container, or a chroot environment. On 2019-05-24 14:23, Christophe Trophime wrote: > Hi, > I would like to try gcc-9 on a Debian/testing. > It seems that install gcc-9 from experimental will update libatomic1 > and libgcc1 > I

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
Hi Paul, On 2019-05-21 23:52, Paul Wise wrote: > Are there any other case studies we could add? Anybody is welcome to open an issue and add more cases to the document. I can dig into them in the future. > Has anyone repeated the training of Mozilla DeepSpeech for example? Generally speaking,

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
Hi Ben, Good catch! I'm quite sure that the 3 categories are not overlapping with each other. And I've fixed the language to make it logically correct: A **ToxicCandy Model** refers to an explicitly free software licensed model, trained from unknown or non-free dataset. A model is

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
problem: In the definition of "Free Model", I mentioned that the model *should be reproducible* with a fixed random seed. This is also a good practice for ML/DL engineers and researchers. On 2019-05-21 12:10, julien.pu...@laposte.net wrote: > Hi > > Le 21 mai 2019 13:4

Re: Bug#929296: libopenblas-base: is libopenblas.so needed?

2019-05-21 Thread Mo Zhou
On 2019-05-21 09:13, Drew Parsons wrote: > > This seems to be the problem. libopenblas.so.0 is used to resolve > symbols instead of liblapack.so.3. The symbol in question in > Bug#914655 is ilaver_ which is part of lapack, not specific to > libopenblas. ilaver_ is indeed a standard fortran

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
Hi Andreas, On 2019-05-21 09:07, Andreas Tille wrote: > Not sure whether this is sensible to be added to the issue > tracker. I always abuse issue track in my personal repository. > Quoting from your section "Questions Not Easy to Answer" > > > 1. Must the dataset for training a Free Model

Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
Hi Paul, They are added to the case study section. And I like that question from ffmpeg-devel: Where is the source for all those numbers? On 2019-05-21 08:02, Paul Wise wrote: > On Tue, May 21, 2019 at 3:11 PM Mo Zhou wrote: > >> I'd better write a draft and shed some light

Bits from /me: A humble draft policy on "deep learning v.s. freedom"

2019-05-21 Thread Mo Zhou
Hi people, A year ago I raised a topic on -devel, pointing out the "deep learning v.s. software freedom" issue. We drew no conclusion at that time, and linux distros who care about software freedom may still have doubt on some fundamental problems, e.g. "is this piece of deep learning software

Re: Bug#929296: libopenblas-base: is libopenblas.so needed?

2019-05-21 Thread Mo Zhou
Hi Drew, I didn't closely investigate into the scipy bug, but I can answer some of your questions. BTW, does anything break in a clean chroot? I mean, making sure a thing works fine in an unclean environment is difficult. On 2019-05-21 04:57, Drew Parsons wrote: > Why is

Anyone tried to build 64-bit-indexing reference *c*blas?

2019-05-20 Thread Mo Zhou
Hi science team, I'm talking about the future updates to src:lapack, the reference implementation. BLAS (fortran), LAPACK (fortran), LAPACKE (C), all of them support building the 64-bit-indexing variant. For fortran we just need to add the "-fdefault-integer-8" flag (for some other fortran

Re: Towards lapack / lapack64 packaging

2019-05-16 Thread Mo Zhou
Hi Gard, On 2019-05-16 20:39, Gard Spreemann wrote: > Incidentally, and somewhat off-topic: Do you know if a similar effort is > being made for the PETSc and SLEPc packages (src petsc and slepc)? Do you mean 64-bit array indexing support for them? According to our dependency chain, they are

Re: Towards lapack / lapack64 packaging

2019-05-15 Thread Mo Zhou
Hi Sébastien, On 2019-05-15 16:23, Sébastien Villemot wrote: >> Sébastien provided some possible solutions: >> >> 1. build a 64-bit indexing variant of src:lapack >> 2. provide a liblapack64-pic (Sébastien prefer this) > > First, note that solution 1 is a superset of solution 2 (i.e. we

Towards lapack / lapack64 packaging

2019-05-15 Thread Mo Zhou
Hi science team, I'm trying to add multi-flavor support to the openblas package, as a part of the ongoing BLAS64 + LAPACK64 work. However, there is some problems need to be discussed. Two problems will be discussed in this email: (1) building problem about OpenBLAS's liblapack64.so (2)

Annoying stderr message when importing sklearn

2019-05-13 Thread Mo Zhou
Hi list, Can someone look into fixing the following bug? https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923937 The deprecation warning is very annoying and I've already seen that useless noise in my terminal for hundreds of times. This not only affects sklearn but also all it's reverse

Re: libtrilinos-teuchos-dev depends on libblas-dev

2019-05-13 Thread Mo Zhou
Hi, Linking programs against reference blas enables us to switch the underlying library smoothly: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries However, in general libopenblas-dev provides libblas.so, that should be enough to replace the reference blas. The possible reason is that

Please start reviewing the updated OpenBLAS packaging for six different flavors

2019-04-28 Thread Mo Zhou
Hi Sébastien and science team, Please start checking the updated openblas packaging here, I'm almost finalizing the changes. The experimental package can be built and installed: https://salsa.debian.org/science-team/openblas/tree/lumin I say "almost" because this is a hell of detail. I still

Re: Help with package a library with python bindings

2019-04-17 Thread Mo Zhou
Hi Christophe, On Wed, Apr 17, 2019 at 01:35:13PM +0200, Christophe Trophime wrote: > I'm moving their build system from scons to cmake to be able to build python > bindings for both python and python3. Please don't create more python2 packages, as per the python policy. > I've managed to get

Re: SIMDebian: Debian Partial Fork with Radical ISA Baseline

2019-04-09 Thread Mo Zhou
Hi Guillem, Thanks for your helpful pointers. On Sat, Apr 06, 2019 at 10:55:35PM +0200, Guillem Jover wrote: > If what you are interested in though is just a small subset of the > archive, another option that would benefit everyone and is perhaps > less cumbersome than having to jugle around

Re: Bug#926019: RM: clsparse -- ROM; dead upstream

2019-04-01 Thread Mo Zhou
On Mon, Apr 01, 2019 at 05:41:59PM +0200, Ghislain Vaillant wrote: > Agreed, though I was warned by the upstream developer that the CLMath suite > was > not much of a priority at AMD. > > CLBlas and CLFFT are in a similar state and might face a similar outcome. Sounds like that AMD has totally

Re: Bug#921207: Octave GEMM error on large matrix due to openmp thread race condition

2019-02-28 Thread Mo Zhou
Hi Sébastien, On Sun, Feb 03, 2019 at 12:07:20PM +, Mo Zhou wrote: > It turns out that the incorrect matrix product is a result of > gomp + iomp library clash: octave is linked against the GNU OMP, > while libmkl-rt.so invokes Intel(LLVM) OMP by default. I got in touch with

Re: Bug#868355: Any reason not to simply upload ceres-solver with adjusted version of libeigen3-dev

2019-02-26 Thread Mo Zhou
On Tue, Feb 26, 2019 at 11:25:49AM +0100, Andreas Tille wrote: > > The eigen3 maintainer and I are happy to simply rebuild affected > > packages after every eigen3 update, but Emilio considers it an upstream bug. > > Unfortunately I could not find anybody able to shed more light on the > > eigen3

Re: libopencv 3.2 for buster or some other version ?

2019-02-23 Thread Mo Zhou
Hi shirish, On Sat, Feb 23, 2019 at 08:43:50AM +, shirish शिरीष wrote: > I know we are close to the freeze but is there a possibility of any > other version of libopencv apart from 3.2 to come to buster ? Impossible. It's too late to get any newer version into Buster and every OpenCV

Re: IBM POWER9 SIMD support? (Was: SIMDebian: ...)

2019-02-16 Thread Mo Zhou
Hi Kingsley, On Sat, Feb 16, 2019 at 11:43:46AM -0800, Kingsley G. Morse Jr. wrote: > My understanding is IBMs "POWER9" CPUs > > 1.) have SIMD instructions[1] and > > 2.) are used by the new, and very cool, open > *hardware* Talos II workstations[2], which > > 3.) already

Started working on OpenBLAS flavors: six variants in total

2019-02-10 Thread Mo Zhou
Hi Sébastien, I started working on the 6 variants of OpenBLAS: (32-bit, 64-bit) x (pthread, openmp, serial) The priorities of pthread, openmp, serial variants are 100, 99, 98 respectively. The work-in-progress updates can be found in the "lumin" branch at

Re: SIMDebian: Debian Partial Fork with Radical ISA Baseline

2019-02-08 Thread Mo Zhou
On Sat, Feb 09, 2019 at 01:14:43PM +0800, Benda Xu wrote: > Hi Mo, > > Very interesting initiative. I understand it will Intel-specific for > the moment. What is your vision in opitmizing with AMD CPUs? Thanks for your interest. This project didn't mention AMD CPU because I have no experience

Re: SIMDebian: Debian Partial Fork with Radical ISA Baseline

2019-02-08 Thread Mo Zhou
Hi Drew, On Sat, Feb 09, 2019 at 01:07:47PM +1100, Drew Parsons wrote: > On 2019-02-09 03:25, Mo Zhou wrote: > > I think it would be more constructive to provide arch-specific packages for > eigen/blas etc on amd64 which Conflict/Replace/Provide the standard > packages. >

SIMDebian: Debian Partial Fork with Radical ISA Baseline

2019-02-08 Thread Mo Zhou
Hi folks, For most programs the "-march=native" option is not expected to bring any significant performance improvement. However for some scientific applications this proposition doesn't hold. When I was creating the tensorflow debian package, I observed a significant performance gap between

Re: Updates about BLAS64, co-installable variants

2019-02-04 Thread Mo Zhou
bug while writing this email. The alternative name should be libblis64.so.***2***-x86_64-linux-gnu) On Fri, Jan 04, 2019 at 10:09:15AM +0100, Sébastien Villemot wrote: > Le mardi 18 décembre 2018 à 15:12 +0000, Mo Zhou a écrit : > > On Tue, Dec 18, 2018 at 12:42:22PM +0100, Sébastien Vi

Re: freefem3d #897752 and clsparse #897722 RC fixes

2019-02-04 Thread Mo Zhou
On Mon, Feb 04, 2019 at 10:56:25PM +, Rebecca N. Palmer wrote: > I have posted (but can't upload) fixes for both. Thank you for your effort. Initially I was planning to take a look and team upload them. > I don't know if these packages are worth saving (both of them appear > to be dead

Re: Is multiple-layers of alternatives a good thing to users?

2019-02-04 Thread Mo Zhou
Hi Guus, On Mon, Feb 04, 2019 at 07:55:41AM +0100, Guus Sliepen wrote: > > libblis.so.2 libblis2 #MINVER# > > If the ABI and API are the same for all variants, a much better > solutions seems to me to have a single libblis2 that can switch at > runtime between the different variants, perhaps

Re: Is multiple-layers of alternatives a good thing to users?

2019-02-03 Thread Mo Zhou
Hi Ian and Thibaut, Inspired by Thibaut's comment, I worked out a good solution for the co-installation problem, which only results in a single layer of alternatives. Thibaut's proposed layout: > Package: libblis2-openmp, Provides: libblas.so.3, libblis.so.2 > Package: libblis2-pthread,

Re: Bug#921207: Octave GEMM error on large matrix due to openmp thread race condition

2019-02-03 Thread Mo Zhou
control: severity -1 important Hi Sébastien and Sylvestre, On Sun, Feb 03, 2019 at 10:16:05AM +0100, Sébastien Villemot wrote: > Control: tags -1 unreproducible > > Dear Lumin, > > I've tried to reproduce the problem with Netlib BLAS, OpenBLAS and > BLIS, but without success (I did not try

Bug#921207: Octave GEMM error on large matrix due to openmp thread race condition

2019-02-02 Thread Mo Zhou
Package: octave Version: 4.4.1-4 Severity: grave X-Debbugs-CC: debian-science@lists.debian.org, ido...@neto.net.il Dear octave maintainer, I received an astonishing bug report[1] saying that MKL returns wrong result for matrix multiplication. However, my further investigation suggests that the

Is multiple-layers of alternatives a good thing to users?

2019-01-31 Thread Mo Zhou
Hi devel, A user suggested[1] that the 6 variants[2] of BLIS should be co-installable. However, making them co-installable would result in multiple layers of alternatives in the update-alternatives system and will possibly confuse users, as discussed in [3]. I wrote this mail in case anyone has a

Best practice to copy(reuse) a shared object and assign it with another SONAME?

2019-01-18 Thread Mo Zhou
Hi mentors, I'm wondering if there is any shared object trick that allows me to create some "stub" shared object that reuses all symbols from another shared object, but has different SONAME compared to the original one. Background: * We have a shared object named libopenblas.so.0, with

Re: dwz: libsleef3.debug: .debug_line reference above end of section

2019-01-15 Thread Mo Zhou
Hi, Thanks for the pointers. But I think I'm not able to fix it even if I tried to diagnose the debug information. ATM I asked several people and it is likely that the dwz error stems from toolchain support problem. Now the build log on non-release archivectures are also available, and it seems

Re: Updates about BLAS64

2019-01-03 Thread Mo Zhou
Hi, On Thu, Jan 03, 2019 at 03:13:00PM +0100, Drew Parsons wrote: > I'd say it's more common for non-coinstallable packages to be installed in > subdirs (e.g. using DESTDIR facilities during configuration and build), and Theoretically yes, but in fact I don't want to make different variants

  1   2   >