Aw: Re: Do we need the Python interface of ghmm
Hi Andreas, it will be best to remove it, I suggest. I do not have any personal contact to upstream, somehwat unfortunate since this looks very nice and educational, but ... unless someone enjoys such a transition, I suggest to just wait for a version 0.10. Best, Steffen > Gesendet: Mittwoch, 07. Februar 2024 um 14:02 Uhr > Von: "Andreas Tille" > An: "Debian Med Project List" > Cc: "Steffen Möller" > Betreff: Re: Do we need the Python interface of ghmm > > Hi again, > > if there is no response of kind "Yes, please try to keep the Python > interface of ghmm" it will be removed soon. > > Kind regards >Andreas. > > Am Tue, Jan 30, 2024 at 07:58:34AM +0100 schrieb Andreas Tille: > > Hi Steffen (and two whom it might concern), > > > > when Israel has written the autopkgtest for ghmm it turned > > out that there are several issues with the Python3 interface. > > In Matrix Nilesh raised the perfectly valid question: > > > >The py interface is living off a patch from py2to3 and is (very) > >broken. It needs quite a lot of patching. Maybe it's a sensible thing to > >evaluate if we even need it? > > > > Could you please state your opinion about this? > > > > We should respond in a timely manner to the atlas removal > > effort (patch in Git) and currently it is blocked by the > > test suite issue of the Python3 interface. > > > > Kind regards > >Andreas. > > > > -- > > http://fam-tille.de > > > > > > -- > http://fam-tille.de >
Aw: Re: Last call for any other location than Berlin (Was: Any suggestions for next Debian Med sprint (in person meeting)?)
> Gesendet: Freitag, 22. Dezember 2023 um 22:37 Uhr > Von: "Pierre Gruet" > An: debian-med@lists.debian.org > Betreff: Re: Last call for any other location than Berlin (Was: Any > suggestions for next Debian Med sprint (in person meeting)?) > > Hi all, > > Le 21/12/2023 à 16:15, Sascha Steinbiss a écrit : > > Hi, > > > > [...] > >>> How about Feb 10-11, for example? My weekends are still free for now > >>> so it's > >>> your choice. > >> > >> I admit I'd prefer Feb. 17-18 over 10-11. Other opinions? > > I will be available on both weekends, although I prefer 10-11 over 17-18 > for teaching and meeting reasons. > > But as I just wrote, 17-18 also works. I'd opt for the earlier date, so we have a bit of a break before some of us I expect to bump into each other again in Hamburg if Holger's event materialises a month later. Best, Steffen
Aw: Re: Any suggestions for next Debian Med sprint (in person meeting)?
> [...] > >> No better idea from my part. The location in Berlin was actually > >> fine and I cannot propose a better (also closer to me) place. Last > >> time we discussed this, you evoked a place in Hamburg. Which > >> would not change a lot of things for me compared to Berlin. > > > > I remember someone suggested Hamburg but it was not me. If the > > person who suggested this would speak up it would be nice. > > Otherwise I would ask Sascha to make suggestions when the known venue > > is open for our sprint. > > I wouldn't mind going somewhere else for a change... but i can surely > ask if we can have another weekend in February. Hamburg would also be fine with me. Important for the med team is increasingly also what the ML team gets up to and the GPU acceleration fronts. Should we somehow reach out? Best, Steffen
Aw: GENtle - some progress
Quick update - upstream was very happy to see a first bunch of patches and created https://github.com/GENtle-persons/gentle-m with membership to GENtle-persons initially offered to Andreas and myself (I already accepted). So, I think my next steps are to transfer d/patches/* to an upstream dev branch and then sync with upstream about a new point release. Best, Steffen > Gesendet: Montag, 31. Juli 2023 um 12:39 Uhr > Von: "Steffen Möller" > An: "Med Team" > Betreff: GENtle - some progress > > Hello, > > I just uploaded another iteration of GENtle to our servers. It has some > quirks, still, as in that wxWidgets' runtime checks kick in with the one or > other window that is opened, something about the windows parents not set > right, but I guess this is usable. I'll finish this up over the next weeks. > > Some background, GENtle comes from a time when the cloning of DNA (as in: get > some large string of unknown DNA of interest connected in a much larger piece > of well-known DNA that can be multiplied by bacteria) was a precondition to > get that DNA sequenced. With technological advancements that has mostly been > substituted with "let us just sequence the whole organism". And there is > software like https://www.snapgene.com/ doing many things right that come to > a rescue for smaller groups and/or smaller projects, and for all the many > tasks like amplifying the gene product of what was cloned (as in producing > insulin, see https://openinsulin.org/). > > IMHO times have changed again in that we already have sequenced of what we > want to see sequenced and now the research community shifted from "get DNA > out from human cells" to "get DNA into (typically cultures of) human cells". > To move any gene across the cell wall, you need a special kind of vehicle, > and technically those look just like those plasmids that GENtle already knows > how to design. So this is not just some old outdated technology lurking with > us, we should just think about how we can modernize the GUI a bit and maybe > add some public data. > > I'll discuss data to be added with my wet-lab colleagues, also some > imperfections in software may be ideal as student projects to > describe/fix/learn, will see. > > Best, > Steffen > >
GENtle - some progress
Hello, I just uploaded another iteration of GENtle to our servers. It has some quirks, still, as in that wxWidgets' runtime checks kick in with the one or other window that is opened, something about the windows parents not set right, but I guess this is usable. I'll finish this up over the next weeks. Some background, GENtle comes from a time when the cloning of DNA (as in: get some large string of unknown DNA of interest connected in a much larger piece of well-known DNA that can be multiplied by bacteria) was a precondition to get that DNA sequenced. With technological advancements that has mostly been substituted with "let us just sequence the whole organism". And there is software like https://www.snapgene.com/ doing many things right that come to a rescue for smaller groups and/or smaller projects, and for all the many tasks like amplifying the gene product of what was cloned (as in producing insulin, see https://openinsulin.org/). IMHO times have changed again in that we already have sequenced of what we want to see sequenced and now the research community shifted from "get DNA out from human cells" to "get DNA into (typically cultures of) human cells". To move any gene across the cell wall, you need a special kind of vehicle, and technically those look just like those plasmids that GENtle already knows how to design. So this is not just some old outdated technology lurking with us, we should just think about how we can modernize the GUI a bit and maybe add some public data. I'll discuss data to be added with my wet-lab colleagues, also some imperfections in software may be ideal as student projects to describe/fix/learn, will see. Best, Steffen
Aw: Re: Please review release notes patch
Heya, > Gesendet: Donnerstag, 25. Mai 2023 um 09:51 Uhr > Von: "Andreas Tille" > An: debian-med@lists.debian.org > Cc: "Steffen Möller" > Betreff: Re: Please review release notes patch > > Hi Nilesh, > > Am Wed, May 24, 2023 at 10:16:22PM +0530 schrieb Nilesh Patra: > > On Wed, May 24, 2023 at 11:28:13AM +0200, Pierre Gruet wrote: > > > Le 24/05/2023 à 08:25, Andreas Tille a écrit : > > > > > > > > https://salsa.debian.org/med-team/community/communication/-/blob/master/releasenotes/bookworm/release-notes.patch > > > > > > > > Please review and comment on it (or just push fixes and enhancements)! > > > Is there any important piece of software we packaged during this release > > > cycle and that could be worth highlighting? From my limited perspective I > > > have none that comes to mind, but maybe it will for someone else. > > > > I do remember that during the bullseye release, we were looking forward > > to get nextflow into bookworm. AFAICS, that did not happen but I do see > > a capsule-nextflow package. > > Although it is mostly a deployment tool, _maybe_ it is worth a mention? > > > > I've CC'ed Steffen for any inputs about the same. > > Steffen? Nextflow has not made it. Pierre summarized the state not to long ago. The problem was the change to the build tool that Debian does not support its latest version. > IMHO I consider it less worth mentioning than shiny-server. shiny-server is a piece of infrastructure to run services, obviously. It is important, although many would say that it is not a core piece of bioinformatics. I propose to adjust the release notes accordingly, also adopting a bit of what Pierre described wrt dependencies. Here my shot: For the past 20 years, Debian has been a trusted distributor of software for the Life Sciences and Medicine, offering all the benefits that come with Debian as a distribution. This commitment has supported education, research, and service providers who rely on web-based software solutions. Particularly in the life sciences field, where data often exceeds transport capacity, Debian's contributions are of immense value. With the release of Debian Bookworm, we are excited to introduce the shiny-server package, which enables the creation of scientific web applications using the statistical environment R. Much of our work goes unnoticed by users who simply see their familiar packages updated. However, behind the scenes, we continuously strive to improve the reliability and quality of our software. Through enhanced Continuous Integration support, the packages maintained by the Debian Med team undergo official auto-tests and additional tests developed by our team. By keeping software dependencies up to date, multiple packages can work seamlessly together, benefiting from the collective identification of issues and ensuring a smooth user experience. Any patches we create are shared back with the original developers, allowing Debian Med to remain closely aligned with the original sciences. Bookworm ships with >1000 packages that are maintained by the Debian Med group that are kept compatible with the very latest versions of the software itself, but most work is invested into maintaining that often aging but established software to remain compatible with the latest versions of the shared libraries that often experiences incompatible changes to their API. This effort has become increasingly difficult over the past years as softwares have grown in complexity with many more dependencies, and also seeing many overlaps with other disciplines, such that the contributors to Debian Med also find themselves contributing to the Science and Electronic teams, or just help with basic Java or Python libraries that have not yet been packaged. But waiting for Debian to provide packages for all the latest softwares available today would slow your Science down. To the rescue may come the updated package of Singularity, i.e. a non-privileged means to install externally prepared software images, and we already know many such images to have Debian as their basis. The Debian Med team values feedback from users, especially regarding requests for packaging previously unpackaged free software or backports to earlier releases that are important to you. To install the packages maintained by the Debian Med team, simply install the metapackages named med-*, which are currently at version 3.8.x for Debian Bookworm. You can explore the full range of biological and medical software available in Debian by visiting the https://blends.debian.org/med/tasks;>Debian Med tasks pages. Please extend/shorten/mod as you see fit. Best, Steffen
Aw: Re: Nextflow - have just used it on our HPC cluster and liked it
> Gesendet: Montag, 08. Mai 2023 um 08:45 Uhr > Von: "Charles Plessy" > An: debian-med@lists.debian.org > Betreff: Re: Nextflow - have just used it on our HPC cluster and liked it > > Hi Steffen and everybody, > > I also use Nextflow at work, and indeed, it makes it very easy to run a > pipeline many times. Also, Nextflow is a single Java executable, which > makes it easy to deploy anywhere Java is already installed. > > You probably also saw the nf-core repository of modules and pipelines. > I like the way they are organised and find them empowering. The > community is very nice too. > > BUT > > With my Debian background it is very hard for me to adapt to the conda / > biocona / quay / galaxy ecosystem. I just can not figure out who is > responsible for what, no idea how long the whole thing will be > supported, where is the source code used to build the the packages into > Docker images in to Singularity images, etc. Not to mention that the > whole paradigm behind "one tool, one minimal image" deprives me from all > the Unix tools that I use to enjoy on in a Debian context. In bioconda > you have no idea whether sed is from GNU of from busybox unless you try > it or dig for a package recipe in GitHub... I am struggling with conda environments, I must admit. This should be mostly analogous to chroot environments, I keep thinking, but still ... > > NOT TO MENTION THAT > > Everybody expects that these images will stay forever and for free at > the URL where they are, while I have not seen any evidence of an > organisation promising that it will really happen for at least a > decade... Without these images and the receipes to create them > (remember the singularity <- docker <- conda <- GitHub fragmentation), > the hope that these pipelines provide reproducibility in the long term > is wishful thiking. I admit to care more about the data than the exact tools. Just rerun it with whatever was proven to be superior. The longevity of https://biocontainers.pro/ will basically determine about what we shall expect and conversely our demands will shape what will be offered. > SO > > Against the stream of minimising image size to the bone while processing > terabytes of sequencing data, I thing that Debian Med images with all of > our packages installed would be a useful alternative in many cases. Yes - for the direct execution but also within images. > I am already doing something along the lines on our HPC cluster to turn > our packages into environment modules (lmod). > > https://github.com/oist/BioinfoUgrp/blob/master/DebianMedModules.md#creation-of-a-new-singularity-image > > The size of the images is a bit less than 8 GiB, and I make a new image > at each point release. Would there be some interest to make such images > in a more official way ? We could have our own Singularity Hub (https://singularityhub.github.io/). @Olivier, Hervé, Matúš et al. - I would happily hear from you that we just do not need anything like that. Best, Steffen
Nextflow - have just used it on our HPC cluster and liked it
Hello,
I must admit that I am rather impressed - a series of images were
auto-downloaded to function together with singularity and this then worked on a
test data collection of the workflow. So, with singularity (or docker) in
Debian, and nextflow, we would have an immediate sync with what upstream offers.
I had a look at the table in Google docs and found that the one dependency that
was once missing to get nextflow through its autotests, i.e. capsule-nextflow
is in the archive now. So I thought to give the main package of nextflow
another look. I am impressed by all work that Pierre already invested into the
packaging. @Pierre, where does that work stand? I only saw that kryo5 is listed
as a dependency that is not in the archive.
Best,
Steffen
A "debian/rules binary" got this far:
Starting process 'Gradle Worker Daemon 1'. Working directory:
/Salsa/Med/nextflow/.gradle/workers Command:
/usr/lib/jvm/java-17-openjdk-amd64/bin/java
@/tmp/gradle-worker-classpath9650940302470954557txt -Dfile.encoding=UTF-8
-Duser.country -Duser.language=en -Duser.variant
worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Worker
Daemon 1'
Successfully started process 'Gradle Worker Daemon 1'
Started Gradle worker daemon (3.129 secs) with fork options
DaemonForkOptions{executable=/usr/lib/jvm/java-17-openjdk-amd64/bin/java,
minHeapSize=null, maxHeapSize=null, jvmArgs=[],
classpath=[/usr/share/maven-repo/org/codehaus/groovy/groovy-ant/debian/groovy-ant-debian.jar,
/usr/share/maven-repo/org/codehaus/groovy/groovy-groovydoc/debian/groovy-groovydoc-debian.jar,
/usr/share/maven-repo/org/codehaus/groovy/groovy-templates/debian/groovy-templates-debian.jar,
/usr/share/maven-repo/org/codehaus/groovy/groovy-xml/debian/groovy-xml-debian.jar,
/usr/share/maven-repo/org/codehaus/groovy/groovy/debian/groovy-debian.jar,
/usr/share/maven-repo/org/apache/ant/ant-junit/debian/ant-junit-debian.jar,
/usr/share/maven-repo/org/apache/ant/ant/debian/ant-debian.jar,
/usr/share/maven-repo/org/apache/ant/ant-launcher/debian/ant-launcher-debian.jar,
/usr/share/maven-repo/org/apache/ant/ant-antlr/debian/ant-antlr-debian.jar,
/usr/share/java/ant-1.10.13.jar, /usr/share/java/ant-launcher-1.10.13.jar],
keepAliveMode=SESSION}.
This JVM does not support getting OS memory, so no OS memory status updates
will be broadcast
Initialized native services in: .../Salsa/Med/nextflow/.gradle/native
Compiling with JDK Java compiler API.
warning: [options] bootstrap class path not set in conjunction with -source 8
Note: org.pf4j.processor.ExtensionAnnotationProcessor init
Note: Options {}
Note: Processing @org.pf4j.Extension
warning: Implicitly compiled files were not subject to annotation processing.
Use -proc:none to disable annotation processing or -implicit to specify a
policy for implicit compilation.
2 warnings
startup failed:
.../Salsa/Med/nextflow/modules/nf-commons/src/main/nextflow/plugin/CustomPluginManager.groovy:
68: [Static type checking] - Cannot find matching method
org.pf4j.SingletonExtensionFactory#(). Please check if the declared type
is correct and if the method exists.
@ line 68, column 16.
return new SingletonExtensionFactory()
Bioconductor 400MB packages in Debian?
Hello, Some expression data crossed my way and while conda install -c bioconda bioconductor-clariomdhumantranscriptcluster.db is no big deal (unless when it is), we do not support it in Debian - it is big. Well, at least there was a time when this was considered big, I am not so sure any more, so here I ask. https://bioconductor.org/packages/release/data/annotation/html/pd.clariom.d.human.html comes in a ~400MB tarball, which is mostly a SQLite database. I would support a decision according to which anybody interested would just go and run the installer, but the dependencies are non-trivial, so you effectively end up in installing all of BioConductor in parallel to what Debian already offers, which I am not a fan of. What do you think? Best, Steffen
Aw: Re: Possibility of In-person sprints in 2023?
> Gesendet: Samstag, 29. Oktober 2022 um 20:55 Uhr > Von: "Andreas Tille" > An: debian-med@lists.debian.org > Betreff: Re: Possibility of In-person sprints in 2023? > > Hi, > > Am Sun, Oct 30, 2022 at 12:58:04AM +0530 schrieb Nilesh Patra: > > I suppose sprints in 2021 and 2022 have been virtual so far due to COVID. > > There were some COVID related sprints in 2020 as well. > > > > Is there a possibilty to have an in-person debian-med sprint in 2023? > > Would someone be interested in it, or so? > > I'm much in favour of an in-person meeting in the beginning of 2023. > Since it would be great to invite Nilesh it makes sense to meet > somewhere close to an international airport. Sascha, would it be > possible again in the rooms of your employer in Berlin? Any other > suggestions? I agree, it would be good to meet. Should we not meet up in your home town, Andreas? (Or Nilesh?) I could collect Nilesh from the Frankfurt Airport to guide him to your mountainous hideout. Alternatively I would contact a few institutes in Germany that perform Covid 19 research on live viruses and ask them if they would host us - Hamburg, Berlin or some of this historically cool places like https://www.fli.de/ (built on an island with a purpose). Best, Steffen
Re: We should do an update of all qiime2 modules as soon as possible (Was: Bug#1014100: qiime: autopkgtest needs update for new version of python-decorator)
On 30.06.22 11:51, Andreas Tille wrote: I'd like to point out to this serious bug. Steffen, would you mind koordinating (not doing alone) an upgrade of the qiime infrastructure to make sure everything will go smoothly? I added "to be updated" tags to all the qiime2 packages on https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1986383821 Yes, these updates are overdue. I'll see what I can do, help is of course much appreciated. Many thanks! Steffen
Re: Accepted ebook-tools 0.2.2-5 (source) into unstable
On 29.05.22 19:06, Aaron M. Ucko wrote: Debian FTP Masters writes: * Make epub-utils conflict with ncbi-entrez-direct, as both ship the einfo executable & manpage. I appreciate the thought, but ncbi-entrez-direct goes out of its way to avoid any such conflict: it diverts epub-utils' instances of those files, substituting a wrapper script that takes advantage of major command-line syntax differences to determine which tool the user presumably meant to run (and a manpage that notes the diversion). AFAICT, this arrangement works fine; have you found otherwise? FTR, there's a similar situation around efetch, where both executables come from biology-related packages where a conflict would be more of a problem: acedb-other and ncbi-entrez-direct (with the latter taking care of deconflicting in the same fashion). acedb-other shall step behind anything that is offered from the NCBI realm, I suggest. Many thanks! Steffen
Freexian-funding for what keeps nagging us?
Hello, I just learned about https://freexian-team.pages.debian.net/project-funding/ and to mind came our difficulties with scala (the Java functional programming flavor). This may be of sufficient interest for Debian at large to get bootstrapped, I think. Other ideas? Best, Steffen
Re: pigx-rnaseq now depends on megadepth
Hello, I just also prepared https://salsa.debian.org/med-team/megadepth Comments and/or upload would be lovely. Many thanks and regards, Steffen On 31.03.22 15:35, Steffen Möller wrote: I was not aware of megadepth - but that is why to pick some experts' workflow system in the first place as a reference. https://salsa.debian.org/r-pkg-team/r-bioc-megadepth needs https://salsa.debian.org/r-pkg-team/r-cran-cmdfun and I am still working on the megadepth binary that is wrapped. So, for the moment this needs to be installed independently. I suggest to nonetheless upload the package already. I would love to see the package(s) reviewed. Many thanks Steffen
pigx-rnaseq now depends on megadepth
I was not aware of megadepth - but that is why to pick some experts' workflow system in the first place as a reference. https://salsa.debian.org/r-pkg-team/r-bioc-megadepth needs https://salsa.debian.org/r-pkg-team/r-cran-cmdfun and I am still working on the megadepth binary that is wrapped. So, for the moment this needs to be installed independently. I suggest to nonetheless upload the package already. I would love to see the package(s) reviewed. Many thanks Steffen
Re: Upgrading biojava-live from 1.7 to 1.9
On 28.03.22 17:15, Andreas Tille wrote: Am Mon, Mar 28, 2022 at 04:42:15PM +0200 schrieb olivier sallou: ... that's fine for me Thanks a lot for this effort Many thanks - brings some memories back! Steffen
RFreview: Another R time series tool - Mfuzz
Hello, https://www.bioconductor.org/packages/release/bioc/vignettes/Mfuzz/inst/doc/Mfuzz.pdf is what I am after and this needs r-bioc-widgettools https://salsa.debian.org/r-pkg-team/r-bioc-widgettools r-bioc-dyndoc https://salsa.debian.org/r-pkg-team/r-bioc-dyndoc r-bioc-tkwidgets https://salsa.debian.org/r-pkg-team/r-bioc-tkwidgets and r-bioc-mfuzz itself https://salsa.debian.org/r-pkg-team/r-bioc-mfuzz Reviews and uploads would be appreciated. Many thanks and regards, Steffen
q2-composition - removed vega* from source tree
Hello, I am tempted to upload the package as it is. It is apparently version 4 of Vega that is expected, not version 5 that just entered the archive (many, many thanks!!) and they are apparently using https://github.com/vega/vega-embed which we yet do not have. However, those embedded vega files will affect the presentation of the results but not the results themselves and as such this package is already useful. I propose to craft a bug report early about the package to prevent its migration to testing and talk to upstream about. https://salsa.debian.org/med-team/q2-composition What do you think? Best, Steffen
Request for peer review (and upload) of r-bioc-tmixclust and two dependencies
Hello, This is all about analysing time-series of gene expression data. I just packed https://www.bioconductor.org/packages/devel/bioc/vignettes/TMixClust/inst/doc/TMixClust.pdf (on https://salsa.debian.org/r-pkg-team/r-bioc-tmixclust) and its dependencies that Debian did not yet offer (https://salsa.debian.org/r-pkg-team/r-bioc-spem and https://salsa.debian.org/r-pkg-team/r-cran-flexclust). Please kindly have a look that I have not missed anything dramatic. An upload would be appreciated, alternatively a quick note that everything is ready to go. Many thanks Steffen
Re: q2-emperor - flagged as "RFS"ed
On 17.02.22 22:01, Nilesh Patra wrote: On 2022-02-18 01:34, Steffen Möller wrote: Hello, q2-emperor is flagged as "RFS" in https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1986383821 and late January, its dependency EMPEROR made it into the distribution. I have not seen any rejection email in my inbox (and it should be flagged as "new" then). May I ask for a peer review and upload? Uploaded after some scrutiny. Thanks! Great! Thank you! Steffen
q2-emperor - flagged as "RFS"ed
Hello, q2-emperor is flagged as "RFS" in https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1986383821 and late January, its dependency EMPEROR made it into the distribution. I have not seen any rejection email in my inbox (and it should be flagged as "new" then). May I ask for a peer review and upload? Best, Steffen
does anyone know of its whereabouts? Fwd: r-bioc-progeny_1.14.0+ds-1_amd64.changes REJECTED
Hello, I just checked but could not find any info on r-bioc-progeny. Andreas has fixed the missing license info for https://afeld.github.io/bootstrap-toc/ (which looks good btw) . @Andreas do you happen to recall if you have uploaded this again? Or was this rejected with a request to provide a package for bootstrap-toc? The package would also work without it, so I may presume, and I would then vote for a simplification. Many thanks and regards Steffen Forwarded Message Subject:r-bioc-progeny_1.14.0+ds-1_amd64.changes REJECTED Date: Tue, 23 Nov 2021 19:00:08 + From: Thorsten Alteholz To: Debian R Packages Maintainers , Steffen Moeller Hi Steffen, please mention the license of bootstrap in your debian/copyright. Thanks! Thorsten === Please feel free to respond to this email if you don't understand why your files were rejected, or if you upload new files which address our concerns.
Re: sambamba and freebayes new releases
Also many thanks to you both and greetings from my side! Best, Steffen On 21.01.22 12:17, Andreas Tille wrote: Am Fri, Jan 21, 2022 at 11:45:48AM +0100 schrieb Pjotr Prins: We release sambamba and freebayes with updated meson build systems. It should work for Debian. If not I am happy to troubleshoot. $ apt policy sambamba sambamba: Installed: (none) Candidate: 0.8.2+dfsg-1 Version table: 0.8.2+dfsg-1 501 501 http://deb.debian.org/debian testing/main amd64 Packages 50 http://deb.debian.org/debian unstable/main amd64 Packages $ $ dput freebayes_1.3.6-1_source.changes Uploading freebayes using ftp to ftp-master (host: ftp.upload.debian.org; directory: /pub/UploadQueue/) running allowed-distribution: check whether a local profile permits uploads to the target distribution running protected-distribution: warn before uploading to distributions where a special policy applies running checksum: verify checksums before uploading running suite-mismatch: check the target distribution for common errors running gpg: check GnuPG signatures before the upload signfile dsc freebayes_1.3.6-1.dsc ti...@debian.org fixup_buildinfo freebayes_1.3.6-1.dsc freebayes_1.3.6-1_amd64.buildinfo signfile buildinfo freebayes_1.3.6-1_amd64.buildinfo ti...@debian.org fixup_changes dsc freebayes_1.3.6-1.dsc freebayes_1.3.6-1_source.changes fixup_changes buildinfo freebayes_1.3.6-1_amd64.buildinfo freebayes_1.3.6-1_source.changes signfile changes freebayes_1.3.6-1_source.changes ti...@debian.org Successfully signed dsc, buildinfo, changes files Uploading freebayes_1.3.6-1.dsc Uploading freebayes_1.3.6.orig.tar.gz Uploading freebayes_1.3.6-1.debian.tar.xz Uploading freebayes_1.3.6-1_amd64.buildinfo Uploading freebayes_1.3.6-1_source.changes So sambamba seems to be the latest version in testing and freebayes should follow soon. Thanks a lot for pinging here which is really welcome Andreas.
Re: RF peer review: r-other-ibi-disgenet2r freshly injected into salsa
On 22.12.21 14:32, Andreas Tille wrote: Hi Steffen, Am Wed, Dec 22, 2021 at 01:47:35PM +0100 schrieb Steffen Möller: https://salsa.debian.org/r-pkg-team/r-other-ibi-disgenet2r just arrived. It needs r-cran-sparql that Andreas kindly uploaded yesterday. I've added a watch file, renamed to what dh-make-R would have named Ok, I have just changed the salsa path accordingly to https://salsa.debian.org/r-pkg-team/r-other-disgenet2r/ . the package and uploaded. Great! Best, Steffen
RF peer review: r-other-ibi-disgenet2r freshly injected into salsa
Hello again, https://salsa.debian.org/r-pkg-team/r-other-ibi-disgenet2r just arrived. It needs r-cran-sparql that Andreas kindly uploaded yesterday. Many thanks and regards, Steffen
RFS: r-cran-sparql - dependency for a more typical Med package
Dear Andreas, dear Nilesh, dear all, I would appreciate some extra scrutiny for https://salsa.debian.org/r-pkg-team/r-cran-sparql Please kindly upload if you consider this "ready-enough" or pass the token back to me, please. This https://www.disgenet.org/disgenet2r is why I (and Debian at large, you would have thought that we long feature a SPARQL package) am after it. Many thanks! Steffen
Re: Moving python-bioblend to med-team?
Please go ahead - thank you both. If there is any particular action for me to perform then please ping me, preferably with a respective tag in the subject line. Best Steffen On 19.12.21 12:05, Andreas Tille wrote: Hi Nilesh and Steffen, Am Sun, Dec 19, 2021 at 01:36:50PM +0530 schrieb Nilesh Patra: python-bioblend is maintained on DPT, however it seems to be made for the purpose of interacting with galaxy project's APIs. Hence I was wondering if it is of our interest? I'm absolutely in favour of this. It has also not seen uploads for a long time, and I was wondering if it makes sense to move it here. I wanted to fix a problem on this package as well. Please go for it - usually Steffen agrees to those kind of moves and I think he is currently very busy. Kind regards Andreas.
r-bioc-txdb.hsapiens.ucsc.hg19.knowngene on salsa Re: please kindly review: r-bioc-fishpond -- Fishpond: differential transcript and gene expression with
Hi all, On 10.12.21 17:46, Andreas Tille wrote: Am Fri, Dec 10, 2021 at 11:57:49AM +0100 schrieb Steffen Möller: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001436 for r-bioc-txdb.hsapiens.ucsc.hg19.knowngene and when I wanted to inject it found that it is already existing at https://salsa.debian.org/r-pkg-team/r-bioc-txdb.hsapiens.ucsc.hg19.knowngene in a "debian folder only" stage and 4 years old. You do not mind if I somehow adopt that? I don't mind at all. I remember that I used debian/ dir only due to the size of the source which might have been an issue at the time of creation. Technically I wonder whether it makes sense to remove the repository first and create it from scratch via prepare_missing_cran_package. It could perfectly be the case that it is the more straight approach to get a sensible package - but I'll leave this decision to you. I did not have the privileges to remove the previous r-bioc-txdb.hsapiens.ucsc.hg19.knowngene repository but could rename it, so that is what I have done and then injected the new one. Best, Steffen
Re: please kindly review: r-bioc-fishpond -- Fishpond: differential transcript and gene expression with
On 10.12.21 07:15, Andreas Tille wrote: Am Fri, Dec 10, 2021 at 03:07:51AM +0530 schrieb Nilesh Patra: On Thu, Dec 09, 2021 at 08:32:46PM +0100, Steffen Möller wrote: Hello this goes out to Andreas in particular, I hope you do not mind me taking over (have a few spare cycles) I definitely encourage everybody to take over anything that was addressed to me in particular! ;-) Same here :) And I have another one . I just prepared this ITP https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1001436 for r-bioc-txdb.hsapiens.ucsc.hg19.knowngene and when I wanted to inject it found that it is already existing at https://salsa.debian.org/r-pkg-team/r-bioc-txdb.hsapiens.ucsc.hg19.knowngene in a "debian folder only" stage and 4 years old. You do not mind if I somehow adopt that? Best, Steffen Uploaded after a few changes. Thanks a lot Andreas.
please kindly review: r-bioc-fishpond -- Fishpond: differential transcript and gene expression with
Hello this goes out to Andreas in particular, I completed the packaging of r-bioc-fishpond earlier this afternoon. This is a suggestion for r-bioc-tximport with some other interesting reverse-dependencies - and tximport makes problems with my rabbit data that I yet do not understand. The package does not seem to be overly complicated, builds in cowbuilder. I just injected everything to https://salsa.debian.org/r-pkg-team/r-bioc-fishpond Many thanks Steffen Forwarded Message Subject:ITP: r-bioc-fishpond -- Fishpond: differential transcript and gene expression with Date: Thu, 09 Dec 2021 15:29:05 +0100 From: Steffen Moeller To: Debian Bug Tracking System Package: wnpp Severity: wishlist Subject: ITP: r-bioc-fishpond -- Fishpond: differential transcript and gene expression with Package: wnpp Owner: Steffen Moeller Severity: wishlist * Package name : r-bioc-fishpond Version : 2.0.1+ds Upstream Author : Anzi Zhu, * URL : https://bioconductor.org/packages/fishpond/ * License : GPL-2 Programming Lang: GNU R Description : Fishpond: differential transcript and gene expression with inferential replicates Fishpond contains methods for differential transcript and gene expression analysis of RNA-seq data using inferential replicates for uncertainty of abundance quantification, as generated by Gibbs sampling or bootstrap sampling. Also the package contains utilities for working with Salmon and Alevin quantification files. Remark: This package is maintained by Debian R Packages Maintainers at https://salsa.debian.org/r-pkg-team/r-bioc-fishpond
Re: Debian Med video conference tomorrow, Wednesday 2021-11-17 18:00 UTC
Heya, On 16.11.21 19:58, Nilesh Patra wrote: For those who are interested in hot topics we want to tackle, here are some action items to be done: - Coordinating work for bioconductor transition (to be started soon) Nice! I am traveling and cannot join you. From my side I can only report that I keep investigating the JavaScript headaches that PiGx RNAseq is causing in its report. Running with guix, everthing went just smoothly. That is so annoying. The vega JS library seems to be something else to cause problems for multiple packages of the Qiime suite, so the JS situation is something I likely "want" to investigate a bit further. Enjoy! Best, Steffen
Re: no report generation, likely not only for pigx-rnaseq Re: What happened to r-cran-jquerylib ?
On 03.11.21 15:55, Andreas Tille wrote: Hi Steffen, Am Wed, Nov 03, 2021 at 12:58:53PM +0100 schrieb Steffen Möller: Hi Andreas et al., We should think about what it means for us not to have a current version of the DT cran package. For one, it kills the PIGX RNA-seq workflow. What do you mean by "we should think"? All we need to do is getting r-cran-jquerylib *properly*. Uploading a rejected package to new without fixing the reasons for a reject is not speeding up things. I layed out what I consider to have good chances to pass ftpmaster inspection (see below). Before I start working on this please confirm that you asked for rejection of the current upload. I presume in the meantime you have found the CC to r-pkg-team in which I was asking for said removal. It predated the email to med by a couple of minutes. Concerning the symbolic links - maybe the idea is better than I had initially thought. Would there be a way to have a warning written when an older version is requested? Maybe we should just not have the older versions and provoke a clear and easily(?) detectable error. The reason I am asking is that with pigx-rnaseq I have generated reports that do not show two interactive figures. And I did not see anything obvious on the JavaScript console. I now installed with guix to compare. The more we fiddle with JavaScript, though, the more likely such difficulties are likely to develop. Steffen On 03.11.21 12:01, Andreas Tille wrote: Hi again Steffen, I'd strongly recommend to ask for rejection of your upload to NEW to not create extra hassle for ftpmaster. Kind regards Andreas. Am Tue, Nov 02, 2021 at 09:43:07PM +0100 schrieb Andreas Tille: Hi Am Sun, Oct 31, 2021 at 07:16:44PM +0100 schrieb Steffen Möller: I just found there is a new upstream version. I'll upload it and then likely see it in new. That will be not successfully, thought. My idea was to remove jquery 1.x and 2.x from the package and symlink to the Debian packaged version of jquerylib 3.x. Unfortunately I did not managed to do this up to now. Feel free to beat me in it - but simply uploading a new version to new will just annoy ftpmaster.
no report generation, likely not only for pigx-rnaseq Re: What happened to r-cran-jquerylib ?
Hi Andreas et al., We should think about what it means for us not to have a current version of the DT cran package. For one, it kills the PIGX RNA-seq workflow. Steffen On 03.11.21 12:01, Andreas Tille wrote: Hi again Steffen, I'd strongly recommend to ask for rejection of your upload to NEW to not create extra hassle for ftpmaster. Kind regards Andreas. Am Tue, Nov 02, 2021 at 09:43:07PM +0100 schrieb Andreas Tille: Hi Am Sun, Oct 31, 2021 at 07:16:44PM +0100 schrieb Steffen Möller: On 31.10.21 19:14, Nilesh Patra wrote: On 10/31/21 11:30 PM, Steffen Möller wrote: Hello, I think this goes out to Andreas. https://salsa.debian.org/r-pkg-team/r-cran-jquerylib/ states that it was on the NEW queue and somewhere deep in me I recall to have seen it on there but it seems gone. It is not in the archive though and I do not recall a rejection email to the list. Right, I just did a quick check, and it does not seem to be rejected $ ssh -t coccia.debian.org 'ls -1 /srv/ftp-master.debian.org/queue/reject/r-cran-* | grep jquery | wc -l' 2>/dev/null 0 Well, I usually CC the ITP bug in my response to a reject - so feel free to see here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994867 I manually checked in the dir as well, but I don't see any reject reason for jquerylib :/ @Andreas, did your dput job fail somehow? Was this uploaded properly? I just found there is a new upstream version. I'll upload it and then likely see it in new. That will be not successfully, thought. My idea was to remove jquery 1.x and 2.x from the package and symlink to the Debian packaged version of jquerylib 3.x. Unfortunately I did not managed to do this up to now. Feel free to beat me in it - but simply uploading a new version to new will just annoy ftpmaster. Kind regards Andreas. -- http://fam-tille.de
Re: snakemake-dependency Fwd: Bug#996868: ITP: python-connection-pool -- thread-safe connection pool for python
On 20.10.21 14:08, Nilesh Patra wrote: On 20 October 2021 6:26:29 am IST, "Steffen Möller" wrote: Snakemake is working for me with python3-connection-pool installed, so I have uploaded. Best, Steffen I had uploaded with my hack, and autopkgtests work in salsa CI. I think we simply revert the hack when we have connection pool in archive Thank you for this. Steffen
snakemake-dependency Fwd: Bug#996868: ITP: python-connection-pool -- thread-safe connection pool for python
Snakemake is working for me with python3-connection-pool installed, so I have uploaded. Best, Steffen Forwarded Message Subject:Bug#996868: ITP: python-connection-pool -- thread-safe connection pool for python Resent-Date:Wed, 20 Oct 2021 00:33:01 + Resent-From:Steffen Moeller Resent-To: debian-bugs-d...@lists.debian.org Resent-CC: moel...@debian.org, w...@debian.org Date: Wed, 20 Oct 2021 02:31:46 +0200 From: Steffen Moeller Reply-To: Steffen Moeller , 996...@bugs.debian.org To: Debian Bug Tracking System Package: wnpp Severity: wishlist Subject: ITP: python-connection-pool -- thread-safe connection pool for python Package: wnpp Owner: Steffen Moeller Severity: wishlist * Package name : python-connection-pool Version : 0.0.3 Upstream Author : ZhouYL <81438...@qq.com> * URL : https://github.com/zhouyl/ConnectionPool * License : Expat Programming Lang: Python Description : thread-safe connection pool for python This package provides a thread-safe connection pool for Python 3. These are sets database connections that are readily available for anticipated repeated requests to the same database. Remark: This package is maintained by Steffen Moeller at https://salsa.debian.org/python-team/packages/python-connection-pool
Re: snakemake_6.9.1+dfsg1-1_source.changes ACCEPTED into unstable
I just ran into this. Will package this with the Python team now. On 19.10.21 23:40, Nilesh Patra wrote: On Wed, 20 Oct 2021 at 00:57, Steffen Möller mailto:steffen_moel...@gmx.de>> wrote: On 19.10.21 21:12, Nilesh Patra wrote: > On 10/20/21 12:40 AM, Steffen Möller wrote: >> Update follows. There is an uninstallable file in the tests folders. I >> am now separating -tests and -doc. > > Thanks, but that'd go a trip to NEW. It'd be nice if you can fix it in > the single-source > package only, for now. -- So as to not stall migration. What do you > think? I see that you did another upload, thanks. But seeing the autopkgtest result[1] It looks like it needs connection_pool package, which does not seem to be present in the archive :-( [1]: https://salsa.debian.org/med-team/snakemake/-/jobs/2098048 <https://salsa.debian.org/med-team/snakemake/-/jobs/2098048>
Re: snakemake_6.9.1+dfsg1-1_source.changes ACCEPTED into unstable
On 19.10.21 21:12, Nilesh Patra wrote: On 10/20/21 12:40 AM, Steffen Möller wrote: Update follows. There is an uninstallable file in the tests folders. I am now separating -tests and -doc. Thanks, but that'd go a trip to NEW. It'd be nice if you can fix it in the single-source package only, for now. -- So as to not stall migration. What do you think? Admittedly I had not thought about that. You have a point. So, yes. I'll have another local iteration. Steffen
Re: snakemake_6.9.1+dfsg1-1_source.changes ACCEPTED into unstable
Update follows. There is an uninstallable file in the tests folders. I am now separating -tests and -doc . And these .snakemake folders come from installdocs, which is why all these removals failed. Getting there. Best, Steffen On 19.10.21 20:17, Nilesh Patra wrote: > Many thanks, Steffen :-) > > On Tue, 19 Oct, 2021, 10:50 pm Debian FTP Masters, mailto:ftpmas...@ftp-master.debian.org>> wrote: > > > > Accepted: > Format: 1.8 Date: Tue, 19 Oct 2021 16:32:04 +0200 Source: snakemake Architecture: source Version: 6.9.1+dfsg1-1 Distribution: unstable Urgency: medium Maintainer: Debian Med Packaging Team mailto:debian-med-packag...@lists.alioth.debian.org>> Changed-By: Nilesh Patra mailto:nil...@debian.org>> Changes: snakemake (6.9.1+dfsg1-1) unstable; urgency=medium . * Team upload. . [ Andreas Tille ] * Fix watchfile to detect new versions on github * New upstream version * Standards-Version: 4.6.0 (routine-update) * Respect DEB_BUILD_OPTIONS in override_dh_auto_test target (routine- update) * Remove trailing whitespace in debian/copyright (routine-update) * Build-Depends: python3-requests-mock . [ Nilesh Patra ] * d/control: + Add versioned Dep on pulp (>= 2.0) + Add B-D on python3-smart-open . [ Steffen Moeller ] * Unselected tests needing internet access * fixing one calling pytest -> pytest-3 * d/rules: Allowing consecutive builds with better cleaning * Added d/TODO to remind about missing "tes" Python module Checksums-Sha1: 9c190e155dfdea8b83447db5dc08535cac38e17c 3202 snakemake_6.9.1+dfsg1-1.dsc ad1d9ef3bd5d57b7923e5282ba0049101ee2ea27 5035772 snakemake_6.9.1+dfsg1.orig.tar.xz ff5459559c30f53a0c064e3cd19ea9866d640a37 21776 snakemake_6.9.1+dfsg1-1.debian.tar.xz 830981a8a5653b9cfbb8bd974f11a3946501bbeb 16999 snakemake_6.9.1+dfsg1-1_source.buildinfo Checksums-Sha256: cead2fce120ea6bd3873c84e0aa83878cf86db7f7d7aefcea8d1a98637d2e7bb 3202 snakemake_6.9.1+dfsg1-1.dsc 6fbef35cee9461e4f9ad8f2ea9608ccafe580bd3eeef2f659df3141b3802da67 5035772 snakemake_6.9.1+dfsg1.orig.tar.xz 5239a71667fff6c9f7424da571cd279fc30fbb581f1dc49c937e2a550193fee8 21776 snakemake_6.9.1+dfsg1-1.debian.tar.xz a3bcfa5dc15652395d203b73c44c953942db741883a6dc62a0dc683a29b92ca7 16999 snakemake_6.9.1+dfsg1-1_source.buildinfo Files: 4bf2d7714f8fe2aee04d01bd5785e7c1 3202 science optional snakemake_6.9.1+dfsg1-1.dsc 284ae10e744cff36b87cf7b729b99b4a 5035772 science optional snakemake_6.9.1+dfsg1.orig.tar.xz bea80d03859e3043379e00be47f00723 21776 science optional snakemake_6.9.1+dfsg1-1.debian.tar.xz e93718609b40a4dfe3287ea19fef630b 16999 science optional snakemake_6.9.1+dfsg1-1_source.buildinfo > > > Thank you for your contribution to Debian. >
Re: Lagging behind several versions for snakemake - lots of failed tests in latest version
On 18.10.21 18:41, Rebecca N. Palmer wrote: On 18/10/2021 08:15, Nilesh Patra wrote: On Mon, Oct 18, 2021 at 01:32:30AM +0200, Steffen Möller wrote: Is anyone continuing the update of snakemake now? @Rebecca, could you look into it now that pulp is done? Currently, there is just a very few failing tests -- I guess most are due to some or the other B-D missing, Yes, and probably. Hello, I also had a look since I need snakemake for pigx-rnaseq and there is a memory constraint problem that this update may fix if we are lucky (https://github.com/BIMSBbioinfo/pigx_rnaseq/issues/103). I removed a few tests that needed external data. I may have saved two though that just invoked the binaries "python" and "pytest" without the 3/-3 suffix. Best, Steffen
Re: Lagging behind several versions for snakemake - lots of failed tests in latest version
Hi Nilesh, On 17.10.21 23:27, Nilesh Patra wrote: Hi Steffen, On Mon, 18 Oct 2021 at 01:41, Steffen Möller mailto:steffen_moel...@gmx.de>> wrote: On 17.10.21 21:39, Nilesh Patra wrote: Hi Steffen, On Mon, 18 Oct 2021 at 01:06, Steffen Möller mailto:steffen_moel...@gmx.de>> wrote: On 17.10.21 21:19, Nilesh Patra wrote: Yeah, snakemake has now moved to pulp 2.0. pulp had another rev-dep on congress which is now removed from archive. pulp is maintained in openstack, and I sent out an email to zigo and openstack-list asking to upgrade, cf: https://lists.debian.org/debian-openstack/2021/10/msg1.html <https://lists.debian.org/debian-openstack/2021/10/msg1.html> I had been there and already got a reply from zigo (June this year) that we can go ahead with an update since pulp is (no longer?) used with Open Stack. I never got around to it, though. I Just paste our exchange below (don't expect anyone to mind). Wonderful, thanks for pasting this! Are you at it? -- If you could do the needful and upload :) Sorry for disappointing you but I am afraid I need to concentrate on something else this week. No worries, I have done so, and pushed my changes to: https://salsa.debian.org/science-team/python-pulp <https://salsa.debian.org/science-team/python-pulp> I really, _really_ need reviews. The very odd thing here is same content in postrm and prerm, and no update-alternatives for installing in postinst -- I admit, I do not understand pulp very well, hence would want reviews. Could you/someone else please, please take a look and upload? And oh, BTW I added you in the uploaders field since you planned to takeover this. Uploaded. lintian -i preferred the removal in prerm. So, it is a bit redundant now. I thought this was fine, though. Is anyone continuing the update of snakemake now? Best, Steffen
Re: Lagging behind several versions for snakemake - lots of failed tests in latest version
On 17.10.21 21:39, Nilesh Patra wrote: Hi Steffen, On Mon, 18 Oct 2021 at 01:06, Steffen Möller mailto:steffen_moel...@gmx.de>> wrote: On 17.10.21 21:19, Nilesh Patra wrote: Yeah, snakemake has now moved to pulp 2.0. pulp had another rev-dep on congress which is now removed from archive. pulp is maintained in openstack, and I sent out an email to zigo and openstack-list asking to upgrade, cf: https://lists.debian.org/debian-openstack/2021/10/msg1.html <https://lists.debian.org/debian-openstack/2021/10/msg1.html> I had been there and already got a reply from zigo (June this year) that we can go ahead with an update since pulp is (no longer?) used with Open Stack. I never got around to it, though. I Just paste our exchange below (don't expect anyone to mind). Wonderful, thanks for pasting this! Are you at it? -- If you could do the needful and upload :) Sorry for disappointing you but I am afraid I need to concentrate on something else this week. Best, Steffen
Re: Lagging behind several versions for snakemake - lots of failed tests in latest version
On 17.10.21 21:19, Nilesh Patra wrote: On 10/18/21 12:13 AM, Rebecca N. Palmer wrote: On 15/10/2021 14:46, Andreas Tille wrote: Hi Rebecca and whoever might care. As usual snakemake is troublesome to upgrade. I have injected the latest version into Git but there are lots of failed tests. It would be great if someone could care about this. I hadn't tried to upgrade snakemake because python-pulp is too old (#963041). Yeah, snakemake has now moved to pulp 2.0. pulp had another rev-dep on congress which is now removed from archive. pulp is maintained in openstack, and I sent out an email to zigo and openstack-list asking to upgrade, cf: https://lists.debian.org/debian-openstack/2021/10/msg1.html I had been there and already got a reply from zigo (June this year) that we can go ahead with an update since pulp is (no longer?) used with Open Stack. I never got around to it, though. I Just paste our exchange below (don't expect anyone to mind). On 6/8/21 1:40 PM, Steffen Möller wrote: Hi Thomas, Am 07.06.2021 um 09:05 schrieb Thomas Goirand: On 6/5/21 7:52 PM, Steffen Möller wrote: Hello, pulp is outdated. Would you mind updating it or having me update it? There is no immediate reason for me to update that version, the error I was looking for that made me check the version but then turned out to be elsewhere. Since I do not run OpenStack myself, I cannot test the effect of such a version bump and would hence prefer the OpenStack team to address this or to guide me towards it. Cheers, Steffen Hi, I don't think anything from OpenStack uses python-pulp, so feel free to take over the package if you care for it. Cheers, Thomas Goirand Ah, nice. That makes it easy. I had a first attempt last week and found the branch structure of your current repository to be incompatible with git-buildpackage (under my hands) and hence with tools like routine-update. For Debian Med (and increasingly Debian Science) routine-update is helping a lot, the automation also eases team maintenance. Hence, if you do not mind, I would leave everything intact for the OpenStack repository, copy'n'paste into a new repository that Debian Science an. Best, Steffen Hi, FYI, the only thing you got to do is have this in your ~/.gbp.conf: [DEFAULT] builder = sbuild --source-only-changes cleaner = /bin/true ignore-branch = True pristine-tar = False no-create-orig = True [buildpackage] export-dir = ../build-area/ The important bit is the ignore-branch and pristine-tar. IMO, these options should be by default in gbp, otherwise one has to maintain it in all of the repositories, and for a large amount of packages, that's too much (useless) work... Cheers, Thomas Goirand (zigo) The test failures look like this isn't the only problem, [...] ACK. Thanks a lot for your work! Yip! Many thanks also from my side. Steffen
salsa-ci initiation for Debian Med - suspended?
Hi all,
I just got
+ PROJECT_ID=64711
+ set +x
Error GETing https://salsa.debian.org/api/v4/user (HTTP 401):
Unauthorized {"error":"invalid_token","error_description":"Toke... at
/usr/share/perl5/Devscripts/Salsa/update_repo.pm line 114.
W: Could not initiate CI for long-read-assembler, please initiate
cd /home/steffen/Med/lra/long-read-assembler && salsa update_repo
--ci-config-path debian/salsa-ci.yml med-team/long-read-assembler
which I expected for those teams for which I do not have the
permissions, but not for the med-team. Is this something to talk about
with the salsa admins?
On a sidenote - https://salsa.debian.org/med-team/long-read-assembler is
for academics only. I thought we wait with an upload to New until the
new queue got a bit shorter.
Best,
Steffen
Nature Methods article on worfklows
Wratten, Laura et al.: Reproducible, scalable, and shareable analysis pipelines with bioinformatics workflow managers Nature Methods. 2021 https://doi.org/10.1038/s41592-021-01254-9 They kindly cited the "Reproducible workflows"-paper. Steffen
Re: Any volunteer to verify autopkgtest of spades?
Hi Tony, On 23.09.21 15:07, Tony Travis wrote: On 23/09/2021 13:53, Andreas Tille wrote: Hi, some time ago I've pushed a packaging update for Spades. Unfortunately the autopkgtest fails with [...] I'm willing to volunteer, but I don't know how to run "autopkgtest": Can I just download your package and test it? You would install both the source tree (with the Debian folder) and install the binary. Then run sh debian/tests/run-unit-test from the source directory. Many thanks! Steffen
Re: Any reason to wait with qiime upgrades? (Was: Bug#994470: please update dependency on openblas)
On 20.09.21 13:34, Andreas Tille wrote: Control: tags -1 pending On Thu, Sep 16, 2021 at 11:54:18AM +0200, Sébastien Villemot wrote: q2-types depends on libopenblas-base, which is a transitional dummy package. Please replace it by libopenblas0. I did this in Git (please leave distribution to UNRELEASED if a package is not uploaded for whatever reason). I'm wondering what might be the reason to wait with upgrading the qiime / q2-* packages. That may have happened to me, i.e. it happened to routine-update and i had not corrected it. I think they can all be upgraded/sent to new. And I have changed my mind when it comes to "shall we first complete what we have in bullseye". Let's just go for the latest version and see how it goes. Best, Steffen
Re: Is pigx-rnaseq ready to be uploaded?
On 15.09.21 23:27, Nilesh Patra wrote: Hi Steffen, On 9/15/21 5:31 PM, Steffen Möller wrote: I don't know about the software myself, if you could take a look and push what you deem sensible, that'd be very cool. Done. Also eliminated some lintian warnings. All dependencies have been transitioned, so it cowbuilds now. Good to go imho. Cool, salsa CI agrees with you as well. Thanks a lot, uploaded Thank you! Also, I saw that you added in me to uploaders, thanks a lot for doing this, but I admit I've no interest in maintaining pigx-rnaseq. I worked on this package as a part of the team-stuff that I do. Hence, I removed myself from Uploaders field and uploaded. Sorry for sounding much like a politician, but I forgot about this. Anyway, I normally do not do this and should have asked. That said, I'm even motivated to remove myself from uploaders in med-packages which I don't have any interest in, and which has co-maintainers. Let's see. I agree. A package shall stimulate you somehow and not be a mere nuisance. Everything else drags your energies. I just suggest you wait until there is an additional reason to trigger the CI build on salsa, preferably a new upstream release :) Best, Steffen
RFS: r-bioc-stringdb and deps r-cran-gsubfn,r-cran-sqldf
I needed the STRINGdb package locally. Was a bit surprised that we did not have it already in our archive, I must admit. It comes with some interesting dependencies. Steffen
Re: Is pigx-rnaseq ready to be uploaded?
On 13.09.21 20:36, Nilesh Patra wrote: On 9/13/21 11:07 PM, Steffen Möller wrote: On 13.09.21 18:27, Nilesh Patra wrote: ... However, I see that you've added this line in d/ch: "hisat2 - upstream changed default genomic aligner" What would this mean? There's no upstream release for hisat2, as I saw, what needs to be done to get this in then? The package already ships in Debian. This was just that d/changelog is also meant to summarize what is new in the respective release. OK, so this was for summary, and not for any TODO, right? Yes. d/control put it as hisat2|rna-star , which is not perfectly correct, I tend to think, since one would need to adjust the testing for rna-star if I get this right. I don't know about the software myself, if you could take a look and push what you deem sensible, that'd be very cool. Done. Also eliminated some lintian warnings. All dependencies have been transitioned, so it cowbuilds now. Good to go imho. ... Would you object it's removal? Also, @Andreas, what d'you think? r-cran-gprofiler should not make it into the next release. But I do not see a point into removing it from testing. I suggest to file an RC bug against it, so we do not forget about it. But do not immediately ask for its removal. Filed #994212 Thank you! I'll contact the pigx-rnaseq upstream group once this transitioned to testing. They should love this but I also know they are using guix on their side (so am I when using pigx-rnaseq on our cluster). So, my next two personal milestones are * Qiime update/upload (which implies an update of scikit-learn) and the automated testing with a qiime-tutorials package * Singularity as part of my routine workflows on our cluster To link this more to our Covid trigger, https://www.nature.com/articles/s41522-021-00232-5 and other work describe the effect that Covid has on the microbiome. Dunning-Kruger here, any such analysis I do not expect to be much more complicated than what we already show in the tutorials. It may be an interesting experiment to see if we as a community could jointly come up with a meta-analysis of these data sets. The meta^2-experiment is how we interact between ourselves and the outside world to get there. Best, Steffen
Re: apache arrow anyone?
On 23.08.21 14:47, Sascha Steinbiss wrote: Hi Andreas, [...] I'd recomment to move the packaging from satta to one of the interested teams (either science or med or something that is close to that audience). I'd be very happy to do that as I am a maintainer in both med and science. Just did not see the relevance for one of these teams back then when I did the first packaging. Sooo, which one would you recommend? I can move the repo as soon as we decide where to. My use case is not of a scientific nature, so Steffen if yours is not primarily med-focused, do we want to simply use `science-team`? I also think that the science-team's repository is where it should go. Nice! Steffen
apache arrow anyone?
Dear all, You once asked me to point to packages that I would consider of particular importance to be packaged. Got one. Apache Arrow https://arrow.apache.org/faq/ knows how to efficiently handle large tabular data. And, while not in our distribution, it blocks some workflows for Debian Med. Arrow comes with interfaces to all the prominent languages, for the Med-workflows it is typically the Python interface pyarrow that is needed. https://arrow.apache.org/install/ presents a repository for Debian with a binary distribution on https://apache.jfrog.io/artifactory/arrow. Now, ideally, Apache would have contributed to Debian directly instead of providing their own repository. Maybe the right way to approach this would be to team up with the Apache folks. I just do not see any means to us just go and use their binary in the meantime since anything depending on it would not be buildable/testable. I am not using Arrow myself, but I presume just like me you all know some project that should be using it :) Best, Steffen
Debian package with a script auto-running a qiime2-tutorial
Hello, You may recall my positive news about the Qiime2 tutorial that I could run - except for a final step that was affected by version incompatibility of scikit-learn with a binary file that was meant to feed a classifier. To run this far, I needed about all the q2-modules that we have currently in salsa. Hence, it felt natural to use the same commands as a test script for our qiime2 packages. But - how? As part of the qiime2 package or the q2cli? Or get the q2cwl into the distribution and transform the tutorials' workflows into formal workflows? I now created a qiime-tutorial package [1] that offers a script to execute all the commands presented in the "moving pictures" tutorial [2]. The other tutorials I would then add once we have updated the qiime packages to its latest release. This first tutorial downloads about 25MByte of data when executed. That is not too bad, I thought. When I started this, I eventually ran into a failure and had difficulties to identify the script that caused the problem. I hence converted the script to a Makefile, so repeated executions are not performing anything redundant. The problem was caused by the dada2 R package that now reproducibly crashes, but a closer look suggests a missing dist package in demux. It has not crashed before I installed various updates in unstable. But that is exactly why this script could be useful. The repository in [1] is prepared to build as a Debian package. I am still somewhat uncertain about how to best automate the testing. And if the "package" should truly hide in a community/tutorials subgroup I am uncertain about, too. Let this be a start. Best, Steffen [1] https://salsa.debian.org/med-team/community/tutorials/qiime-tutorials [2] https://docs.qiime2.org/2021.4/tutorials/
Re: Pytest-ordering - keeps failing in cowbuilder
On 19.08.21 16:56, Nilesh Patra wrote: On Thu, 19 Aug, 2021, 8:12 pm Steffen Möller, mailto:steffen_moel...@gmx.de>> wrote: On 19.08.21 15:37, Nilesh Patra wrote: > > On 8/19/21 5:52 PM, Steffen Möller wrote: >> Dear all, >> >> This is another small dependency nagging me: >> https://salsa.debian.org/python-team/packages/pytest-ordering <https://salsa.debian.org/python-team/packages/pytest-ordering> >> >> pytest-3 does not find the local module when runnig cowbuilder, but it >> is just fine with gbp or dpkg-buildpackage. Does this ring any bell for you? > FWIW, > > [1]: https://github.com/ftobia/pytest-ordering/issues/66 <https://github.com/ftobia/pytest-ordering/issues/66> > [2]: https://github.com/ftobia/pytest-ordering/issues/32 <https://github.com/ftobia/pytest-ordering/issues/32> > [3]: https://github.com/pytest-dev/pytest-order <https://github.com/pytest-dev/pytest-order> Thank you for finding this. I checked out https://github.com/pytest-dev/pytest-order <https://github.com/pytest-dev/pytest-order> and .. after some toying around ... came to the conclusion that this needs to be a separate package since it is not compatible. On a sidenote - I think about three Python packages were accepted into main this day, all created or sponsored by you. Many thanks! This is also what triggered me in my attempt to address pytest-ordering :) So, for the moment I think we should go for two packages - pytest-ordering and pytest-order. But correct me if I am wrong. tl;dr: Ask upstream and do not package both Both serve the same purpose, it doesn't make sense to package both. You should rather ask upstream to start using a dependency that is maintained instead. Pushing things to the archive that are unmaintained increases workload. pytest-ordering is broken ATM, tests fail and those failures are legit. There are some PRs that should be merged, but that contains a lot of extra work for us as downstream if we patch a lot of code. We have limited time/energy ofcourse, right? So, you are suggesting to shift the efforts towards changing the code that currently uses pytest-ordering. Towards maintenance I think that this end-of-the-line thingy is easier since it will not be updated and I do not need to change anything upstream, I must admit. Also, it is build-time only and mostly of cosmetic value since the tests do all need to pass, anyway. Nothing security-sensitive, I mean. I just checked on conda - they have both, with ordering seeing a few (but not many) more downloads. My immediate reaction is that my time-slot for this has ended and I should do something else, i.e. neither package for now :) But Debian should have both, too, at least for this release. Best, Steffen
Re: Pytest-ordering - keeps failing in cowbuilder
On 19.08.21 15:37, Nilesh Patra wrote: On 8/19/21 5:52 PM, Steffen Möller wrote: Dear all, This is another small dependency nagging me: https://salsa.debian.org/python-team/packages/pytest-ordering pytest-3 does not find the local module when runnig cowbuilder, but it is just fine with gbp or dpkg-buildpackage. Does this ring any bell for you? FWIW, [1]: https://github.com/ftobia/pytest-ordering/issues/66 [2]: https://github.com/ftobia/pytest-ordering/issues/32 [3]: https://github.com/pytest-dev/pytest-order Thank you for finding this. I checked out https://github.com/pytest-dev/pytest-order and .. after some toying around ... came to the conclusion that this needs to be a separate package since it is not compatible. On a sidenote - I think about three Python packages were accepted into main this day, all created or sponsored by you. Many thanks! This is also what triggered me in my attempt to address pytest-ordering :) So, for the moment I think we should go for two packages - pytest-ordering and pytest-order. But correct me if I am wrong. Best, Steffen
Pytest-ordering - keeps failing in cowbuilder
Dear all, This is another small dependency nagging me: https://salsa.debian.org/python-team/packages/pytest-ordering pytest-3 does not find the local module when runnig cowbuilder, but it is just fine with gbp or dpkg-buildpackage. Does this ring any bell for you? Many thanks Steffen
Today on the 17th and previously Re: Debian Med videoconference tomorrow, Monday 2021-08-02 18:00 UTC
Hi all, Andreas is not attending today if I got this right, so I thought I just give a quick reminder. I am not really sure about what will come up, from my side it is * the packages for the qiime2 workflow + how to do the QA for them * transitions - bioconductor (and qiime2) Best, Steffen On 01.08.21 21:28, Andreas Tille wrote: Hi, this is the call for the next video conference of the Debian Med team that are an established means to organise the tasks inside our team. We do these conferences twice per month on every 2th and 17th of a month. Usually it takes us only 15-20min depending what we are talking about and how many people are joining. The next meeting is tomorrow 18:00 UTC https://www.timeanddate.com/worldclock/fixedtime.html?msg=Debian+CoViD-19+Biohackathon+Video+Conference=20210802T18 The meeting is on the Debian Social channel https://jitsi.debian.social/DebianMedCovid19 These video meetings were started in the Debian Med Biohackathon. The topic is what contributors have done in the past period and to coordinate the work until the next meeting. For those who are interested in hot topics we want to tackle, here are some items: - Effort to package tensorflow and other ML software which is used in several applications that are used to fight COVID-19. The precondition bazel to package tensorflow was a direct consequence of a Debian Med hackathon and it was even acknowledged[1] - Package software that is used to fight COVID-19 which we are listing in some spreadsheet[2]. It reaches from small tools up to complex software packages. There should be targets for everybody who wants to join us. - General Debian Med issues not directly connected to COVID-19 - Finally we should discuss how to start the next release cycle since this really the last video conference before the Debian 11 release. Newcomers are always welcome. Lets keep on the great work and see you today Andreas. [1] https://blog.bazel.build/2021/03/04/bazel-debian-packaging.html [2] https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=543782716
Re: q2-diversity tests Re: report on first qiime tutorial with Debian packages
Hi Tony, On 12.08.21 20:34, Tony Travis wrote: On 12/08/2021 19:16, Steffen Möller wrote: -- Ran 153 tests in 557.862s Hi, Steffen. QIIME/QIIME2 are notoriously sensitive to the versions of supporting programs Thank you for pointing this out. I admit to be impressed by qiime2, and or some reason that does not make me worry too much. We have a respective reference result file provided with the tutorials and can thus ensure that a) everything runs and b) we get equivalent results. If there are notable differences between the results then this is a bug - in one of the libraries or in qiime2. so I think it is important to validate your packaging of QIIME2 against the same analysis using the upstream-supported Bioconda package. The results files of the tutorials are offered online. I suggest we implement packages with the code that are contributing to the tutorials and run them. Then everyone can compare if any deviation is critical or interesting. Quite a few packages are outdated in Debian that are used in qiime. So, this likely means that we will be updating them and then be newer than what qiime was using when the tutorials were created. I hence do not strive for binary identity. The overall trends should be the same, though. I've taught QIIME2 using Jupyter Notebooks on Cyverse: https://cyverse-jupyter-qiime2.readthedocs-hosted.com/en/latest/README.html This was well received by the course participants in comparison to when they were just using the CLI. I had a look at https://cyverse.org and it seems very compatible with what Debian is after, but I have not fully grasped it all, yet. Thank you for this pointer (and your work). My plan for "after the tutorial packages are functional" is to prepare decent descriptions of the packages, i.e. the text in the debian/control files for each package. Should you (or anyone reading this) find(s) the leisure to help out with that, this would be much appreciated. I made a first attempt at https://salsa.debian.org/med-team/q2-diversity/-/blob/master/debian/control which felt o.k.-ish while I was writing it, yesterday, but today reads a bit weird :o) Best, Steffen
Re: report on first qiime tutorial with Debian packages
On 12.08.21 20:12, Steffen Möller wrote: BTW, q2-diversity (Build-+)Depends on q2-emperor which again depends on python3-emperor which are not in the archive. And both do not look ready for upload yet (in interim state) If you make them ready, let me know. I'll have a look. My immediate questions are answered: Debian can perform the compute and create output files. Since this was running remotely, I admittedly have not yet checked if emperor (which provides an interactive 3D visualisation in your web browser) is truly functional. I had that look, both at emperor and q2-emperor. emperor fails the reprotest https://salsa.debian.org/med-team/emperor/-/jobs/1805122/artifacts/browse/debian/output but that report did not ring any immediate bells. Best, Steffen
q2-diversity tests Re: report on first qiime tutorial with Debian packages
-- Ran 153 tests in 557.862s OK This was nosetests3. 10 minutes. Best, Steffen
report on first qiime tutorial with Debian packages
Hello, TL;DR: Qiime tutorial is directly usable with the packages in salsa for 2020.11.1, no need for anything redundant from Debian. We need to work on JavaScript to get q2-diversity into main. I followed https://docs.qiime2.org/2021.4/tutorials/moving-pictures/ and must admit that I am impressed. This is some nice work. The command lines can be copy'n'pasted directly from that page. What surprised me a bit was that I needed about all the modules we have currently functional - it is not that there is a core functionality in the qiime package and the additional modules are just making it somewhat nicer or so - no. You have phylogeny analyses or you don't. You have diversity anlalyses or you don't. The first trouble I could spot with this very first attempt is with the q2-diversity package. This needs JavaScript, which I (not reading the README) had missed in my first package. This however is huge: npm install --no-save && \ npm run build && \ cp licenses/* dist/ npm WARN deprecated babel-preset-es2015@6.24.1: Thanks for using Babel: we recommend using babel-preset-env now: please read https://babeljs.io/env to update! npm WARN deprecated eslint-loader@1.9.0: This loader has been deprecated. Please use eslint-webpack-plugin npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated npm WARN deprecated chokidar@1.7.0: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies. npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated npm WARN deprecated circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor. npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2. npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead. npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. npm WARN deprecated core-js@2.6.12: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js. added 640 packages, and audited 641 packages in 38s Yes. 640 packages. We likely have quite a few of them in Debian already and can somehow help npm to find them, but - ouch. No idea how you feel about this, but I somehow sense that we should not be doing this without upstream. And preferably, we sync this with JavaScript packaging for conda - otherwise, we would always have versions out of sync. So, what I have now come up with is the invocation via the Makefile that is provided that downloads what is missing. The other concern is that the tutorial provided a classifier in a file format of what was from scikit-learn IIRC that was from a newer version of what we have in the archive. They want to have their users classify that themselves in a future version, so that was not too much of my immediate concern. Best, Steffen
Re: bokeh - run time deps in salsa Re: q2-* - did a few
On 10.08.21 12:28, Nilesh Patra wrote: On 8/10/21 3:49 PM, Steffen Möller wrote: Hello again, I just pushed Pillow to s.d.o/python-team/packages/python-pillow. https://tracker.debian.org/pkg/pillow :) Argh! That was because it provides python3-pil (which makes sense), not python3-pillow. Again in archive, please consider to close ITP and purge repo. Please consider doing an apt-file find/search when you do a package next Good idea. Removal request sent. Have not tried compiling bokeh, yet, though. There are three missing Python packages that we need for testing, still: channels , geckodriver, pytest-html. I guess there are also a few JS dependencies along with it - did you have a chance to check? No, have not looked. But I just found "channels" to likely be https://salsa.debian.org/python-team/packages/python-django-channels. One down. Steffen
bokeh - run time deps in salsa Re: q2-* - did a few
Hello again,
On 08.08.21 21:11, Steffen Möller wrote:
On 08.08.21 14:59, Steffen Möller wrote:
On 07.08.21 23:49, Nilesh Patra wrote:
On 8 August 2021 2:57:46 am IST, "Steffen Möller"
wrote:
On 07.08.21 22:13, Nilesh Patra wrote:
[...]
Do you need some help?
There is a partial dependency of
https://salsa.debian.org/med-team/gneiss (dep of q2-gneiss) on
https://github.com/bokeh/bokeh . The latter seems to produce nice
graphics, no idea how much work that would be to package. Should go to
science rather than med, I suggest. If you could pioneer the packaging
of bokeh then this would be nice.
Bokeh is a beast, and I honestly don't have the time to get it all by
myself. However I can ofcourse hell in the process.
I think Shayan (in CC) was doing something to package Bokeh. @Shayan,
any updates on this?
My suggestion would be to have a section reserved on the spreadsheet
that summarizes all the bokeh dependencies and whatever needs to be done
for it.
bokeh is also essential for the visualization tool
https://github.com/microscopium/microscopium. Seems like we are missing
something.
For a first look at how many dependencies we are missing, I added a
column for bokeh on the spreadsheet ("Others"-tab) on
https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=853602383
and that looks good. A few packages need updates in Debian, but when it
comes to build- and runtime environments, we are mostly fine.
I just pushed Pillow to s.d.o/python-team/packages/python-pillow . Have
not tried compiling bokeh, yet, though. There are three missing Python
packages that we need for testing, still: channels , geckodriver,
pytest-html . My hunch is that these would be not too bad to package and
good for training - ping me for an extracurricular "mentoring of the
week" :)
Steffen
Re: q2-* - did a few
On 08.08.21 14:59, Steffen Möller wrote:
On 07.08.21 23:49, Nilesh Patra wrote:
On 8 August 2021 2:57:46 am IST, "Steffen Möller"
wrote:
On 07.08.21 22:13, Nilesh Patra wrote:
[...]
Do you need some help?
There is a partial dependency of
https://salsa.debian.org/med-team/gneiss (dep of q2-gneiss) on
https://github.com/bokeh/bokeh . The latter seems to produce nice
graphics, no idea how much work that would be to package. Should go to
science rather than med, I suggest. If you could pioneer the packaging
of bokeh then this would be nice.
Bokeh is a beast, and I honestly don't have the time to get it all by
myself. However I can ofcourse hell in the process.
I think Shayan (in CC) was doing something to package Bokeh. @Shayan,
any updates on this?
My suggestion would be to have a section reserved on the spreadsheet
that summarizes all the bokeh dependencies and whatever needs to be done
for it.
bokeh is also essential for the visualization tool
https://github.com/microscopium/microscopium. Seems like we are missing
something.
For a first look at how many dependencies we are missing, I added a
column for bokeh on the spreadsheet ("Others"-tab) on
https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=853602383
and that looks good. A few packages need updates in Debian, but when it
comes to build- and runtime environments, we are mostly fine.
Best,
Steffen
Re: q2-* - did a few
On 07.08.21 23:49, Nilesh Patra wrote: On 8 August 2021 2:57:46 am IST, "Steffen Möller" wrote: On 07.08.21 22:13, Nilesh Patra wrote: [...] Do you need some help? There is a partial dependency of https://salsa.debian.org/med-team/gneiss (dep of q2-gneiss) on https://github.com/bokeh/bokeh . The latter seems to produce nice graphics, no idea how much work that would be to package. Should go to science rather than med, I suggest. If you could pioneer the packaging of bokeh then this would be nice. Bokeh is a beast, and I honestly don't have the time to get it all by myself. However I can ofcourse hell in the process. I think Shayan (in CC) was doing something to package Bokeh. @Shayan, any updates on this? My suggestion would be to have a section reserved on the spreadsheet that summarizes all the bokeh dependencies and whatever needs to be done for it. I do not think that the upload of gneiss should be blocked by that missing dependency, but anyway, this looks good to have and the circular diagrams that gneiss depends on bokeh for (if I understood this right) are indeed a thing in microbiome analyses since no absolute abundances of microbiomes can be measured - only relative, as nicely representable in a circle. So I think we have two options: a) Upload gneiss w/o support of Bokeh for now, and add Bokeh related support/enable Bokeh related tests when we have Bokeh packaged. We have an entire release to get it in b) Only one or two sub parts of gneiss use Bokeh, so For now, translate the code that uses Bokeh to something that we have in the archive. For instance code that uses Bokeh to use matplotlib, or seaborn for dataset visualisations. What do you suggest/think? a. Otherwise, you typically bring packages to another level and I appreciate whatever you find time to address - once the release is out, I suggest. :-) I do not know yet about what qiime2 tutorial (from https://docs.qiime2.org/2021.4/tutorials/ I presume) I want to run on Debian. Suggestions are welcome. The simplest as a start :) I kind of picture a Wiki page that references an established tutorial and then it should be possible to come up with a "qiime2-debian-test" package that runs that code. A side-effect I hope to be that I improve the package descriptions of the q2-* packages. I think it'd be worth asking upstream too about it, right? Yes. What I would very much like to see online inspectable is the output of the tutorial-package test runs. If you happen to have ideas how to get there - I would love this as a "tangible" proof of the Debian package's reliable and as educational material. Q2-taxa needs a bit of cleanup with the JavaScript it is shipping. But otherwise, I tend to think we are done. I believe I made q2-taxa ready for its entry to the archive. What $cleanup does it need exactly? Aargh. I just still had the "D3-scale-chromatic, node-d3-scale-chromatic was my first ever contribution to Debian :-) It was done two years ago thenBy, natural-sort JavaScript dependencies" note in the spreadsheet and apparently missed/forgot about your patches on the mailing list. I had packaged all of these, and q2-taxa is in the archive, and will be a part of bullseye too To hear that this is out of the way is really nice! \o/ These JavaScript packages are both really cumbersome and also something that Debian can be proud of. Best, Steffen
Re: q2-* - did a few
On 07.08.21 22:13, Nilesh Patra wrote: [...] Do you need some help? There is a partial dependency of https://salsa.debian.org/med-team/gneiss (dep of q2-gneiss) on https://github.com/bokeh/bokeh . The latter seems to produce nice graphics, no idea how much work that would be to package. Should go to science rather than med, I suggest. If you could pioneer the packaging of bokeh then this would be nice. I do not think that the upload of gneiss should be blocked by that missing dependency, but anyway, this looks good to have and the circular diagrams that gneiss depends on bokeh for (if I understood this right) are indeed a thing in microbiome analyses since no absolute abundances of microbiomes can be measured - only relative, as nicely representable in a circle. Otherwise, you typically bring packages to another level and I appreciate whatever you find time to address - once the release is out, I suggest. I do not know yet about what qiime2 tutorial (from https://docs.qiime2.org/2021.4/tutorials/ I presume) I want to run on Debian. Suggestions are welcome. The simplest as a start :) I kind of picture a Wiki page that references an established tutorial and then it should be possible to come up with a "qiime2-debian-test" package that runs that code. A side-effect I hope to be that I improve the package descriptions of the q2-* packages. Q2-taxa needs a bit of cleanup with the JavaScript it is shipping. But otherwise, I tend to think we are done. I believe I made q2-taxa ready for its entry to the archive. What $cleanup does it need exactly? Aargh. I just still had the "D3-scale-chromatic, thenBy, natural-sort JavaScript dependencies" note in the spreadsheet and apparently missed/forgot about your patches on the mailing list. To hear that this is out of the way is really nice! Steffen
q2-* - did a few
Hello, The qiime plugins are kind of nagging me. We were kind of on track get complete this prominent microbiome pipeline but seem to have stopped even though most missing dependencies have found entry in the distribution. I had another look at the microbiome tab of the spreadsheet (https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1986383821) and filled a few white spots with new packages on salsa, so they are now yellow spots. I went not for the latest version but for what we have in the archive now, i.e. 2020.11.1. I have not fully thought this through but my hunch was that it may be of value to complete the pipeline in the earlier release that we already have in parts in bullseye. I'll also do the still missing q2-composition and q2-longitudinal and then - I think then it is "tutorial time" for me. Q2-taxa needs a bit of cleanup with the JavaScript it is shipping. But otherwise, I tend to think we are done. Best, Steffen
Bioconductor API Bump to 3.13
Hello, Our distro currently holds $ apt-cache show r-bioc-biocgenerics| grep ^Provides Provides: r-api-bioc-3.12 but about all of BioConductor now needs 3.13. The new r-bioc-biocgenerics package providing 3.13 can be installed without problems, but the then missing 3.12 of that API means that all other reverse dependencies are considered invalid and are uninstalled with an "apt --fix ...". In our videoconference I had suggested to have some fun events like the bulk updating of qiime packages but we should give some extra thoughts on how to deal with the BioConductor updates. Should this possibly be automated? Best, Steffen
RFS r-bioc-biocgenerics - after the release
This provides the new R API Bioc and is deadly missed. My motivation are changes to the inject-into-salsa-git for which I need new packages that all have this as a build-dependency, not the update itself, I must admit :) Best, Steffen
Re: Towards AlphaFold
On 02.08.21 07:28, Nilesh Patra wrote: On 2 August 2021 2:28:13 am IST, "Steffen Möller" wrote: Heya, @Nilesh, concerning extra files for testing I am just fine. Thank you for doing this. Do you think it's good to go for an upload? If yes, I'll sign and dput Yes, this would be nice, please do. Many thanks! Steffen
python-billiard - dep of PyRosetta
Hello, I know, PyRosetta is not free enough to be packaged, but I hope to eventually revive an older project that uses it and would like to run it on Debian. It seems like just two packages are missing - billiard, which I have just addressed on https://salsa.debian.org/python-team/packages/python-billiard seems to be a fine package for parallel computing. I hope to find the opportunity to read up about it and (yeah!) also add a test. But please feel free to beat me to it. :o) Remaining is the https://github.com/dask/dask-jobqueue package and that visualization JavaScript py3Dmol (https://github.com/3dmol/3Dmol.js) , well, yes, would be nice, but I do not see this blocking the building of PyRosetta. Best, Steffen
Re: Towards AlphaFold
Heya, @Nilesh, concerning extra files for testing I am just fine. Thank you for doing this. On 01.08.21 20:39, Andreas Tille wrote: On Sun, Aug 01, 2021 at 10:27:29PM +0530, Nilesh Patra wrote: I was not sure about the repository that should host this package. Med felt wrong, Science maybe, for https://salsa.debian.org/deeplearning-team this did not feel neither deep nor learnish enough, so I went for Python but am not very opinionated about it. Since this definitely is some machine learning tool, deeplearning team is a better fit for this one I admit I would also avoid to spread packages to to many teams. Its really hard to watch those with our QA tools. I would consider the language teams (be it Python, Java, etc.) only the second best option since when packages are maintained there we usually learn about RC bugs via testing removal warnings and info about new upstream versions come not very visibly to our attention if the packages are not in our tasks (which is the case for those not really bio-medical preconditions). I tend to agree. The only itch that I have is that I would like more people to see the close-to-regular-Python-needs ml_collections package. From my understanding all it does is to arrange observations with factors, much like anndata (ok, that is in Med :o) ) . The next packages are from within https://github.com/deepmind and I would like to keep them together, which would now mean within the Deep Learning Team, even when some packages are of mere auxiliary nature. Is that fine with you? Or rather Science? deep learning team sounds best. However you might want to send in an email to -ai to check w/ rest ai team members Fine for me. I agree that the other packages should go to Deep Learning, also because of bazel and other shared dependencies. Steffen
Re: Towards AlphaFold
Hello again, On 01.08.21 16:47, Steffen Möller wrote: Hello, I just prepared a package for https://github.com/google/ml_collections on https://salsa.debian.org/python-team/packages/python-ml-collections . A review with an optional upload to New would be nice. It is nothing too special, just something that is on our way to AlphaFold, as summarised on https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1840067013 I was not sure about the repository that should host this package. Med felt wrong, Science maybe, for https://salsa.debian.org/deeplearning-team this did not feel neither deep nor learnish enough, so I went for Python but am not very opinionated about it. The next packages are from within https://github.com/deepmind and I would like to keep them together, which would now mean within the Deep Learning Team, even when some packages are of mere auxiliary nature. Is that fine with you? Or rather Science? bazel is about everywhere in the other packages. And tensorflow is a dependency already of Jax. No idea what I had expected. A few more easy targets, presumably :) Steffen
Towards AlphaFold
Hello, I just prepared a package for https://github.com/google/ml_collections on https://salsa.debian.org/python-team/packages/python-ml-collections . A review with an optional upload to New would be nice. It is nothing too special, just something that is on our way to AlphaFold, as summarised on https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1840067013 I was not sure about the repository that should host this package. Med felt wrong, Science maybe, for https://salsa.debian.org/deeplearning-team this did not feel neither deep nor learnish enough, so I went for Python but am not very opinionated about it. The next packages are from within https://github.com/deepmind and I would like to keep them together, which would now mean within the Deep Learning Team, even when some packages are of mere auxiliary nature. Is that fine with you? Or rather Science? Best, Steffen
Re: Autopkgtest for Yanagiba and unc-fish
On 31.07.21 14:47, Nilesh Patra wrote: On 31/07/21 11:23 AM, Shruti Sridhar wrote: yanagiba is almost ready, since you downloaded data from ncbi database, can you write a script for fetching data by running it? Just like the way I did it for pilercr[5] I don't have a good idea of navigating this file at the right location Note that doing this is ``optional``, and let me know if it cannot be done I have pushed the get-test-data script. That would be in complete analogy to the createmanpages script (except for the hyphens). I like that! Could you please have a look at it? Thank you! Looks pretty good. The fastq-dump command piped with head was giving me a weird SIGPIPE error so I split it into two operations instead. And the data seemed to have changed (just a little) so I updated this and pushed. Nice! Steffen
Re: Removing r-cran-gprofiler from the archive?
Hi Nilesh, On 28.07.21 21:19, Nilesh Patra wrote: [CC'ing Steffen who maintains gprofiler] Hi, NOTE: This is for action *after* bullseye release gprofiler description in CRAN[1] itself states that gprofiler is unmaintained and new users should use gprofiler2 instead. The only reverse(build) dependency of r-cran-gprofiler is pigx-rnaseq, which has *also* switched to using gprofiler2 in its latest release. Hence, IMO this can be asked for a removal, and we could upload the "new" r-cran-gprofiler2 package. I've committed the packaging of this to salsa already[2] I've three questions: 1) Would anyone have any objections there? As long as either version is in the archive, this is just fine. 2) Should this have a Breaks+Replaces on r-cran-gprofiler? Yes. 3) Since we are pretty close to a release, would it be a better idea to upload it to NEW post release? I do not see why not both could be submitted together. The version that pigx-rnaseq depends on should be in. Many thanks! Steffen [1]: https://cran.r-project.org/web/packages/gProfileR/index.html [2]: https://salsa.debian.org/r-pkg-team/r-cran-gprofiler2 Nilesh
Re: Broadinstitute catch - elegant way to disable tests requiring online access?
On 28.07.21 14:27, Nilesh Patra wrote: Hi Steffen, On 28/07/21 01:06 PM, Steffen Möller wrote: Hi Nilesh, The import of pristine-tar has worked after removing .gitattributes, but then the git lfs references were still in the tarball and the upstream branch. pristine-tar could be pushed, but then the other branches would trigger git lfs errors when pushed to salsa. Only after having all fasta.gz lfs files removed, the upload went smoothly and you all now find this on https://salsa.debian.org/med-team/broad-catch Is that really intended? We would not be able to run tests in this case since you essentially ended up repacking it , since several tests seem to be using that data. BTW, I tried to do a little solution for the lfs thingy, for it to not store "references" and committed it to my personal repository[1] Can you have a look and let me know if it looks sensible? Also, you might as well want to have a look at pristine-lfs[2] which could be interesting to use. I've attempted to use this too, please consider taking a look I admit, I'm not very used to the lfs workflow, so something could be wrong for sure. Not sure why the CI fails though -- probably it does not work fine with lfs, but just thinking of it as a ground for more ideas here :) I can clone it locally in a different space and re-produce the tarball, and I fixed a few tests (not all) -- please consider trying once and let me know [Edit]: all tests pass now on a clean chroot for me You have outperformed yourself on this one. Thank you tons! I just removed the broad-catch repository of mine from salsa, please kindly inject what you have, instead. I am saying that also since apparently my SALSA_TOKEN apparently does not have authority to adjust the CI settings. Pushed to med-team salsa https://salsa.debian.org/med-team/broad-catch May I also ask you to upload the package? I actually wanted to ask a few things before that: 0) Are my changes even correct - is this the way out? This is my first time packaging a lfs based repository No idea if it is _the_ way out - but it was _a_ way out at least that allowed the package to test. 1) When I did the changes, I realised then that the size of repository goes really, really huge $ du -sh broad-catch 785Mbroad-catch I did not anticipate the package to grow _that_ much. Without the lfs data it was just at 22M. (well, a good portion of this is due to both pristine-tar and pristine-lfs branches) So do we really want to go that way? I was initially of the impression that it would be a few megs, but that's quite unfortunately not the case We can alternatively go with what you pushed earlier and disable tests instead 2) If we intend to upload this as is, then we really, _really_ need to either remove installation of all the example fasta files, or we need to do a separate -examples binary package Yes. Or get a solution with getData as a test-dependency. But we would not want to run this with every commit. 3) Would FTP masters be happy with this? Is such a large size OK to go into the archive? I first read this a "zero K" :) The data they ship was last updated in 2018 if I read this right. In that sense it is nothing volatile. If we had our own Debian Med repository then this would all be a no-brainer. We would upload the whole thing or just the data to it. 4) Is the package *that* useful to do that sort of solution for the same? No. And even if. Heck, let the folks install it themselves :) 5) Is it easy to long-term maintain it in the current state? I admit, my connection would not be so reliable to upload all of it in one shot - atleast not today. @Andreas, can you take over the uploading work? And would you also chime into the discussion here? Next steps would be tell routine-update not to try updating this one. I can do that. I think uscan will work fine on this. The next step would be to introduce pristine-lfs support instead, I guess? I need to educate myself on pristine-lfs. And then I would like someone from upstream to comment on this package and direct us a bit on what else would be good to have in Debian to help their cause. My "plan" about that is to just go the official route and introduce the package in a github issue - if it is a regular user of that package replying, I guess we are just about as happy. Yes, that sounds sensible However, I mightn't have a lot of time to spend over this Maybe we should just leave it in salsa for now and wait if there is any demand for that software surfacing on the mailing list. The issue github we can also post with the work residing in salsa. Best, Steffen
Re: ArrayExpressHTS ran into removed fastx-toolkit
On 28.07.21 06:30, Andreas Tille wrote: Hi, On Wed, Jul 28, 2021 at 01:13:38PM +0900, Charles Plessy wrote: I had a quick lookKind regards Andreas. (git grep) at the source code of ArrayExpressHTS and although it has an option to indicate the path to fasta_formatter, it appears to never use it. I patched it out. Many thanks for spotting that! ... and may be report this issue to ArrayExpressHTS upstream? With that boost, I also had a look at patching out tophat but it seems like that protocol is much depending on all the bits and pieces that were nice and shiny when the package was introduced in 2010. We should discuss with upstream what to do about the package. At the moment, the only values I see are a) retrieving data from ArrayExpress for RNA-seq data (judge yourselves on https://www.bioconductor.org/packages/release/bioc/vignettes/ArrayExpressHTS/inst/doc/ArrayExpressHTS.R) which was my motivation to package it b) increasing %age of what we cover from Bioconductor c) increasing %age of what we cover from CRAN Best, Steffen
Re: Broadinstitute catch - elegant way to disable tests requiring online access?
Hi Nilesh, On 27.07.21 23:53, Nilesh Patra wrote: On 28 July 2021 3:00:08 am IST, Nilesh Patra wrote: Hi Steffen, On Tue, 27 Jul 2021 at 23:05, Steffen Möller wrote: Hi Nilesh, Thank you tons for thinking along. It took me a bit but. Too long. The answer is that git does not lose the sensation of a file being a git lfs reference even when you download a tar.gz. For some reason I had expected that all genomes were truly gzipped fasta files, but no, they were still references. Maybe I had inadvertently transformed a few to what they point to during a first build and that is why I then did not find the issue a bit earlier. The import of pristine-tar has worked after removing .gitattributes, but then the git lfs references were still in the tarball and the upstream branch. pristine-tar could be pushed, but then the other branches would trigger git lfs errors when pushed to salsa. Only after having all fasta.gz lfs files removed, the upload went smoothly and you all now find this on https://salsa.debian.org/med-team/broad-catch Is that really intended? We would not be able to run tests in this case since you essentially ended up repacking it , since several tests seem to be using that data. BTW, I tried to do a little solution for the lfs thingy, for it to not store "references" and committed it to my personal repository[1] Can you have a look and let me know if it looks sensible? Also, you might as well want to have a look at pristine-lfs[2] which could be interesting to use. I've attempted to use this too, please consider taking a look I admit, I'm not very used to the lfs workflow, so something could be wrong for sure. Not sure why the CI fails though -- probably it does not work fine with lfs, but just thinking of it as a ground for more ideas here :) I can clone it locally in a different space and re-produce the tarball, and I fixed a few tests (not all) -- please consider trying once and let me know Edit: all tests pass now on a clean chroot for me You have outperformed yourself on this one. Thank you tons! I just removed the broad-catch repository of mine from salsa, please kindly inject what you have, instead. I am saying that also since apparently my SALSA_TOKEN apparently does not have authority to adjust the CI settings. May I also ask you to upload the package? Next steps would be tell routine-update not to try updating this one. I can do that. And then I would like someone from upstream to comment on this package and direct us a bit on what else would be good to have in Debian to help their cause. My "plan" about that is to just go the official route and introduce the package in a github issue - if it is a regular user of that package replying, I guess we are just about as happy. Many thanks again! Steffen
Re: Broadinstitute catch - elegant way to disable tests requiring online access?
Hi Nilesh, Thank you tons for thinking along. On 27.07.21 17:47, Nilesh Patra wrote: On Tue, 27 Jul, 2021, 9:02 pm Steffen Möller, mailto:steffen_moel...@gmx.de>> wrote: Hello, This package is one of the closest that we have on our radar to suit virologists, to I went for CATCH from the broad institute at https://github.com/broadinstitute/catch <https://github.com/broadinstitute/catch> . Conda has it already, does not surface on bio.tools, though. Testing takes quite a while (~15mins) and the few that fail seem to depend on viral sequence data that cannot be downloaded. So, I consider the package functional, but don't really know what to do with the test that fail because they have to fail. Are there suggestions from your sides what to do about them? On another note, when I just wanted to inject it all to salsa, it failed to create the pristine-tar: > [...] That file is small $ du -sh catch/datasets/data/achimota_rubulavirus_1.fasta.gz 8.5K catch/datasets/data/achimota_rubulavirus_1.fasta.gz so there is no need for git-lsf in the first place, right? Yes, no need. It looks weird on first look though Any idea? If you have the sources in a non-git repository could you do something like: $ dpkg-source -b . $ cd .. $ gbp import-dsc --pristine-tar And see what you get? > > Huge test log > Some of these don't work due to a missing .tsv file, others because they try to access internet w/ urllib, few others due to a missing fasta file. But it's difficult to debug till we have a repository in salsa to look stuff up and fix, so if you could commit, it'd be easy to move forward It took me a bit but. Too long. The answer is that git does not lose the sensation of a file being a git lfs reference even when you download a tar.gz. For some reason I had expected that all genomes were truly gzipped fasta files, but no, they were still references. Maybe I had inadvertently transformed a few to what they point to during a first build and that is why I then did not find the issue a bit earlier. The import of pristine-tar has worked after removing .gitattributes, but then the git lfs references were still in the tarball and the upstream branch. pristine-tar could be pushed, but then the other branches would trigger git lfs errors when pushed to salsa. Only after having all fasta.gz lfs files removed, the upload went smoothly and you all now find this on https://salsa.debian.org/med-team/broad-catch . Tests seem to do just as fine as before, still. Best, Steffen
Broadinstitute catch - elegant way to disable tests requiring online access?
Hello,
This package is one of the closest that we have on our radar to suit
virologists, to I went for CATCH from the broad institute at
https://github.com/broadinstitute/catch . Conda has it already, does not
surface on bio.tools, though.
Testing takes quite a while (~15mins) and the few that fail seem to
depend on viral sequence data that cannot be downloaded. So, I consider
the package functional, but don't really know what to do with the test
that fail because they have to fail.
Are there suggestions from your sides what to do about them?
On another note, when I just wanted to inject it all to salsa, it failed
to create the pristine-tar:
steffen@wurzel:~/Med/catch/broad-catch$ gbp import-orig --pristine-tar
../broad-catch_1.4.0.orig.tar.gz
What will be the source package name? [broad-catch]
What is the upstream version? [1.4.0]
gbp:info: Importing '../broad-catch_1.4.0.orig.tar.gz' to branch
'upstream'...
gbp:info: Source package is broad-catch
gbp:info: Upstream version is 1.4.0
gbp:error: Import of ../broad-catch_1.4.0.orig.tar.gz failed: Error
running git add: git-lfs filter-process: 1: git-lfs: not found
fatal: the remote end hung up unexpectedly
gbp:error: Error detected, Will roll back changes.
gbp:info: Rolling back branch 'upstream' by deleting it
gbp:error: Rolled back changes after import error.
And with git-lsf installed this does not look too different as
gbp:info: Importing '../broad-catch_1.4.0.orig.tar.gz' to branch
'upstream'...
gbp:info: Source package is broad-catch
gbp:info: Upstream version is 1.4.0
gbp:error: Import of ../broad-catch_1.4.0.orig.tar.gz failed: Error
running git reset: Downloading
catch/datasets/data/achimota_rubulavirus_1.fasta.gz (5.2 KB)
Error downloading object:
catch/datasets/data/achimota_rubulavirus_1.fasta.gz (895bdf9): Smudge
error: Error downloading
catch/datasets/data/achimota_rubulavirus_1.fasta.gz
(895bdf954b08626bc420f660693aa13fce85523743f67bbcdfbca65821f8b912):
batch request: missing protocol: ""
Errors logged to
/home/steffen/Med/catch/broad-catch/.git/lfs/logs/20210727T172624.851291063.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: catch/datasets/data/achimota_rubulavirus_1.fasta.gz: smudge
filter lfs failed
gbp:error: Error detected, Will roll back changes.
gbp:info: Rolling back branch 'upstream' by deleting it
gbp:info: Rolling back branch 'pristine-tar' by deleting it
gbp:info: Rolling back tag 'upstream/1.4.0' by deleting it
gbp:info: Rolling back branch 'master' by deleting it
gbp:error: Automatic rollback failed [('master', 'branch', 'delete',
None, GitRepositoryError("Can't delete the branch you're on"))]
gbp:error: Clean up manually and please report a bug: [('master',
'branch', 'delete', None, GitRepositoryError("Can't delete the branch
you're on"))]
That file is small
$ du -sh catch/datasets/data/achimota_rubulavirus_1.fasta.gz
8.5K catch/datasets/data/achimota_rubulavirus_1.fasta.gz
so there is no need for git-lsf in the first place, right?
Any idea?
Thanks,
Steffen
==
ERROR: test_n_string
(catch.filter.tests.test_candidate_probes.TestCandidateProbesOnEbolaZaire)
Test that no probes have a string of two or more 'N's.
--
Traceback (most recent call last):
File
"/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/filter/tests/test_candidate_probes.py",
line 144, in setUp
for gnm in seq_io.read_dataset_genomes(zaire_ebolavirus)]
File
"/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/utils/seq_io.py",
line 71, in read_dataset_genomes
seqs = list(read_fasta(fn).values())
File
"/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/utils/seq_io.py",
line 152, in read_fasta
with gzip.open(fn, 'rt') as f:
File "/usr/lib/python3.9/gzip.py", line 58, in open
binary_file = GzipFile(filename, gz_mode, compresslevel)
File "/usr/lib/python3.9/gzip.py", line 173, in __init__
fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory:
'/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/datasets/data/zaire_ebolavirus.fasta.gz'
==
ERROR: test_probe_count
(catch.filter.tests.test_candidate_probes.TestCandidateProbesOnEbolaZaire)
Test that probe counts are in roughly a correct ratio.
--
Traceback (most recent call last):
File
"/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/filter/tests/test_candidate_probes.py",
line 144, in setUp
for gnm in seq_io.read_dataset_genomes(zaire_ebolavirus)]
File
"/home/steffen/Med/catch/catch-1.4.0/.pybuild/cpython3_3.9_catch/build/catch/utils/seq_io.py",
Re: ArrayExpressHTS ran into removed fastx-toolkit
On 27.07.21 12:53, Nilesh Patra wrote: On Tue, 27 Jul, 2021, 4:16 pm Nilesh Patra, mailto:nil...@debian.org>> wrote: and its reverse dependencies like ArrayExpressHTS with it, then? Not needed for reverse dependencies, if we push those to ofiicial bullseye-backports. Erm, correction here. Ofcourse, we will need to push in reverse-deps of fastx-toolkit to fasttrack. But I think in such a case we might end up pushing relatively more useful (chain of) packages in there. But everything depends on if fastx-toolkit is absolutely impossible to have in testing. We do have several softwares in the archive where the upstream is dead. Unless fastx-toolkit is completely not suited for release, we could think about maintaining it ourselves And in my mind, keeping such older software afloat is part of the service that Debian provides to the community. So, let's hope for a path back into testing. Best, Steffen
ArrayExpressHTS ran into removed fastx-toolkit
Hello, fastx-toolkit https://tracker.debian.org/pkg/fastx-toolkit ran into a bug in one of its dependencies and was removed for that reason. ArrayExpressHTS still depends on fasta_formatter from that library. In the meantime, Pierre has fixed that bug on that dependency of fastx-toolkit, so I think this could go back in, right? If we are concerned about taking responsibility for fastx-toolkit since upstream officially declared it as unsupported, then maybe this would be something for https://fasttrack.debian.net/ and its reverse dependencies like ArrayExpressHTS with it, then? But since complicates everything, I would like not go that route. Steffen
Re: q2-* d/watch files updated
On 25.07.21 22:04, Nilesh Patra wrote: Howdy On 7/25/21 10:25 PM, Steffen Möller wrote: Hello, there is a 2021.4.0 version tagged that should be found by all the d/watch files but that was not the case for quite a number of the q2-* packages. I have updated them and gave them all a quick "uscan --verbose" to test. Cool, thanks for doing this. As you might've noticed, a few of these do not build on 32-bit arches due to missing python3-skbio dep there, so I disabled salsa CI for these Nice. I mean, not the missing dep, but .. you know what. Also, there's this package: "q2-vsearch" that you pushed in a commit to, this is not in the archive -- the build fails too as seen in CI too. Is this useful? Should this be fixed and uploaded (to NEW)? I had a look but did not immediately grasp why this failed. In principle, yes, should be fixed and uploaded. How ultimately useful this is I cannot tell, really. My microbiome skills are a bit, but not much, above those of our pets. :-) @Andreas, https://blends.debian.org/med/tasks/bio does not give the "Newer update" note for any of these packages. But it should have for those for which uscan has worked fine, right? This is because github changed its fetch URL from ./archive/... to ./archive/.*/ Similar results could be seen on UDD as well[1] -- you can see several "uscan returned an error: In debian/watch no matching files for watch line https://github.com; there I also sent in a mail about it to mass fix watch files[2], and there was a longish discussion on -devel too about this[3] [1]: https://udd.debian.org/dmd/?debian-med-packaging%40lists.alioth.debian.org#todo [2]: https://lists.debian.org/debian-med/2021/04/msg00025.html [3]: https://lists.debian.org/debian-devel/2021/03/msg00179.html I think it was a mixture of that transition and a copy'n'paste of a dependency on a github.com/.../release/latest URL that does not seem to be preserved. This qiime package and its modules is one of those corners of Debian Med that keep nagging me. The packages are now all a bit dated, looking forward to a working unstable and then backports, really. We just need one regular experiment to run as part of the CI tests and we are good, I tend to think. Best, Steffen
q2-* d/watch files updated
Hello, there is a 2021.4.0 version tagged that should be found by all the d/watch files but that was not the case for quite a number of the q2-* packages. I have updated them and gave them all a quick "uscan --verbose" to test. @Andreas, https://blends.debian.org/med/tasks/bio does not give the "Newer update" note for any of these packages. But it should have for those for which uscan has worked fine, right? Best, Steffen
Re: DeepMind’s AI Advanced Protein Structure Prediction tool Open Sourced
On 19.07.21 09:28, Michael Banck wrote: On Sun, Jul 18, 2021 at 08:47:23PM +0200, Steffen Möller wrote: Following the references in https://www.nature.com/articles/d41586-021-01968-y I found this reference https://github.com/RosettaCommons/RoseTTAFold/tags but I admittedly cannot tell that I'd have fully grasped who is doing what and publishes what software where, yet. https://arstechnica.com/science/2021/07/google-details-its-protein-folding-software-academics-offer-an-alternative/ tried to break it down. Reddit and Michael from Rechenkraft.net pointed me to https://www.nature.com/articles/s41586-021-03828-1 which announces AlphaFold predictions for all human proteins - ready to download from the EBI. In my mind this makes it more, not less, important to get this all into Debian, so people can also start fiddling with the model and not only create new predictions (which they should also do). Best, Steffen
Viral-NGS packages for genomic sequencing
https://github.com/broadinstitute/viral-ngs/blob/master/requirements-conda.txt lists dependencies for https://viral-ngs.readthedocs.io/en/latest/ that I have just transferred into a column on the "Virus" tab of that spreadsheet https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=543782716 We have most of it. But not all. The documentation may also help our discussions. Best, Steffen
Re: Should sambamba be uploaded at this stage?
On 21.07.21 13:06, Pjotr Prins wrote: On Wed, Jul 21, 2021 at 12:23:44PM +0200, Steffen Möller wrote: Yes, please. I know we aim for all architectures, but no one runs sambamba on less that 64 bits, so we have not supported that. all other 64bit platforms would be fine? Like PPC64, mips64el, sparc64, and riscv64? These are just unofficial platforms ([1]https://wiki.debian.org/SupportedArchitectures) but I suggest not to exclude them. Untested, but may work. The BAM format is little-endian, so that may be an issue on some. That would then be my preferred solution. Tell me what I can do upstream. Praise the package on your home page with a badge once your are happy with it :) [3]https://badges.debian.net/badges/debian/unstable/sambamba/version.svg there is a badge already! Best-possible answer. And such a nice one! Fixed in https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=543782716 Thanks! Steffen
Re: Should sambamba be uploaded at this stage?
Hey Pjotr, On 21.07.21 07:26, Pjotr Prins wrote: Hi Nilesh, Sambamba co-author here. On Wed, Jul 21, 2021 at 02:47:17AM +0530, Nilesh Patra wrote: Hi, It came to my notice that sambamba does not list copyright holders, and copyrgiht holders of several files are missing. I wonder if this is a RC bug, since I've seen a few of these earlier, for example #823865 I fixed the copyright stuff, but along with that I did a bunch of other changes: I'll add those fixes upstream. - Repacked shunit2, since it makes no sense to vendor an embedded copy - Added autopkgtests - Minor changed to d/watch, d/salsa-ci.yml - Added d/gbp.conf - Fixed an override All changes on salsa Also, I see here[1] that sambamba builds only on amd64 and arm64 because of missing B-D - so should supported arches in d/control be changed to only these two? Yes, please. I know we aim for all architectures, but no one runs sambamba on less that 64 bits, so we have not supported that. all other 64bit platforms would be fine? Like PPC64, mips64el, sparc64, and riscv64? These are just unofficial platforms (https://wiki.debian.org/SupportedArchitectures) but I suggest not to exclude them. Similar changes in "Architecture" field in d/tests/control I do not have more time to spend with this package, so if @Andreas, @Etienne, someone else could revert/add/edit a few changes and sponsor me an upload, I'd really appreciate that provided this is an RC bug (which I think it is) [1]: https://buildd.debian.org/status/package.php?p=sambamba Nilesh Tell me what I can do upstream. Praise the package on your home page with a badge once your are happy with it :) https://badges.debian.net/badges/debian/unstable/sambamba/version.svg Cheers, Steffen
ncbi-toolbox: building winmaster, need help with assignment to source package
Hello, I think this goes out to Aaron, https://github.com/wdecoster/nano-snakemake/tree/master/bin points to a binary of https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/lxr/source/src/app/winmasker/ which lies next to Cn3D in the source tree, which got me confused enough to ask if this qualifies as a separate tool or if instead this should be built together with ncbi-cn3d as part of the ncbi-tools6 source package? And while I looked through all that, I found igblast at https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/lxr/source/src/app/igblast/ which I previously took from https://ftp.ncbi.nih.gov/blast/executables/igblast/release with loads of redundancies that I had failed to clean up. Are there any directions for me what to take from where? Best, Steffen
Re: DeepMind’s AI Advanced Protein Structure Prediction tool Open Sourced
On 19.07.21 20:03, Andrius Merkys wrote: Hi Nilesh, On 2021-07-19 10:24, Nilesh Patra wrote: On 19 July 2021 12:50:03 pm IST, Andrius Merkys wrote: Currently I am looking into ProMod3 [3], which seems to be the engine behind the great SWISS-MODEL service [4]. I seem to have figured out the dependencies, will go on to packaging next. Let us know if you need help with packaging the chain, in case you need helping hands :-) Thanks a lot for your kind offer! I will let you know as soon as I reach a point of effort parallelization :) I looked at the dependencies for alphaFold and broke them down on https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=1840067013 May I ask you to add arrange the dependencies of SWISS-MODEL/ProMod3 in that sheet, too? Best, Steffen
cuteSV - tests missing, but otherwise already nice
Hello, I gave Andreas' cuteSV package a help2man-based man page and an extra dependency that --help needed but otherwise I think this is ready to be uploaded. I'll see what I can do wrt tests. Best, Steffen
DeepMind’s AI Advanced Protein Structure Prediction tool Open Sourced
Following the references in https://www.nature.com/articles/d41586-021-01968-y I found this reference https://github.com/RosettaCommons/RoseTTAFold/tags but I admittedly cannot tell that I'd have fully grasped who is doing what and publishes what software where, yet. The context: The biochemistry happens in 3D as proteins (and some functional RNA), while all we easily get are 1D DNA sequence data from which one can predict the sequences of amino acids that form the protein. To get from that polymer of amino acids to the 3D structure one typically looks for patterns of sequences that have been observed in structures that have already been determined. Or one try computing that structure de novo, i.e. without a template, and ... wait. Once we have the protein structures one can better understand the effect of mutations, and start other sorts of simulations, like disturbing protein interactions with compounds, i.e. find new drugs. Best, Steffen
NGMLR update - catching division by 0
Hello, I had some fun with https://salsa.debian.org/med-team/ngmlr/ and helped upstream with the VA_ARGS in the preprocessor macros a bit more. This was all triggered by a division by 0 error that affected us locally and was fixed in a user-contributed patch in May (https://github.com/philres/ngmlr/pull/96). Some wording in another github issue had made me believe that the patch was already adopted in some other form, but it wasn't. I have now updated to that latest -dev version which I now wish I had not done. Anyway. The type error reported and fixed by Nilesh no longer surfaces and said patch also interferes with the handling of variable length arguments, so I disabled that one. @Nilesh, I interpret va_start differently and please kindly check with said patch enabled that --version truly prints the version and not the format string with the %s placeholder. Best, Steffen
"Awesome Bioinformatics" list on github
A posting on reddit distracted me and eventually pointed me to https://github.com/danielecook/Awesome-Bioinformatics which I found surprisingly good. I even found two or three packages of my interest that I did not know about before. While we cover most of these packages, we should possibly cover them all if these are not web resources. This is to give our distribution a sensation of "completeness". What I hope to find the time to do over this week is to add a column "awesome" in the spreadsheets and flag the ones that are listed. Best, Steffen
Aw: Re: packages for analyses of DNA structural variations
> Gesendet: Montag, 12. Juli 2021 um 13:18 Uhr > Von: "Nilesh Patra" > An: debian-med@lists.debian.org > Betreff: Re: packages for analyses of DNA structural variations > > On 11/07/21 10:04 PM, Steffen Möller wrote: > > Hello, > > > > I went a bit over packages listed in the "genome structure" tab > > https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=701092179 > > > > and have > > > surpyvor > > Did the following things: > > * Add missing build-depends and depends -- please build in a clean chroot :-) > * Updated d/u/metadata > > However, this depends on the upsetplot package[1], which is not in the > archive -- can you please package this? Sorry, yes. Repeated offender, I admit. > > I can't even run a --help command without this package installed. > > $ grep -irn upsetplot > surpyvor/plots.py:6:from upsetplot import plot as upsetplot > surpyvor/plots.py:50:def upset_plot(upsets, outname="UpSetPlot.png"): > surpyvor/plots.py:51:upsetplot(upsets, sort_by='cardinality') > surpyvor/parse_arguments.py:172: > default="UpSetPlot.png") > > > tandem-genotypes considered worthy for sponsoring. > > > > * Updated to new upstream > * Upstream seems to vendor tests, but it does not seem to vendor those > in the tarball. So for now, I pulled those manually and did a multi-orig > tarball solution for the same. > Maybe we can convince upstream to vendor these in the next release I am in contact with upstream about this. > * Added autopkgtests > * minor fixes in d/control > * Fixed shebang > > I also went ahead and added myself to uploaders, hoping that to be OK > Since I was happy with the package, I uploaded. This is nice to hear. Thank you for making this package shine. > > Please consider enabling Salsa-CI for any packages you work on, I did it > for thesew two for now > and as I said earlier, I also added this in the inject-into-salsa-git > script[2] so this is done by default please consider to pull my changes > from there Ah, this explains it. I had used that script and had considered that settled because of that, but likely my local version was outdated. I'll fix that. > With this, I have 32 packages in new, and I really do hope that FTP > masters get to it soonish. > > Thanks a lot for having worked on these packages, and please consider to > do the needful to get surpyvor in as well > > [1]: https://github.com/jnothman/UpSetPlot > [2]: > https://salsa.debian.org/med-team/community/helper-scripts/-/commit/ae237adad67d6501459706f53eba89992b21e694 We almost got the nano-snakemake pipeline complete. A bit of a bummer is that pbsv is non-free, despite the declared BSD license in the repository. Upstream points to conda as their sole distribution channel - you can read this up in the issue I referenced in my previous email. Need to think a bit more what to do about this. One solution could be to prepare a binary-package that we derive from conda. I do not think that I want that, though. Or we get conda into Debian and execute that as a post-install script of a pbsv-via-conda package. Best, Steffen
packages for analyses of DNA structural variations
Hello, I went a bit over packages listed in the "genome structure" tab https://docs.google.com/spreadsheets/d/1tApLhVqxRZ2VOuMH_aPUgFENQJfbLlB_PFH_Ah_q7hM/edit#gid=701092179 and have surpyvor and tandem-genotypes considered worthy for sponsoring. npInv gives me $ lintian -i ../npinv_1.24+ds-1*changes W: npinv: bad-jar-name usr/share/java/npInv-1.24.jar N: W: bad-jar-name N: N: The package ships the specified "public" Jar file under N: /usr/share/java/, but the name does not correspond to Java policy N: guidelines. This can cause tools in the Debian Java toolchain to fail. N: N: Refer to Debian policy for Java section 2.4 (Java libraries) for N: details. N: N: Severity: warning N: N: Check: languages/java but I have cross-checked that but did not find anything weird except for the capital "I". Ideas are welcome. I also need to extend the package description. Anyone feeling creative - please go for it :) I got stuck with pbmm2 (https://salsa.debian.org/med-team/pbmm2) , which needs a library (C++ API) for minimap2. and pbsv misses the source tree in the repository (reported as https://github.com/PacificBiosciences/pbbioconda/issues/418). These structural variations are of direct clinical interest - not so much for the viral side of the sequencing but for the patient side, e.g. when looking for reasons why some people are more succeptible to develop long covid. Such analyses were more difficult when reads where only short, but with the Nanopore or the PacBio HiFi, longer reads become increasingly available. Many thanks and kind regards, Steffen
Aw: ASTM Lab equipment protocol
Hi Rob, > Which Debian packages support the ASTM lab equipment (over TCP) > protocol? An overview would be nice. Nothing in Debian, and nothing anywhere, so it felt. Found a Python package implementing the ASTM standard from 2013, which we could (help you to) maintain in Debian. But otherwise, there was not too much Free software available from what a quick web search offered. What would you like to see? Best, Steffen
Re: mcaller - may be ready
Hi, > Gesendet: Montag, 21. Juni 2021 um 16:38 Uhr > Von: "Andreas Tille" > An: debian-med@lists.debian.org > Betreff: Re: mcaller - may be ready > > Hi, > > On Mon, Jun 21, 2021 at 07:40:47PM +0530, Nilesh Patra wrote: > > > >override_dh_auto_test: > > > > cp -a testdata testdata_save > > > > dh_auto_test > > > > rm -rf testdata > > > > mv testdata_save testdata > > > > > > > > or something like this. While I agree that autopkgtest is more > > > > important > > > > than build time test both falvours make sense. > > > Build time tests are very important for apriori (pre-upload) detection > > > of breakages introduced by updates/modifications of reverse > > > dependencies. ratt loses much of its power without these tests. > > > > OK, added. > > Good! > > > But you might want to check out ruby-team/meta[1] script, which also checks > > for > > autopkgtest breakeages. Admittedly, I've always liked this more. > > Here's the results for this, if you'd like to have a look just as an example > > > > [1]: https://salsa.debian.org/ruby-team/meta > > [2]: https://lists.debian.org/debian-med/2020/07/msg00160.html > > That's all perfect. However, we just have seen that not all team members > are even building in a chroot. My guess is that not everybody is running > autopkgtest before uploading. Thus just running the build time test if > available can at least ensure proper builds. > > > On Mon, 21 Jun 2021 at 10:46, Andreas Tille wrote: > > > H, but this looks like an attempt to access some remote location. > > > > Works now > > https://salsa.debian.org/med-team/mcaller/-/jobs/1716727 > > Nice. > > > > I'm not sure. The *.pkl files do not really look "binary" but rather > > > badly saved UTF-8 code. But I agree that there is a slight chance that > > > ftpmaster will stumble upon it. Thus clarifying how that files were > > > created (=can be changed) makes sense. > > > > Looks like it has been trained with passing the --train argument to > > mCaller.py > > Do you think asking upstream about it, and put the way to reproduce that > > file in d/README.Source can > > help there? > > Whatever might be faster out of "asking upstream" or running with --train > argument and make a diff. If the latter works its IMHO relatively safe > to assume that this method was used and mentioning this in d/README.Source > as "it was verified that the data can be reprocuded by" should do the > trick. I like it. Well done, all. In general I would not expect a binary identity in a stochastic learning process. But, whatever works for now. Steffen
Re: mcaller - may be ready
Am 20.06.2021 um 21:32 schrieb Andreas Tille:
Hi Steffen,
thanks for working on this package.
On Sun, Jun 20, 2021 at 01:22:56PM +0200, Steffen Möller wrote:
Peer review and (if not too bad) upload would be appreciated.
dh_auto_build
help2man ./mCaller.py --version-string=`head -n1 debian/changelog |cut -f1 -d- |
cut -f2 -d'(' ` --no-info > debian/mCaller.1
help2man: can't get `--help' info from ./mCaller.py
My guess is it works on your local machine and I'd like to repeat my
plea to build in a chroot at least once before asking for review.
Right. I had only updated the build environment but not actually run it :o/
Sorry and thanks! I saw a bunch of nice changes by Nilesh, and was also
happy when I saw that Nilesh had prepared the libmmmulti library last
year, you did libflathashmap, so I gave seqwish another look.
routine-update was busy with a version bump but somehow I did not take
the first (should be easy) hurdle of a pthread link error. Another day.
There is a submodule "gzip_reader" that we have not yet packaged, but I
would like to see how far we get without it.
These packages are from the "virus" tab. We truly need to finish some
more columns.
It does not help (or it does since this does not seem to have many
dependencies) that by reading my notes to the right of that tab, I was
pointed to https://github.com/ncbi/vadr. This seems very interesting and
still timely to have, so I added it as one of our workflows. If I get
this right then you use it to submit viral genome sequences and this
tool tells you what they are and is special about them. I had a quick
look at the documentation and this depends on the typical blast, hmmer
and cmbuild (== infernal) tools - we have all that.
Steffen
Meta: We should think more about the Windows Subsystem for Linux
Hello, What can I say, you hear stories about WSL being fun to work with. Still deprived of my Mac (VDH002 error), I got myself a nicely refurbished X230 on ebay that happened to have Win10 already on it. In short: I installed Debian as an app. And it was (mostly) as simple as that. Transition from buster to bullseye was fine, google fixed one little hickup that I would not have had with WSL2. You have immediate access to all files on Windows. And all of Debian Med with it. Where I see this to be particularly useful is when you have Windows machines that generate various sorts of data that you then want post-processed in some way. And, ahem, it is one less reason to use conda on Windows. Also, the academic institution feeding me has Windows in its student compute labs - maybe WSL would be a possibility to sneak in. Anyway, there is something. And yes, we are too quiet about it, even if it feels much like MS is embracing us again, but it is better than what we have. And I wish we would have the same for the mac. Well, we do, it is called conda/brew/guix, but I mean without a recompilation. Once we have identified a series of core workflows that Debian Med supports, I suggest to have these tested in routine also with WSL. Best, Steffen
