Re: https://github.com/nf-core/covid19 - nextflow pipeline
On Sun, Apr 12, 2020 at 8:22 AM Jun Aruga wrote: > > > https://github.com/nf-core/covid19 > > This URL is 404 not found right now. > What happened? I talked with the project's people on nf-core Slack channel #covid19. They deleted the project intentionally. Instead of that, they are developing nf-core/arctic and nf-core/viralrecon pipelines. Here are the projects for covid19 analysis. https://github.com/nf-core/nanoseq (it looks stable) https://github.com/nf-core/artic (in development) https://github.com/nf-core/viralrecon (in development) -- Jun | He - His - Him
Re: https://github.com/nf-core/covid19 - nextflow pipeline
> > https://github.com/nanoporetech/ - the nanopore is a prime tool for > > covid-19 > > Hmmm, that github directory lists several tools but none with this > description neither the name nanopore. I've found a software named > nanopore on Github here: Hi Andreas, "nanopore" in "the nanopore is a prime tool for covid-19" possibly means the way or devices or tools to sequence DNA by the company: Oxford Nanopore technology [1] 's sequencing device such as MiniON, GridION, PromethION [1]. It is a sequencing way to detect base pairs (A, T, G, C) of DNA by passing it to the device's protein nano (small) pore (hole). As it is a sequencing of long reads, and the specific tools for the long reads are used. It's not like a sequencing of short reads for example for Illumina sequencers. [1] https://nanoporetech.com/ -- Jun | He - His - Him
Re: https://github.com/nf-core/covid19 - nextflow pipeline
> https://github.com/lh3/minimap Mistake. Here is the right tool. https://github.com/lh3/minimap2 -- Jun | He - His - Him
Re: https://github.com/nf-core/covid19 - nextflow pipeline
> The Nanopore tools evolve fast, and most of them are not under a Free > license, but the following might be useful: I am not sure how the tools are used with Nanopore sequencing in the Nextflow pipeline, how about just using alternative tools under a free license? Such as https://github.com/ocxtal/minialign https://github.com/jts/nanopolish https://github.com/lh3/minimap https://github.com/lh3/miniasm -- Jun | He - His - Him
Re: https://github.com/nf-core/covid19 - nextflow pipeline
Hi Andreas, Le Wed, Mar 25, 2020 at 11:36:27AM +0100, Andreas Tille a écrit : > > > > https://github.com/nanoporetech/ - the nanopore is a prime tool for > > covid-19 > > Hmmm, that github directory lists several tools but none with this > description neither the name nanopore. I've found a software named > nanopore on Github here: The Nanopore tools evolve fast, and most of them are not under a Free license, but the following might be useful: To convert old formats to new ones: https://github.com/nanoporetech/ont_fast5_api To demultiplex "barcoded" sequence reads: https://github.com/nanoporetech/qcat > > Also, this > > runs into the missing bits of the popular r-cran-locfit package. > > Charles, are you reading here? We really need to push this issue > forward. I thought that the problem was solved since the package landed in unstable and testing ? https://packages.debian.org/sid/r-cran-locfit Have a nice Sunday, Charles -- Charles Plessy Akano, Uruma, Okinawa, Japan Debian Med packaging team http://www.debian.org/devel/debian-med Tooting from work, https://mastodon.technology/@charles_plessy Tooting from home, https://framapiaf.org/@charles_plessy
Re: https://github.com/nf-core/covid19 - nextflow pipeline
> https://github.com/nf-core/covid19 This URL is 404 not found right now. What happened? -- Jun | He - His - Him
Re: https://github.com/nf-core/covid19 - nextflow pipeline
On Tue, 2020-03-24 at 23:07 +0100, Steffen Möller wrote: > In the new queue is > > vienna-rna - and we are talking about a virus with single RNA strand > as > a genome, also a dependency of a dependency for bcbio > dnapi - another RNA-one, dependency for bcbio > python-numpy-groupies - forgot what this was for - some dependency > for FYI numpy-groupies is a dependency for loompy pypi doesn't know of any other dependencies using numpy-groupies right now https://libraries.io/pypi/numpy-groupies/dependents
Re: https://github.com/nf-core/covid19 - nextflow pipeline
On 25.03.20 11:36, Andreas Tille wrote: [@Charles, please read below] Hi Steffen, On Tue, Mar 24, 2020 at 11:07:38PM +0100, Steffen Möller wrote: Wrt package lists I know you need this for the blends structure. Well, not for the structure. We need to mention packaging tasks and links to useful software could be put on our todo list. The tool I currently prefer is the blends structure (until we have something more practical). A single package is neat to see packaged, but we need to complete a workflow and test that workflow intensively. I suggest we agree on a set of workflows we want to bring to Debian and then we do that. Non-free, experimental, preferably main - but Debian, in a way that official images of docker/amazon can find and install it. I don't have these lists, What I have is a small list of lists: https://github.com/nanoporetech/ - the nanopore is a prime tool for covid-19 Hmmm, that github directory lists several tools but none with this description neither the name nanopore. I've found a software named nanopore on Github here: https://github.com/mitenjain/nanopore Which one do you mean? The is the official repository of the creator of the Nanopore. I want them all, but we should start with the easy ones and the ones and the ones required from workflows the we decide to bring to Debian. I just looked at is https://github.com/nanoporetech/qcat/blob/master/Dockerfile . This starts with an Ubuntu image and compiles the package on the fly. Conda is not mentioned in that repository but is has it https://anaconda.org/bioconda/qcat . My preferred package name would be nanoporetech-qcat . https://github.com/nf-core That's also a directory. I've got a previous hint to https://salsa.debian.org/med-team/chip-seq which is close to ready for uploading (should be listed on the covid-19 task soon). What else besided chip-seq would be most relevant. I suggest we take the workflow of interest, say https://github.com/nf-core/epitopeprediction/blob/master/environment.yml and look at the packages loaded from Conda. For the epitope prediction this is - python=2.7.15 #dont upgrade, FRED2 needs python 2.7 for now - multiqc=1.6 #cant upgrade due to Python2 fix - snpsift=4.3.1t - csvtk=0.19.1 - fred2=2.0.5 - mhcflurry=1.4.3 - mhcnuggets=2.3.2 - conda-forge::r-markdown=1.1 Seems like we need to give FRED2 a bit of a technical push. And that is exactly how Debian is helping. I am not sure about the amont of energy I want to continue to invest into bcbio at the very moment, I must admit. https://salsa.debian.org/med-team/bcbio/-/blob/master/debian/TODO I'll re-check your list. Postpone that until the package is found relevant also for another workflow. and there is another RNAseq library that is _almost_ there, i.e. we have all the packages, but https://salsa.debian.org/med-team/pigx-rnaseq/-/blob/master/debian/TODO describes issues about getting the css files into the R packages to have nicer reports, without which the tests of pigx-rnaseq fail. I can have a look. It needs multiqc which in turn has a non-free dependency[1]. I might try to follow the hint that we simply drop Highcharts and document in README.Debian that the functionality is restricted. This might help the dependencies of multiqc. What do you think about this? I have kept referencing the online repositories of the JavaScript library, i.e. I introduced a privacy violation. The package may then go to non-free or so but I am fine with that - biology first. For me. And bcbio already is in contrib because of a dependency on vienna-rna (which is one of those "almost free" licenses). Also, this runs into the missing bits of the popular r-cran-locfit package. Charles, are you reading here? We really need to push this issue forward. This happens in deseq2. [1] https://github.com/ewels/MultiQC/issues/800 It is not on me to decide what workflow should be followed up. I suggest to collect a series of options and the community should somehow distribute the work. Steffen
Re: https://github.com/nf-core/covid19 - nextflow pipeline
[@Charles, please read below] Hi Steffen, On Tue, Mar 24, 2020 at 11:07:38PM +0100, Steffen Möller wrote: > Wrt package lists I know you need this for the blends structure. Well, not for the structure. We need to mention packaging tasks and links to useful software could be put on our todo list. The tool I currently prefer is the blends structure (until we have something more practical). > I don't > have these lists, What I have is a small list of lists: > > https://github.com/nanoporetech/ - the nanopore is a prime tool for > covid-19 Hmmm, that github directory lists several tools but none with this description neither the name nanopore. I've found a software named nanopore on Github here: https://github.com/mitenjain/nanopore Which one do you mean? > https://github.com/nf-core That's also a directory. I've got a previous hint to https://salsa.debian.org/med-team/chip-seq which is close to ready for uploading (should be listed on the covid-19 task soon). What else besided chip-seq would be most relevant. > I am not sure about the amont of energy I want to continue to invest > into bcbio at the very moment, I must admit. > > https://salsa.debian.org/med-team/bcbio/-/blob/master/debian/TODO I'll re-check your list. > and there is another RNAseq library that is _almost_ there, i.e. we have > all the packages, but > https://salsa.debian.org/med-team/pigx-rnaseq/-/blob/master/debian/TODO > describes issues about getting the css files into the R packages to have > nicer reports, without which the tests of pigx-rnaseq fail. I can have a look. It needs multiqc which in turn has a non-free dependency[1]. I might try to follow the hint that we simply drop Highcharts and document in README.Debian that the functionality is restricted. This might help the dependencies of multiqc. What do you think about this? > Also, this > runs into the missing bits of the popular r-cran-locfit package. Charles, are you reading here? We really need to push this issue forward. > In the new queue is > > vienna-rna - and we are talking about a virus with single RNA strand as > a genome, also a dependency of a dependency for bcbio > dnapi - another RNA-one, dependency for bcbio > python-numpy-groupies - forgot what this was for - some dependency for > something in nf-core? > minimap2 - reupload of something already in Debian, added a -dev package > as requested by nanopolish - for the nanopore I'll send you a private mail about this. > I must admit that I thought there were more, still. Some have been > rejected that were kindly adopted by Andreas, others have been rejected > and I have not reuploaded. Again, make a list. I'll re-check. > pizzly (another indirect bcbio dependency) > was accepted - very nice, many thanks, missed that one. :-) > The acceptance of vienna-rna or minimap2 will trigger other uploads. The > first nim-packages have recently been accepted - will have a look into > uploading what we have prepared on salsa.debian.org/nim-team. Let me know if you need help. I'd currently lower my constant bug hunting effort (and would be **really** happy if anybody could at least squash a few here) and will prefer COVID-19 relevant packages over it. Kind regards Andreas. [1] https://github.com/ewels/MultiQC/issues/800 -- http://fam-tille.de
Re: https://github.com/nf-core/covid19 - nextflow pipeline
Hi Andreas, Wrt package lists I know you need this for the blends structure. I don't have these lists, What I have is a small list of lists: https://github.com/nanoporetech/ - the nanopore is a prime tool for covid-19 https://github.com/nf-core I am not sure about the amont of energy I want to continue to invest into bcbio at the very moment, I must admit. https://salsa.debian.org/med-team/bcbio/-/blob/master/debian/TODO and there is another RNAseq library that is _almost_ there, i.e. we have all the packages, but https://salsa.debian.org/med-team/pigx-rnaseq/-/blob/master/debian/TODO describes issues about getting the css files into the R packages to have nicer reports, without which the tests of pigx-rnaseq fail. Also, this runs into the missing bits of the popular r-cran-locfit package. In the new queue is vienna-rna - and we are talking about a virus with single RNA strand as a genome, also a dependency of a dependency for bcbio dnapi - another RNA-one, dependency for bcbio python-numpy-groupies - forgot what this was for - some dependency for something in nf-core? minimap2 - reupload of something already in Debian, added a -dev package as requested by nanopolish - for the nanopore I must admit that I thought there were more, still. Some have been rejected that were kindly adopted by Andreas, others have been rejected and I have not reuploaded. pizzly (another indirect bcbio dependency) was accepted - very nice, many thanks, missed that one. The acceptance of vienna-rna or minimap2 will trigger other uploads. The first nim-packages have recently been accepted - will have a look into uploading what we have prepared on salsa.debian.org/nim-team. Steffen On 24.03.20 17:12, Andreas Tille wrote: Hi Steffen, thanks a lot for your clarification. Can we expect a list of links pointing to packages that are needed? As I wrote in my other mail ftpmaster seems to be motivated. Currently its us who need to deliver. Kind regards Andreas. On Tue, Mar 24, 2020 at 03:37:22PM +0100, Steffen Möller wrote: Hi Tony, On 23.03.20 17:04, Tony Travis wrote: On 23/03/2020 14:38, Steffen Möller wrote: Dear all, With the enormous difficulties to get bcbio into our distribution (a considderable list of sequential dependencies for nim, java libraries, golang-bio, sigh) I also had an eye on nf-core and their nextflow-based workflows. I think I uploaded the one or other package from their dependencies but, well, treated it more like an inspiration than a project. From what I oversee it is far easier to address than bcbio. [...] Hi, Steffen. I've just had a look at the "nf-core" repo and read this: "please only use Conda as a last resort" ;) You read something else elsewhere. They recommend running "nf-core" under Docker or Singularity, which I believe would make it unsuitable for packaging as part of "med-bio". No, not at all. We get the packages into our distribution and there will be an additional recommendation. We just need to show that our packages are functional and they have done a nice job with their collection of test data. I (once upon a time) had an email exchange with upstream - they are very Debian friendly. And so are the folks behind conda and bcbio. I'm currently running "bcbio_nextgen" under Ubuntu-MATE 18.04 + med-bio, but the developers advised me NOT to use the 'external' versions of tools like "bwa" or "bowtie2" from Debian-Med but install everything isolated from the 'system'. This avoid confusion. I'd recommend the same. I think this sort of application is more effort than it's worth to package for "med-bio". However, I think that a bare-bones package equivalent to bootstrapping "bioconda" using "miniconda" would be worthwhile to provide a ready-to-use "bioconda" within Debian-Med. I concur. And this will happen. Anybody who is packaging for Debian is very likely to come up with patches for upstream. This is what also makes upstream stronger. The benefit of packaging after aims set by the nf-core community is that we get our act together towards something tangible (a set of proven workflows with test data with relevance for the ongoing pandemia) and to improve the packages upstream. Steffen
Re: https://github.com/nf-core/covid19 - nextflow pipeline
On 24/03/2020 14:37, Steffen Möller wrote: [...] "please only use Conda as a last resort" ;) You read something else elsewhere. Hi, Steffen. No, I read it here: https://github.com/nf-core/covid19#quick-start Quick Start i. Install nextflow ii. Install either Docker or Singularity for full pipeline reproducibility (please only use Conda as a last resort; see docs) That's why I thought it would be difficult to package things for Debian. FWIW, Tony. -- Minke Informatics Limited, Registered in Scotland - Company No. SC419028 Registered Office: 3 Donview, Bridge of Alford, AB33 8QJ, Scotland (UK) tel. +44(0)19755 63548http://minke-informatics.co.uk mob. +44(0)7985 078324mailto:tony.tra...@minke-informatics.co.uk
Re: https://github.com/nf-core/covid19 - nextflow pipeline
Hi Steffen, thanks a lot for your clarification. Can we expect a list of links pointing to packages that are needed? As I wrote in my other mail ftpmaster seems to be motivated. Currently its us who need to deliver. Kind regards Andreas. On Tue, Mar 24, 2020 at 03:37:22PM +0100, Steffen Möller wrote: > Hi Tony, > > On 23.03.20 17:04, Tony Travis wrote: > > On 23/03/2020 14:38, Steffen Möller wrote: > > > Dear all, > > > > > > With the enormous difficulties to get bcbio into our distribution (a > > > considderable list of sequential dependencies for nim, java libraries, > > > golang-bio, sigh) I also had an eye on nf-core and their > > > nextflow-based workflows. I think I uploaded the one or other package > > > from their dependencies but, well, treated it more like an inspiration > > > than a project. From what I oversee it is far easier to address than > > > bcbio. > > > [...] > > > > Hi, Steffen. > > > > I've just had a look at the "nf-core" repo and read this: > > > > "please only use Conda as a last resort" > ;) You read something else elsewhere. > > > > They recommend running "nf-core" under Docker or Singularity, which I > > believe would make it unsuitable for packaging as part of "med-bio". > No, not at all. We get the packages into our distribution and there will > be an additional recommendation. We just need to show that our packages > are functional and they have done a nice job with their collection of > test data. I (once upon a time) had an email exchange with upstream - > they are very Debian friendly. And so are the folks behind conda and bcbio. > > > > I'm currently running "bcbio_nextgen" under Ubuntu-MATE 18.04 + > > med-bio, but the developers advised me NOT to use the 'external' > > versions of tools like "bwa" or "bowtie2" from Debian-Med but install > > everything isolated from the 'system'. > This avoid confusion. I'd recommend the same. > > > > I think this sort of application is more effort than it's worth to > > package for "med-bio". However, I think that a bare-bones package > > equivalent to bootstrapping "bioconda" using "miniconda" would be > > worthwhile to provide a ready-to-use "bioconda" within Debian-Med. > > I concur. And this will happen. > > Anybody who is packaging for Debian is very likely to come up with > patches for upstream. This is what also makes upstream stronger. The > benefit of packaging after aims set by the nf-core community is that we > get our act together towards something tangible (a set of proven > workflows with test data with relevance for the ongoing pandemia) and to > improve the packages upstream. > > Steffen > > > -- http://fam-tille.de
Re: https://github.com/nf-core/covid19 - nextflow pipeline
Hi Tony, On 23.03.20 17:04, Tony Travis wrote: On 23/03/2020 14:38, Steffen Möller wrote: Dear all, With the enormous difficulties to get bcbio into our distribution (a considderable list of sequential dependencies for nim, java libraries, golang-bio, sigh) I also had an eye on nf-core and their nextflow-based workflows. I think I uploaded the one or other package from their dependencies but, well, treated it more like an inspiration than a project. From what I oversee it is far easier to address than bcbio. [...] Hi, Steffen. I've just had a look at the "nf-core" repo and read this: "please only use Conda as a last resort" ;) You read something else elsewhere. They recommend running "nf-core" under Docker or Singularity, which I believe would make it unsuitable for packaging as part of "med-bio". No, not at all. We get the packages into our distribution and there will be an additional recommendation. We just need to show that our packages are functional and they have done a nice job with their collection of test data. I (once upon a time) had an email exchange with upstream - they are very Debian friendly. And so are the folks behind conda and bcbio. I'm currently running "bcbio_nextgen" under Ubuntu-MATE 18.04 + med-bio, but the developers advised me NOT to use the 'external' versions of tools like "bwa" or "bowtie2" from Debian-Med but install everything isolated from the 'system'. This avoid confusion. I'd recommend the same. I think this sort of application is more effort than it's worth to package for "med-bio". However, I think that a bare-bones package equivalent to bootstrapping "bioconda" using "miniconda" would be worthwhile to provide a ready-to-use "bioconda" within Debian-Med. I concur. And this will happen. Anybody who is packaging for Debian is very likely to come up with patches for upstream. This is what also makes upstream stronger. The benefit of packaging after aims set by the nf-core community is that we get our act together towards something tangible (a set of proven workflows with test data with relevance for the ongoing pandemia) and to improve the packages upstream. Steffen
Re: https://github.com/nf-core/covid19 - nextflow pipeline
On 23/03/2020 14:38, Steffen Möller wrote: Dear all, With the enormous difficulties to get bcbio into our distribution (a considderable list of sequential dependencies for nim, java libraries, golang-bio, sigh) I also had an eye on nf-core and their nextflow-based workflows. I think I uploaded the one or other package from their dependencies but, well, treated it more like an inspiration than a project. From what I oversee it is far easier to address than bcbio. [...] Hi, Steffen. I've just had a look at the "nf-core" repo and read this: "please only use Conda as a last resort" They recommend running "nf-core" under Docker or Singularity, which I believe would make it unsuitable for packaging as part of "med-bio". I'm currently running "bcbio_nextgen" under Ubuntu-MATE 18.04 + med-bio, but the developers advised me NOT to use the 'external' versions of tools like "bwa" or "bowtie2" from Debian-Med but install everything isolated from the 'system'. I think this sort of application is more effort than it's worth to package for "med-bio". However, I think that a bare-bones package equivalent to bootstrapping "bioconda" using "miniconda" would be worthwhile to provide a ready-to-use "bioconda" within Debian-Med. Bye, Tony. -- Minke Informatics Limited, Registered in Scotland - Company No. SC419028 Registered Office: 3 Donview, Bridge of Alford, AB33 8QJ, Scotland (UK) tel. +44(0)19755 63548http://minke-informatics.co.uk mob. +44(0)7985 078324mailto:tony.tra...@minke-informatics.co.uk
