Re: [Nix-dev] Using R with Bioconductor
Quoting Jeffrey David Johnson (2015-06-18 06:20:57) Wonderful, thanks for pushing it forward! I did some more on it but then got sidetracked by grad school and kind of dropped the ball. There is one more thing though... After splitting CRAN and BIOC packages up, I discovered that many of the BIOC ones require other Annotation or Experiment data packages. So I added those too. They're hosted on the same bioconductor servers but under different subdirectories. I added them as separate files because it was easier with my existing code, but hoped to integrate them into the same BIOC file and update the broken packages list before doing a pull request. Then I didn't get around to it. This is a working implementation (just missing the updated broken list) that shows the new subdirectories in default.nix around line 50: https://github.com/jefdaj/nixpkgs/tree/master/pkgs/development/r-modules Any chance that could make it into the release too? Jeff Hi Jeffrey, This really looks nice, I would love to see this in 15.06. I see 3 TODO items here [1] I we want to remove them. Is there anythigs left on you todo list? And maybe R language should get a short section in the manual [2] since there is no mention of language support of R. Instructions how to update the manual are here [3]. [1] https://github.com/NixOS/nixpkgs/compare/master...jefdaj:master#diff-789598f937b7fe2f8e737d7e3f4e206dR3 [2] http://nixos.org/nixpkgs/manual/#chap-language-support [3] https://nixos.org/wiki/Contributing_to_Nix_documentation -- Rok Garbas - http://www.garbas.si signature.asc Description: signature ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Nope just that really, but it could use a little more explanation: when
downloading the new packages I found them to be unreliable. That is
each time some of the curl calls would fail but it would be different
ones. I think that might have to do with my heavily refactored
generate-r-packages.R, or with my university internet. It works slower
but more reliably from my laptop off campus. Best guess is that
either I triggered some kind of limit on concurrent or daily downloads
from bioconductor.statistik.tu-dortmund.de, or I was downloading faster
than my hard drive could write (such a first world problem!). I
partially fixed it by running the script repeatedly.
I'd also like to write something short in the manual/wiki.
Embarassingly I haven't figured out how to write/build my own R
packages using buildRPackage yet though. It never came up because my
current project is a mess of R + python + bash code glued together with
makefiles. But any of the pre-hashed R packages can be installed or used
in a shell like you'd expect:
with import nixpkgs {};
stdenv.mkDerivation {
buildInputs = with rPackages; [ package1 package2 ... ];
}
Jeff
On Thu, 18 Jun 2015 14:04:41 +0200
Rok Garbas r...@garbas.si wrote:
Quoting Jeffrey David Johnson (2015-06-18 06:20:57)
Wonderful, thanks for pushing it forward! I did some more on it but
then got sidetracked by grad school and kind of dropped the ball. There
is one more thing though...
After splitting CRAN and BIOC packages up, I discovered that
many of the BIOC ones require other Annotation or Experiment data
packages. So I added those too. They're hosted on the same bioconductor
servers but under different subdirectories. I added them as separate
files because it was easier with my existing code, but hoped to
integrate them into the same BIOC file and update the broken packages
list before doing a pull request. Then I didn't get around to it.
This is a working implementation (just missing the updated broken
list) that shows the new subdirectories in default.nix around line 50:
https://github.com/jefdaj/nixpkgs/tree/master/pkgs/development/r-modules
Any chance that could make it into the release too?
Jeff
Hi Jeffrey,
This really looks nice, I would love to see this in 15.06.
I see 3 TODO items here [1] I we want to remove them. Is there anythigs left
on
you todo list?
And maybe R language should get a short section in the manual [2] since there
is no mention of language support of R. Instructions how to update the manual
are here [3].
[1]
https://github.com/NixOS/nixpkgs/compare/master...jefdaj:master#diff-789598f937b7fe2f8e737d7e3f4e206dR3
[2] http://nixos.org/nixpkgs/manual/#chap-language-support
[3] https://nixos.org/wiki/Contributing_to_Nix_documentation
--
Rok Garbas - http://www.garbas.si
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Hi Jeffrey, we now have full Bioconductor support in master. According to http://hydra.cryp.to/jobset/nixpkgs/r-updates, we can build about 7000 packages for R in Nix. Thank you very much for your help! Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Wonderful, thanks for pushing it forward! I did some more on it but then got sidetracked by grad school and kind of dropped the ball. There is one more thing though... After splitting CRAN and BIOC packages up, I discovered that many of the BIOC ones require other Annotation or Experiment data packages. So I added those too. They're hosted on the same bioconductor servers but under different subdirectories. I added them as separate files because it was easier with my existing code, but hoped to integrate them into the same BIOC file and update the broken packages list before doing a pull request. Then I didn't get around to it. This is a working implementation (just missing the updated broken list) that shows the new subdirectories in default.nix around line 50: https://github.com/jefdaj/nixpkgs/tree/master/pkgs/development/r-modules Any chance that could make it into the release too? Jeff On Wed, 17 Jun 2015 23:23:06 +0200 Peter Simons sim...@cryp.to wrote: Hi Jeffrey, we now have full Bioconductor support in master. According to http://hydra.cryp.to/jobset/nixpkgs/r-updates, we can build about 7000 packages for R in Nix. Thank you very much for your help! Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Finally got around to writing it! https://github.com/NixOS/nixpkgs/compare/master...jefdaj:bioc-pullreq-twofiles I'm getting an error when installing a Bioconductor package with CRAN dependency though... is that just me? I think they should be able to find each other since the two sets get unioned. Jeff On Tue, 26 May 2015 09:31:35 -0700 Jeffrey David Johnson jef...@gmail.com wrote: Sure that sounds good. It would make storing the mirror type for each package unneccesary too. I'll see if I can set it up later today so the script takes 'cran' or 'bioc' as an argument and reads/writes cran-packages.nix or bioc-packages.nix, then default.nix can read both sets and map the right urls onto them before combining. Jeff On Tue, 26 May 2015 10:35:21 +0200 Peter Simons sim...@cryp.to wrote: Hi Jeffrey, https://github.com/NixOS/nixpkgs/compare/master...jefdaj:bioc-pullreq this looks good to me. Thank you very much for figuring this out! The only thing I'd prefer is to have packages from CRAN and Bioconductor in two separate files so that it's possible to re-generate them separately. Both files should be imported into one attribute set called rPackages, though, that's clear. There might be conflicts between the Bioconductor and CRAN release schedules. I don't believe there's any release schedule. People just upload new packages to the repository whenever they've finished a new version. I doubt that's a coordinated effort. Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Hi Jeffrey, https://github.com/NixOS/nixpkgs/compare/master...jefdaj:bioc-pullreq this looks good to me. Thank you very much for figuring this out! The only thing I'd prefer is to have packages from CRAN and Bioconductor in two separate files so that it's possible to re-generate them separately. Both files should be imported into one attribute set called rPackages, though, that's clear. There might be conflicts between the Bioconductor and CRAN release schedules. I don't believe there's any release schedule. People just upload new packages to the repository whenever they've finished a new version. I doubt that's a coordinated effort. Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
Sure that sounds good. It would make storing the mirror type for each package unneccesary too. I'll see if I can set it up later today so the script takes 'cran' or 'bioc' as an argument and reads/writes cran-packages.nix or bioc-packages.nix, then default.nix can read both sets and map the right urls onto them before combining. Jeff On Tue, 26 May 2015 10:35:21 +0200 Peter Simons sim...@cryp.to wrote: Hi Jeffrey, https://github.com/NixOS/nixpkgs/compare/master...jefdaj:bioc-pullreq this looks good to me. Thank you very much for figuring this out! The only thing I'd prefer is to have packages from CRAN and Bioconductor in two separate files so that it's possible to re-generate them separately. Both files should be imported into one attribute set called rPackages, though, that's clear. There might be conflicts between the Bioconductor and CRAN release schedules. I don't believe there's any release schedule. People just upload new packages to the repository whenever they've finished a new version. I doubt that's a coordinated effort. Best regards, Peter ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] Using R with Bioconductor
OK, turns out it's not so bad! We can skip biocLite and install packages directly, same as from CRAN. Draft changes here: https://github.com/NixOS/nixpkgs/compare/master...jefdaj:bioc-pullreq Opinions? I considered splitting off a separate bioc-modules folder, but then I noticed the majority of broken R packages are just missing Bioconductor dependencies and decided the two aren't really separate. Plus it would be a lot of duplicated code, some of which I don't understand well enough to maintain. There might be conflicts between the Bioconductor and CRAN release schedules. I think a unified set of packages could require holding back from upgrading to the latest CRAN releases on occasion, so if that's a big problem maybe we do need two sets of packages (current CRAN, previous CRAN + current Bioconductor). Anyone familiar enough with Bioconductor to say for sure? Should I go ahead and make a pull request? I've attached the bash script I made to list remaining dependencies. After I ran it I had to go through and manually correct some things it doesn't pick up, like packages that exist but are known not to build. Jeff On Sun, 8 Mar 2015 12:00:27 -0700 Jeffrey David Johnson jef...@gmail.com wrote: Do any of you guys/gals use R and Bioconductor? The only page I can find related to setting it up is https://nixos.org/wiki/R. Bioconductor is a whole additional ecosystem on top of R/CRAN, with its own installer. You normally use it like: source(http://bioconductor.org/biocLite.R;) biocLite(PackageName) They explain why at http://bioconductor.org/install/#why-biocLite. How hard would it be to package that using Nix? Thanks Jeff list-still-broken.sh Description: Binary data ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
[Nix-dev] Using R with Bioconductor
Do any of you guys/gals use R and Bioconductor? The only page I can find related to setting it up is https://nixos.org/wiki/R. Bioconductor is a whole additional ecosystem on top of R/CRAN, with its own installer. You normally use it like: source(http://bioconductor.org/biocLite.R;) biocLite(PackageName) They explain why at http://bioconductor.org/install/#why-biocLite. How hard would it be to package that using Nix? Thanks Jeff ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
