Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On 21 August 2007 at 12:19, Dirk Eddelbuettel wrote: | | Hi Thomas, | | On 21 August 2007 at 19:02, Thomas Weber wrote: | | Hi, | | | | Am Dienstag, den 21.08.2007, 11:11 -0500 schrieb Dirk Eddelbuettel: | | E.g. a good example of how it's nice to have the code in one place (as | | opposed to dozens of debian/rules files) is that just recently I learned about | | a neat x11 framebuffer server wrapper, | | | | Which package is that? I might need this wrapper for Octave. | | I was wondering myself ;-) so I untarred the pbuilder image into which I | installed it. The script is called 'xvfb-run' and comes with the xvfb | packages, as I recall I also needed to install xfonts-base to make xvfb | useable. Addendum: An additional package you'd need is xbase-clients for xauth. Dirk -- Three out of two people have difficulties with fractions. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Sun, 2007-08-19 at 02:10 +0200, Steffen Moeller wrote:
On Saturday 18 August 2007 22:45:56 Soeren Sonnenburg wrote:
On Sat, 2007-08-18 at 14:44 +0200, Steffen Moeller wrote:
On Saturday 18 August 2007 12:36:41 Soeren Sonnenburg wrote:
[...]
Well I don't know how much should be split up. But I guess this depends
also on the number of debian ready packages we are talking about? The
next problem is I don't really know how to judge whether a package is
'ready for debian' or not.
One could of course start with the core/essential packages and then
slowly increase the package number. Robert Gentlemen suggested to start
with the packages in BioCLite, which is
affy affydata affyPLM annaffy annotate Biobase Biostrings DynDoc gcrma
genefilter geneplotter hgu95av2 limma marray matchprobes multtest
reposTools ROC vsn xtable.
What do yo think?
For the packages you listed above you do not really need our Debian gimmicks
for. It is easily installed with a few commands of R. But yes, except that I
would rather go for hgu133 they should all be in, ok the example data needs
the hgu95. I think I am aiming at affylmGUI and RBGL and such that are
standard but do not compile too easily for novices since they require to
install some extra bits. This way we (I include you here :-) ) can show off
with how cool Debian is a bit more.
Well having the core packages in (no matter how difficult it is to
install them manually) is always a good idea... it would be great if you
(or someone else with svn write access) could create a file that
contains the packages one could consider base and maybe start with the
highlights of the other
(microarray,annotation,statistics,graphs,technology) packages.
Something like debian-main/{base,microarray,...} etc.
The remaining R-packages could be packaged as single debian-packages as
you proposed to do it and maybe even hosted a bioconductor.org? In case
a package seems more mature it can enter any of the categories and one
could add proper conflicts/replaces as an upgrade path. BTW, this also
solves the `not-up-to-date issue', as more mature packages don't
require weekly/monthly updates.
Hm. I am not sure. The problem with hiding it all is that we also do not
use apt-cache search to find the proper BioC packages in the first place.
We hide this information away in the superpackages. It also impedes the
communication of Debian users with R developers and the assignment of
Bugs.
Yes you are right, that may be problematic. If we don't talk about
hundreds of packages it is probably also OK...
Fine. I personally think we are at about 25 packages in BioC that I'd
consider
part of core use cases. We did not have discussed this yet. The packaging is
automated. Some more pretty printing should probably be done before we
move bits into Debian main. Technically the packages are functional. Updates
are happening more frequently than you may think, actually, I would not want
to do it manually.
If one can create high quality packages in a automated way - fine why
not. However I think in the end it will be semi-automatic (as some final
checking is needed...)...
Btw, wouldn't you be interested to join our effort? I'd offer sponsoring
SHOGUN for Debian as a compensation :-)
Indeed I am interested, but I don't have any experience with debian+R
other than from packaging shogun-r. So I wonder whether for there exist
general cdbs helpers for r bioc.
Well, check out http://wiki.debian.org/AliothPkgBioc. For more detailed
R-packaging related issues Dirk [EMAIL PROTECTED] is your man.
OK.
Soeren.
--
Sometimes, there's a moment as you're waking, when you become aware of
the real world around you, but you're still dreaming.
signature.asc
Description: This is a digitally signed message part
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Sun, 2007-08-19 at 20:15 -0500, Dirk Eddelbuettel wrote: Hi Soeren, Hi Steffen Hi Dirk, [lots of agreement that would want to have more high quality R packages] | Btw, wouldn't you be interested to join our effort? I'd offer sponsoring | SHOGUN for Debian as a compensation :-) | | Indeed I am interested, but I don't have any experience with debian+R | other than from packaging shogun-r. So I wonder whether for there exist Look at any of my existing r-cran-* packages, and you see that they use a _very_ formulaic approach which is even distilled into a one-line debian/rules files, thanks to a) the standardization at the R package level (that, interestingly enough, is inspired by Debian) and b) the magic powers of cdbs which we use in the file /usr/share/R/debian/r-cran.mk --- which does all the actual package building and installing and provides the code for the one-liner calling it. So the key really is that Debian Policy is embedded in the is r-cran.mk file. The other debian/* files are standard. OK, I just had a look at r-cran-fseries, it is indeed a one-liner as you said. So creating debian packages from cran-r-packages seems easy. What I still don't understand is how you make sure that a package remains compatible with future r-versions (I could not find any indication that this is done via dependencies/conflicts). Taking the r-cran-fseries package as an example: $ dpkg -L r-cran-fseries | grep .so$ /usr/lib/R/site-library/fSeries/libs/fSeries.so $ ldd /usr/lib/R/site-library/fSeries/libs/fSeries.so linux-gate.so.1 = (0xb7f07000) libRlapack.so = /usr/lib/R/lib/libRlapack.so (0xb7873000) libblas.so.3 = /usr/lib/atlas/sse2/libblas.so.3 (0xb7295000) libgfortran.so.2 = /usr/lib/libgfortran.so.2 (0xb71fe000) libm.so.6 = /lib/i686/cmov/libm.so.6 (0xb71d9000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb71ce000) libR.so = /usr/lib/R/lib/libR.so (0xb6ef3000) libc.so.6 = /lib/i686/cmov/libc.so.6 (0xb6dab000) libg2c.so.0 = /usr/lib/libg2c.so.0 (0xb6d83000) /lib/ld-linux.so.2 (0x8000) libreadline.so.5 = /lib/libreadline.so.5 (0xb6d51000) libpcre.so.3 = /usr/lib/libpcre.so.3 (0xb6d2c000) libbz2.so.1.0 = /lib/libbz2.so.1.0 (0xb6d1c000) libz.so.1 = /usr/lib/libz.so.1 (0xb6d07000) libdl.so.2 = /lib/i686/cmov/libdl.so.2 (0xb6d03000) libncurses.so.5 = /usr/lib/libncurses.so.5 (0xb6cc2000) From that one can see that fSeries.so is dynamically linked to /usr/lib/R/lib/libR.so ... However libR.so has no SO name and if incompatible changes happen in R, that package will just not work until it is recompiled... I think the solution other people are using is to explicitly encode the version in the .so file, e.g. libR-2.5.so (setting the sonmae to libR-2.5.so) and including the version name in the package name, i.e. r-base-core version 2.5 = package r-base-core2.5... | general cdbs helpers for r bioc. Also I am still confused that | r-base-dev contains no header files (they are all in r-base-core) and Well, Doug chose the name years ago; r-base-dev is meant to provide all dependencies so that R users can do call install.packages() from R and not fall over because headers and libs such as readline or curses are missing; it is not a -dev package in the normal sense (which doesn't work for R as R is not a library you compile against). I don't understand... A R user that just wants to use R as is (i.e. not compile/install new packages) should not need to install header files, but all the header files are in r-base-core. So I still don't see why the files necessary to build R extensions (like header files) are not in r-base-dev? | that all the libR.so has no SO name (at least | objdump -p /usr/lib/R/lib/libR.so | grep SO does not report anything). Because ld.so does not see /usr/lib/R/lib as it doesn't need to know about it (as one typically does not compile against it). [ You have a harder job with You are right for many cran packages that are written in vanilla R (contain only .R files...), but for cran-packages containing C-extensions that is a problem (I remember having installed bioC locally and having it to recompile when a newer R version entered debian...). shogun and a better example for you would be Ggobi and r-cran-rggobi so look there instead of comparing with random CRAN package that are called into R -- I believe you call R into Shogun so the flow is different requiring a different setup. ] r-cran-ggobi only depends on r-base-core (= 2.5.0) ... so I don't see anything providing a smooth upgrade path... | So from my understanding the only build dependency is r-base-core, but | how does one ensure smooth upgrades when switching to new (potentially | incompatible) R releases? I don't follow exactly what the questions is. Maybe take this to private mail? Yes I am sorry, we can discuss this in private and maybe then only post the
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
Dirk Eddelbuettel dijo [Tue, Aug 14, 2007 at 09:26:31PM -0500]: UseR! 2007 at Iowa State University, Ames, IA, August 8-10, 2007 (...) Wow - Congratulations, this paper is quite interesting to me - partly because next week I will be presenting one quite similar to it ;-) http://vienna.yapceurope.org/ye2007/talk/513 http://people.debian.org/~gwolf/integrating_perl_in_distro.pdf Of course, the approach the pkg-perl group has taken is quite different. We have adopted a tool (dh-make-perl) similar to what you discuss, using CPAN's CPAN.pm module for the CPAN-updating part (AFAICT, equivalent to your r_pkg_update.pl), and takes care of basically extracting each module's metadata and building its package. Although some users have requested this functionality (and we even have a patch available implementing it), I have resisted the temptation of introducing an infrastructure-generating module - dh-make-perl deals only with individual packages. This is because I strongly oppose blindly generating packages, even if CPAN's quality is also quite acceptable and sanity tests are carried out. Anyway... I will take a closer look to your presentation (I only skimmed it) - Thanks for sharing! -- Gunnar Wolf - [EMAIL PROTECTED] - (+52-55)5623-0154 / 1451-2244 PGP key 1024D/8BB527AF 2001-10-23 Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
Hi Soeren, On 21 August 2007 at 11:20, Soeren Sonnenburg wrote: | OK, I just had a look at r-cran-fseries, it is indeed a one-liner as you | said. So creating debian packages from cran-r-packages seems easy. What Yes. debian/rules just calls /usr/share/R/debian/r-cran.mk which has the required smarts, and provides the infrastructure centrally. E.g. a good example of how it's nice to have the code in one place (as opposed to dozens of debian/rules files) is that just recently I learned about a neat x11 framebuffer server wrapper, so calling 'R CMD INSTALL ...' inside pbuilder will think it has x11 -- nice one. Kudos to Charles Plessy who had used it somewhere else. Now once I upgrade the R package with that, we'll have it centrally; a few packages need that and I can just test for presence of that wrapper and if found, use it. End of digression :) | I still don't understand is how you make sure that a package remains | compatible with future r-versions (I could not find any indication that | this is done via dependencies/conflicts). Taking the r-cran-fseries Fair point -- and we just don't! I think a need for this arose only once in ten years of R development in Debian when R 2.0.0 changed enough internals to warrant it. In that case I just hand-rebuilt all of the r-cran-* I maintained and bugged the maintainers of the other ones. Worked well enough. There isn't really enough use of R outside of its own packages to warrant the heavier hand of encoding major versions, and requiring builds. CRAN package authors typically upload new packages for new R versions as the CRAN archive is pretty adamant about tests with R CMD check against current, patched and devel versions of R. | From that one can see that fSeries.so is dynamically linked | to /usr/lib/R/lib/libR.so ... However libR.so has no SO name and if | incompatible changes happen in R, that package will just not work until | it is recompiled... Yes. Remember that libR.so is not an exported API (as stated multiple time in the R docs) and not known to ld.so and friends. So this doesn't have to behave like libz or libjpeg with soname encodings, multiple versions, ... | I think the solution other people are using is to explicitly encode the | version in the .so file, e.g. libR-2.5.so (setting the sonmae to | libR-2.5.so) and including the version name in the package name, i.e. | r-base-core version 2.5 = package r-base-core2.5... But only for 'visible' libraries in /usr/lib etc. | | general cdbs helpers for r bioc. Also I am still confused that | | r-base-dev contains no header files (they are all in r-base-core) and | | Well, Doug chose the name years ago; r-base-dev is meant to provide all | dependencies so that R users can do call install.packages() from R and not | fall over because headers and libs such as readline or curses are missing; it | is not a -dev package in the normal sense (which doesn't work for R as R is | not a library you compile against). | | I don't understand... A R user that just wants to use R as is (i.e. not | compile/install new packages) should not need to install header files, 'du -csk /usr/share/R/include' yields 420k. I have thought in the past about splitting them off, but weighted that against the total size of the package, the savings are so small as to not matter. | but all the header files are in r-base-core. So I still don't see why | the files necessary to build R extensions (like header files) are not in | r-base-dev? Trust me that for almost every R installation you want both r-base-core and r-base-dev. Exactly what would we achieve by moving them around, other than a few kilobytes of header files that would be in the 'arch all' package r-base-dev rather than in each binary r-base-core? Also, these may have arch-specific configure results in them so I could create new and subtle bugs by moving them. I'd rather not, unless you have a real issue with this. | | that all the libR.so has no SO name (at least | | objdump -p /usr/lib/R/lib/libR.so | grep SO does not report anything). | | Because ld.so does not see /usr/lib/R/lib as it doesn't need to know about it | (as one typically does not compile against it). [ You have a harder job with | | You are right for many cran packages that are written in vanilla R | (contain only .R files...), but for cran-packages containing | C-extensions that is a problem (I remember having installed bioC locally | and having it to recompile when a newer R version entered debian...). but BioC != CRAN. BioC tags releases to R releases. CRAN doesn't. So this does not apply as such. And besides, R packages typically test for the R version themselves, so no need to double it via soname encodings. | shogun and a better example for you would be Ggobi and r-cran-rggobi so look | there instead of comparing with random CRAN package that are called into R -- | I believe you call R into Shogun so the flow is different requiring a | different
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
Hi, Am Dienstag, den 21.08.2007, 11:11 -0500 schrieb Dirk Eddelbuettel: E.g. a good example of how it's nice to have the code in one place (as opposed to dozens of debian/rules files) is that just recently I learned about a neat x11 framebuffer server wrapper, Which package is that? I might need this wrapper for Octave. Thanks Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
Hi Thomas, On 21 August 2007 at 19:02, Thomas Weber wrote: | Hi, | | Am Dienstag, den 21.08.2007, 11:11 -0500 schrieb Dirk Eddelbuettel: | E.g. a good example of how it's nice to have the code in one place (as | opposed to dozens of debian/rules files) is that just recently I learned about | a neat x11 framebuffer server wrapper, | | Which package is that? I might need this wrapper for Octave. I was wondering myself ;-) so I untarred the pbuilder image into which I installed it. The script is called 'xvfb-run' and comes with the xvfb packages, as I recall I also needed to install xfonts-base to make xvfb useable. Regards, Dirk -- Three out of two people have difficulties with fractions. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
Hi Soeren, Hi Steffen Now back from a short vacation in Michigan, I'll chime in on the ongoing discussion. On 18 August 2007 at 22:45, Soeren Sonnenburg wrote: | On Sat, 2007-08-18 at 14:44 +0200, Steffen Moeller wrote: | On Saturday 18 August 2007 12:36:41 Soeren Sonnenburg wrote: | On Tue, 2007-08-14 at 21:26 -0500, Dirk Eddelbuettel wrote: | [...] | esoteric/brand new research/unstable R-packes. However I would want to | see the more mature bioconductor packages in debian... | | Again, I think we can agree on this, | | OK great. Yes, we all agree on 'more is better' as well as on 'maybe these should not be in debian' as two thousand machine generated packages. | One could of course start with the core/essential packages and then | slowly increase the package number. Robert Gentlemen suggested to start | with the packages in BioCLite, which is | | affy affydata affyPLM annaffy annotate Biobase Biostrings DynDoc gcrma | genefilter geneplotter hgu95av2 limma marray matchprobes multtest | reposTools ROC vsn xtable. | | What do yo think? Yes, and as an aside, these could of course be maintained 'by hand' as well just like our current 50 or 60 CRAN packages. For Debian proper, this may be better than the machine generated ones. BioC is high-profile enough that we may find a few volunteers to join a packaging group on Alioth. So getting 'core BioC' into Debian is actually a different task. Maybe someone should pursue it (and I won't necessarily jump in as I am more on the CRAN side of things). But yes, this would be a good thing. | Btw, wouldn't you be interested to join our effort? I'd offer sponsoring | SHOGUN for Debian as a compensation :-) | | Indeed I am interested, but I don't have any experience with debian+R | other than from packaging shogun-r. So I wonder whether for there exist Look at any of my existing r-cran-* packages, and you see that they use a _very_ formulaic approach which is even distilled into a one-line debian/rules files, thanks to a) the standardization at the R package level (that, interestingly enough, is inspired by Debian) and b) the magic powers of cdbs which we use in the file /usr/share/R/debian/r-cran.mk --- which does all the actual package building and installing and provides the code for the one-liner calling it. So the key really is that Debian Policy is embedded in the is r-cran.mk file. The other debian/* files are standard. | general cdbs helpers for r bioc. Also I am still confused that | r-base-dev contains no header files (they are all in r-base-core) and Well, Doug chose the name years ago; r-base-dev is meant to provide all dependencies so that R users can do call install.packages() from R and not fall over because headers and libs such as readline or curses are missing; it is not a -dev package in the normal sense (which doesn't work for R as R is not a library you compile against). | that all the libR.so has no SO name (at least | objdump -p /usr/lib/R/lib/libR.so | grep SO does not report anything). Because ld.so does not see /usr/lib/R/lib as it doesn't need to know about it (as one typically does not compile against it). [ You have a harder job with shogun and a better example for you would be Ggobi and r-cran-rggobi so look there instead of comparing with random CRAN package that are called into R -- I believe you call R into Shogun so the flow is different requiring a different setup. ] | So from my understanding the only build dependency is r-base-core, but | how does one ensure smooth upgrades when switching to new (potentially | incompatible) R releases? I don't follow exactly what the questions is. Maybe take this to private mail? Good to be discussing R on debian-devel :) Cheers, Dirk -- Three out of two people have difficulties with fractions. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Tue, 2007-08-14 at 21:26 -0500, Dirk Eddelbuettel wrote: Hi Dirk, UseR! 2007 at Iowa State University, Ames, IA, August 8-10, 2007 [...] II. Paper / presentation on 2000 new Debian packages -- Would you like 2000 new Debian packages with that? Something we hadn't really reported back to Debian is the relative success and current status of the 'pkg-bioc' project at Alioth. [...] The big news is that we can now build most of the around two thousand source packages [ around 900 from CRAN and 1100 from BioC, I concentrate more on CRAN; Steffen focusses more on BioC, and David does everything ] for R from the CRAN and BioConductor archives. That's what our paper that I presented was about, as well as an earlier presentation / paper Steffen gave two months ago in Italy [1]. [...] The big question is what to do with these 2000 packages. The process is still too manual and fragile to be called 'production class'. Eventually this should move somewhere -- either CRAN itself, or, less likely, be part of Debian. I do say less likely here because I don't hink that two-thousand machine-generated packages could reach the Debian QA standards. They are also, as a large class, too esoteric. CRAN, on the other hand, builds binaries for Windows, so this could be a better fit. Someone suggested R-Forge -- which is a rather recent 'SF / Alioth for R' based on Debian's gforge packages. Also, one question had to do with how to avoid 'waits' for new packages -- people wouldn't want to wait for packages to reach testing when this can take months. So a distinct backport service may be the best option. Manpower and mirrors may be the crux. First of all I really like your efforts of debianizing R packages. I think debian is currently well suited to be used in ``science'' but yes there is a lot more one can do. I was indeed missing the nowadays quite standard bioconductor packages when I had to do some microarray analysis. Regarding machine generated debian packages it is a first step and probably the only way given this amount of R-packages. I also don't think they could be in debian. This especially holds for the more esoteric/brand new research/unstable R-packes. However I would want to see the more mature bioconductor packages in debian... Thinking about it, *I* think it would be best to proceed in a similar way as the texlive people, i.e. have debian packages for all major categories which include the major mature R-packages of that category r-bioc-base r-bioc-microarray r-bioc-annotation r-bioc-statistics r-bioc-graphs r-bioc-technology The remaining R-packages could be packaged as single debian-packages as you proposed to do it and maybe even hosted a bioconductor.org? In case a package seems more mature it can enter any of the categories and one could add proper conflicts/replaces as an upgrade path. BTW, this also solves the `not-up-to-date issue', as more mature packages don't require weekly/monthly updates. Soeren -- For the one fact about the future of which we can be certain is that it will be utterly fantastic. -- Arthur C. Clarke, 1962 signature.asc Description: This is a digitally signed message part
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Saturday 18 August 2007 12:36:41 Soeren Sonnenburg wrote: On Tue, 2007-08-14 at 21:26 -0500, Dirk Eddelbuettel wrote: First of all I really like your efforts of debianizing R packages. I think debian is currently well suited to be used in ``science'' but yes there is a lot more one can do. I was indeed missing the nowadays quite standard bioconductor packages when I had to do some microarray analysis. It is what we feel. Many thanks for your positive feedback. Regarding machine generated debian packages it is a first step and probably the only way given this amount of R-packages. I also don't think they could be in debian. This especially holds for the more esoteric/brand new research/unstable R-packes. However I would want to see the more mature bioconductor packages in debian... Again, I think we can agree on this, Thinking about it, *I* think it would be best to proceed in a similar way as the texlive people, i.e. have debian packages for all major categories which include the major mature R-packages of that category r-bioc-base r-bioc-microarray r-bioc-annotation r-bioc-statistics r-bioc-graphs r-bioc-technology Hm. I kind of like it, though I rather see this implemented as virtual packages that come rather naturally from the Biotags that BioConductor assigns to itself. The remaining R-packages could be packaged as single debian-packages as you proposed to do it and maybe even hosted a bioconductor.org? In case a package seems more mature it can enter any of the categories and one could add proper conflicts/replaces as an upgrade path. BTW, this also solves the `not-up-to-date issue', as more mature packages don't require weekly/monthly updates. Hm. I am not sure. The problem with hiding it all is that we also do not use apt-cache search to find the proper BioC packages in the first place. We hide this information away in the superpackages. It also impedes the communication of Debian users with R developers and the assignment of Bugs. Btw, wouldn't you be interested to join our effort? I'd offer sponsoring SHOGUN for Debian as a compensation :-) Many greetings from the fairly sunny Baltic Sea to my former home Berlin Steffen -- Dr. Steffen Möller University of Lübeck Institute for Neuro- and Bioinformatics Ratzeburger Allee 160 23538 Lübeck Germany T: +49 451 500 5504 F: +49 451 500 5502 [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part.
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Sat, 2007-08-18 at 14:44 +0200, Steffen Moeller wrote: On Saturday 18 August 2007 12:36:41 Soeren Sonnenburg wrote: On Tue, 2007-08-14 at 21:26 -0500, Dirk Eddelbuettel wrote: [...] esoteric/brand new research/unstable R-packes. However I would want to see the more mature bioconductor packages in debian... Again, I think we can agree on this, OK great. Thinking about it, *I* think it would be best to proceed in a similar way as the texlive people, i.e. have debian packages for all major categories which include the major mature R-packages of that category r-bioc-base r-bioc-microarray r-bioc-annotation r-bioc-statistics r-bioc-graphs r-bioc-technology Hm. I kind of like it, though I rather see this implemented as virtual packages that come rather naturally from the Biotags that BioConductor assigns to itself. Well I don't know how much should be split up. But I guess this depends also on the number of debian ready packages we are talking about? The next problem is I don't really know how to judge whether a package is 'ready for debian' or not. One could of course start with the core/essential packages and then slowly increase the package number. Robert Gentlemen suggested to start with the packages in BioCLite, which is affy affydata affyPLM annaffy annotate Biobase Biostrings DynDoc gcrma genefilter geneplotter hgu95av2 limma marray matchprobes multtest reposTools ROC vsn xtable. What do yo think? The remaining R-packages could be packaged as single debian-packages as you proposed to do it and maybe even hosted a bioconductor.org? In case a package seems more mature it can enter any of the categories and one could add proper conflicts/replaces as an upgrade path. BTW, this also solves the `not-up-to-date issue', as more mature packages don't require weekly/monthly updates. Hm. I am not sure. The problem with hiding it all is that we also do not use apt-cache search to find the proper BioC packages in the first place. We hide this information away in the superpackages. It also impedes the communication of Debian users with R developers and the assignment of Bugs. Yes you are right, that may be problematic. If we don't talk about hundreds of packages it is probably also OK... Btw, wouldn't you be interested to join our effort? I'd offer sponsoring SHOGUN for Debian as a compensation :-) Indeed I am interested, but I don't have any experience with debian+R other than from packaging shogun-r. So I wonder whether for there exist general cdbs helpers for r bioc. Also I am still confused that r-base-dev contains no header files (they are all in r-base-core) and that all the libR.so has no SO name (at least objdump -p /usr/lib/R/lib/libR.so | grep SO does not report anything). So from my understanding the only build dependency is r-base-core, but how does one ensure smooth upgrades when switching to new (potentially incompatible) R releases? Regarding shogun, Torsten Werner is already sponsoring the uploads - so don't worry :-) Many greetings from the fairly sunny Baltic Sea to my former home Berlin Actually I am planning to be at the baltic sea next weekend to take part in the 'vilmschwimmen'! Soeren -- For the one fact about the future of which we can be certain is that it will be utterly fantastic. -- Arthur C. Clarke, 1962 signature.asc Description: This is a digitally signed message part
Re: Short report on Debian at UseR! 2007 conference at Iowa State Univ.
On Saturday 18 August 2007 22:45:56 Soeren Sonnenburg wrote: On Sat, 2007-08-18 at 14:44 +0200, Steffen Moeller wrote: On Saturday 18 August 2007 12:36:41 Soeren Sonnenburg wrote: [...] Well I don't know how much should be split up. But I guess this depends also on the number of debian ready packages we are talking about? The next problem is I don't really know how to judge whether a package is 'ready for debian' or not. One could of course start with the core/essential packages and then slowly increase the package number. Robert Gentlemen suggested to start with the packages in BioCLite, which is affy affydata affyPLM annaffy annotate Biobase Biostrings DynDoc gcrma genefilter geneplotter hgu95av2 limma marray matchprobes multtest reposTools ROC vsn xtable. What do yo think? For the packages you listed above you do not really need our Debian gimmicks for. It is easily installed with a few commands of R. But yes, except that I would rather go for hgu133 they should all be in, ok the example data needs the hgu95. I think I am aiming at affylmGUI and RBGL and such that are standard but do not compile too easily for novices since they require to install some extra bits. This way we (I include you here :-) ) can show off with how cool Debian is a bit more. The remaining R-packages could be packaged as single debian-packages as you proposed to do it and maybe even hosted a bioconductor.org? In case a package seems more mature it can enter any of the categories and one could add proper conflicts/replaces as an upgrade path. BTW, this also solves the `not-up-to-date issue', as more mature packages don't require weekly/monthly updates. Hm. I am not sure. The problem with hiding it all is that we also do not use apt-cache search to find the proper BioC packages in the first place. We hide this information away in the superpackages. It also impedes the communication of Debian users with R developers and the assignment of Bugs. Yes you are right, that may be problematic. If we don't talk about hundreds of packages it is probably also OK... Fine. I personally think we are at about 25 packages in BioC that I'd consider part of core use cases. We did not have discussed this yet. The packaging is automated. Some more pretty printing should probably be done before we move bits into Debian main. Technically the packages are functional. Updates are happening more frequently than you may think, actually, I would not want to do it manually. Btw, wouldn't you be interested to join our effort? I'd offer sponsoring SHOGUN for Debian as a compensation :-) Indeed I am interested, but I don't have any experience with debian+R other than from packaging shogun-r. So I wonder whether for there exist general cdbs helpers for r bioc. Well, check out http://wiki.debian.org/AliothPkgBioc. For more detailed R-packaging related issues Dirk [EMAIL PROTECTED] is your man. Regarding shogun, Torsten Werner is already sponsoring the uploads - so don't worry :-) Nice. Many greetings from the fairly sunny Baltic Sea to my former home Berlin Actually I am planning to be at the baltic sea next weekend to take part in the 'vilmschwimmen'! Enjoy Ruegen, then. Steffen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Short report on Debian at UseR! 2007 conference at Iowa State Univ.
UseR! 2007 at Iowa State University, Ames, IA, August 8-10, 2007 Following two successful UseR! conferences in Vienna in 2004 and 2006, the first North American UseR! was help last week at Iowa State. I presented two papers of which one has specific Debian content (more on that one below). I. General Debian / Ubuntu feedback === Having maintained R and related packages for a number of years, and having gone to four R conferences, I am somewhat recognised as the R/Debian link. There are a number of Debian users among the R users and developers, though there may now be more Ubuntu users. One sees the odd Fedora Core machine, but in general there is always a surprisingly large number of Macs and the obvious wintel ones. People seem happy with Debian and Ubuntu, though still too few R users and developers seem to know that there are 'backports' of the current R release for testing and different Ubuntu releases on every CRAN mirror below bin/linux/. [ I help in this backporting effort, as did Chris Steigies in the past, but it is mostly non-DDs Johannes Ranke (for Debian backports) and Vincent Goulet (for Ubuntu) ]. So I was asked a few questions about how to get R 2.5.1 onto Ubuntu 6.06 LTS or 7.04 and it is really just an apt-get update away. I gave a talk on automated builds of CRAN/BioC package (see below) and at the outset asked the audience how many actually ran Debian or Ubuntu and about a fifth to a quarter raised their hand -- but that is of course a self-selected group. We generally have a pretty decent reputation in this community. No other distro had included R as early as 1997, and we have more add-ons and related software than other distros. People acknowledge that Debian/Ubuntu 'just work' and most are happy that this allows them to concentrate on their work. Also, quite a few (if not most Linux machines) of the backends of CRAN, and releated services, are using Debian. II. Paper / presentation on 2000 new Debian packages -- Would you like 2000 new Debian packages with that? Something we hadn't really reported back to Debian is the relative success and current status of the 'pkg-bioc' project at Alioth. It goes back to something that Matt Hope (aiming for BioConductor.org, an R repository for bioinformatics) and I (aiming for CRAN, R's core repo) had started sort-of at the same time around 2003 or 2004, then merged and which puddled along somewhat slowly. Steffen Moeller, an Alioth guest-member for years and a very recent new DD, and David Vernazobres, also an Alioth guest-member, did A LOT of work over the last 12 to 18 months, and I started to chip in little bits here or there too. The big news is that we can now build most of the around two thousand source packages [ around 900 from CRAN and 1100 from BioC, I concentrate more on CRAN; Steffen focusses more on BioC, and David does everything ] for R from the CRAN and BioConductor archives. That's what our paper that I presented was about, as well as an earlier presentation / paper Steffen gave two months ago in Italy [1]. This was well received at the conference. Among the self-selected crowd of Debian (or Ubuntu) users, the upside of this was clear, and most seem to agree with the reasons we give in the paper as motivation. (In a nutshell: dependencies work, convenience of 'build one install often', quality control, scalability, common platform, different architectures, large audience -- see the paper for discussion). The big question is what to do with these 2000 packages. The process is still too manual and fragile to be called 'production class'. Eventually this should move somewhere -- either CRAN itself, or, less likely, be part of Debian. I do say less likely here because I don't hink that two-thousand machine-generated packages could reach the Debian QA standards. They are also, as a large class, too esoteric. CRAN, on the other hand, builds binaries for Windows, so this could be a better fit. Someone suggested R-Forge -- which is a rather recent 'SF / Alioth for R' based on Debian's gforge packages. Also, one question had to do with how to avoid 'waits' for new packages -- people wouldn't want to wait for packages to reach testing when this can take months. So a distinct backport service may be the best option. Manpower and mirrors may be the crux. [1] Slides and papers are at http://dirk.eddelbuettel.com/presentations.html but note that neither paper had a DD as the target audience Comments especially on II would be welcome. Please CC me on replies as I am no longer subscribed to debian-devel. Regards, Dirk -- Three out of two people have difficulties with fractions. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
