Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-05-02 Thread Kasper Daniel Hansen
We did this recently in minfi and getting acceptable performance was not trivial. We are still sorting out issues; we will eventually write about our experiences. Pete is working on some blog posts on this and we hope to do something more formal later. We would all love experience from more

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-05-02 Thread Elizabeth Purdom
Thanks Hervé and Stephanie for your suggestions. I am really looking for a S4 methods solution however, given how my package is already set up. Also, I have several functions that I need to adapt in this way, so it seems cleaner and simpler to do the class union, which sounds like is not a

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Hervé Pagès
Interesting. I tried something like that in the past i.e. start with a unary setClassUnion() but then got into problems when I tried to add new members to the union by **extending** the union class: https://stat.ethz.ch/pipermail/r-devel/2016-March/072489.html So it seems like I should have

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Michael Lawrence
It would be great to be able to define a matrix-like abstraction independent of 'matrix' and 'DelayedMatrix'. It could also encompass objects from the Matrix package and potentially other things. So you could define a parent class of 'matrix' using setClassUnion() and then use setIs() to establish

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Hervé Pagès
The class union should probably be: setClassUnion("matrixOrDelayed", c("matrix", "DelayedMatrix")) i.e. use DelayedMatrix instead of DelayedArray. So in addition to the class union and to Stephanie's solution, which IMO are both valid solutions, you could also go for something like this:

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Hervé Pagès
Just to mention that the issue with rowSums() on a big DelayedMatrix objects that you are referring to Tim is duly noted and will be one of my first priorities once we're done with the release process. Cheers, H. On 04/30/2018 09:57 AM, Tim Triche, Jr. wrote: much obliged -- and the packages

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Tim Triche, Jr.
much obliged -- and the packages are terrific, I am not surprised that a big step is accompanied with some growing pains. Thanks to you and Herve and Keegan for enthusiastically chasing down, and spending your time fixing, this and other bugs. Having fiddled with bigMemory and bigMatrix backends

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Stephanie M. Gogarten
Rather than a class union, how about an internal function that is called by the methods for both matrix and DelayedArray: setGeneric("myNewRowMeans", function(x,...) { standardGeneric("myNewRowMeans")}) #' @importFrom DelayedArray rowMeans .myNewRowMeans <- function(x,...){ # a lot of

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Peter Hickey
Tim: As the developer of DelayedMatrixStats (and enthusiastic 'canary down the coal mine' user-dev of DelayedArray) I'm obviously invested in reducing the confusion around these packages I'm going to write some blog posts-cum-vignettes-cum-F1000 around these issues over the coming weeks, with the

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Tim Triche, Jr.
But if you merge methods like that, the error method can be that much more difficult to identify. It took a couple of weeks to chase that bug down properly, and it ended up down to rowMeans2 vs rowMeans. I suppose the merged/abstracted method allows to centralize any such dispatch into one place

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Martin Morgan
But that issue will be fixed, so Tim's advice is inappropriate. On 04/30/2018 10:42 AM, Tim Triche, Jr. wrote: Don't do that. Seriously, just don't. https://github.com/Bioconductor/DelayedArray/issues/16 --t On Mon, Apr 30, 2018 at 10:02 AM, Elizabeth Purdom < epur...@stat.berkeley.edu>

Re: [Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Tim Triche, Jr.
Don't do that. Seriously, just don't. https://github.com/Bioconductor/DelayedArray/issues/16 --t On Mon, Apr 30, 2018 at 10:02 AM, Elizabeth Purdom < epur...@stat.berkeley.edu> wrote: > Hello, > > I am trying to extend my package to handle `HDF5Matrix` class ( or more > generally

[Bioc-devel] Virtual class for `matrix` and `DelayedArray`? (or better strategy for dealing with them both)

2018-04-30 Thread Elizabeth Purdom
Hello, I am trying to extend my package to handle `HDF5Matrix` class ( or more generally `DelayedArray`). I currently have S4 functions for `matrix` class. Usually I have a method for `SummarizedExperiment`, which will call call the method on `assay(x)` and I want the method to be able to deal