Re: [R-sig-phylo] dist.nodes crashing with big trees

[Gustavo Burin Ferreira]

Hey David and Nick,

thanks a lot for the quick responses! I think I wasn't very clear in the
first e-mail. What I get is actually an error from within dist.nodes, not
when calling it.

I've tried to use chainsaw2 and in the beginning it appeared to be working
quite well. However after some running time, I get the same (original)
error that motivated me writing to the list:


> *Error in double(nm * nm) : vector size cannot be NA*
> *In addition: Warning message:**In nm * nm : NAs produced by integer
> overflow*


Digging into the functions called within chainsaw2, I found that at some
point it uses the function get_max_height_tree, that calls dist.nodes and
that's where I think the problem lies. The error I got now is almost
exactly the same as I got from timeSliceTree (because both cases use
dist.nodes):


> *dist.nodes*
> *function (x) *
> *{*
> *x <- reorder(x)*
> *n <- Ntip(x)*
> *m <- x$Nnode*
> *nm <- n + m*
> *d <- .C(dist_nodes, as.integer(n), as.integer(m),
> as.integer(x$edge[, *
> *1] - 1L), as.integer(x$edge[, 2] - 1L),
> as.double(x$edge.length), *
> *as.integer(Nedge(x)), double(nm * nm), NAOK = TRUE)[[7]]*
> *dim(d) <- c(nm, nm)*
> *dimnames(d) <- list(1:nm, 1:nm)*
> *d**}*


I tried changing the highlighted part to something like
double(as.numeric(nm) * as.numeric(nm)), and when I try running it, I get
the error I wrote on the first e-mail:


> *Error in dist.nodes(tree) (from #7) : **  long vectors (argument 7) are
> not supported in .Fortran*


Thus, I think that to solve this problem some tweak in the C/Fortran code
that is called within dist.nodes (from ape) might be required, but I have
no expertise on that. So if someone can help me with that, I'll appreciate
it!

Thanks again for the help so far!

Best,


*Gustavo Burin Ferreira, **Msc.*
Instituto de Biociências
Universidade de São Paulo
Tel: (11) 98525-8948

On Fri, Oct 16, 2015 at 5:06 PM, Nick Matzke  wrote:

> Hi!  I re-did chainsaw at some point, now there is chainsaw2.  However,
> googling that gets you horror movies, so here is a link with example code:
>
> https://groups.google.com/d/msg/biogeobears/Jy9uYckOL7s/XuNZ0B3jAwAJ
>
> (the discussion there points out a rare case where this crashes, but for
> most trees it should work fine)
>
> Cheers, Nick
>
> On Fri, Oct 16, 2015 at 2:17 PM, David Bapst  wrote:
>
> > Hi Gustavo,
> >
> > I'm paleotree's author and maintainer. Just to be clear that I
> > understand your problem, I believe you are saying that when you use
> > timeSliceTree, you are getting an error that the internal call to
> > dist.nodes is failing? Is that right?
> >
> > The first thought I have is that maybe the solution here is to avoid
> > dist.nodes, as it is somewhat overkill. I use dist.nodes in that code,
> > which I wrote in 2011, to get the distance of tips and nodes from the
> > root. A better solution may now exist in another R package. I'd have
> > to investigate (although maybe someone on the list can suggest one).
> >
> > The second thought I have is that there might be alternative functions
> > that do something lie timeSliceTree in another R package. Off the top
> > of my head, I recall that Nick Matzke had a similar, 'chainsaw'
> > function, which you can find here and appears not to call dist.nodes:
> >
> > https://stat.ethz.ch/pipermail/r-sig-phylo/2011-July/001483.html
> >
> > Again, maybe someone on the list knows of a good alternative function.
> >
> > I'll try to give this more thought, but for now, maybe see if you can
> > use Nick's function succesfully. Overall though, I've discovered the
> > use of truly gigantic trees can often run into unexpected problems.
> >
> > Cheers,
> > -Dave
> >
> >
> >
> > On Fri, Oct 16, 2015 at 12:47 PM, Gustavo Burin Ferreira
> >  wrote:
> > > Dear list,
> > >
> > > I'm trying to perform a time travel in simulated phylogenies with both
> > > extant and extinct species using the timeSliceTree function form the
> > > paleotree package. My aim is to have the molecular phylogenies derived
> > from
> > > the complete phylogeny (attached) in different points in time.
> > >
> > > However, when I try that with big trees (bigger than 2 tips
> total), I
> > > get an error of integer overflow coming from the dist.nodes function.
> > After
> > > slightly tweaking the dist.nodes function (changing nm from integer to
> > > numeric/double), I get the following message:
> > >
> > > Error in dist.nodes(tree) (from #7) :
> > >   long vectors (argument 7) are not supported in .Fortran
> > >
> > > Since I don't know much about C or Fortran, I couldn't find a way of
> > solving
> > > this by myself, so any help will be greatly appreciated.
> > >
> > > I'm sending one tree attached for example.
> > >
> > > Thank you very much in advance!
> > >
> > > Best,
> > >
> > > Gustavo Burin Ferreira, Msc.
> > > Instituto de Biociências
> > > Universidade de São Paulo
> > > Tel: 

Re: [R-sig-phylo] dist.nodes crashing with big trees

[David Bapst]

Hi Gustavo,

I'm paleotree's author and maintainer. Just to be clear that I
understand your problem, I believe you are saying that when you use
timeSliceTree, you are getting an error that the internal call to
dist.nodes is failing? Is that right?

The first thought I have is that maybe the solution here is to avoid
dist.nodes, as it is somewhat overkill. I use dist.nodes in that code,
which I wrote in 2011, to get the distance of tips and nodes from the
root. A better solution may now exist in another R package. I'd have
to investigate (although maybe someone on the list can suggest one).

The second thought I have is that there might be alternative functions
that do something lie timeSliceTree in another R package. Off the top
of my head, I recall that Nick Matzke had a similar, 'chainsaw'
function, which you can find here and appears not to call dist.nodes:

https://stat.ethz.ch/pipermail/r-sig-phylo/2011-July/001483.html

Again, maybe someone on the list knows of a good alternative function.

I'll try to give this more thought, but for now, maybe see if you can
use Nick's function succesfully. Overall though, I've discovered the
use of truly gigantic trees can often run into unexpected problems.

Cheers,
-Dave



On Fri, Oct 16, 2015 at 12:47 PM, Gustavo Burin Ferreira
 wrote:
> Dear list,
>
> I'm trying to perform a time travel in simulated phylogenies with both
> extant and extinct species using the timeSliceTree function form the
> paleotree package. My aim is to have the molecular phylogenies derived from
> the complete phylogeny (attached) in different points in time.
>
> However, when I try that with big trees (bigger than 2 tips total), I
> get an error of integer overflow coming from the dist.nodes function. After
> slightly tweaking the dist.nodes function (changing nm from integer to
> numeric/double), I get the following message:
>
> Error in dist.nodes(tree) (from #7) :
>   long vectors (argument 7) are not supported in .Fortran
>
> Since I don't know much about C or Fortran, I couldn't find a way of solving
> this by myself, so any help will be greatly appreciated.
>
> I'm sending one tree attached for example.
>
> Thank you very much in advance!
>
> Best,
>
> Gustavo Burin Ferreira, Msc.
> Instituto de Biociências
> Universidade de São Paulo
> Tel: (11) 98525-8948
>
> ___
> R-sig-phylo mailing list - R-sig-phylo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/



-- 
David W. Bapst, PhD
Adjunct Asst. Professor, Geology and Geol. Eng.
South Dakota School of Mines and Technology
501 E. St. Joseph
Rapid City, SD 57701

http://webpages.sdsmt.edu/~dbapst/
http://cran.r-project.org/web/packages/paleotree/index.html

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/

Re: [R-sig-phylo] dist.nodes crashing with big trees

[Nick Matzke]

Hi!  I re-did chainsaw at some point, now there is chainsaw2.  However,
googling that gets you horror movies, so here is a link with example code:

https://groups.google.com/d/msg/biogeobears/Jy9uYckOL7s/XuNZ0B3jAwAJ

(the discussion there points out a rare case where this crashes, but for
most trees it should work fine)

Cheers, Nick

On Fri, Oct 16, 2015 at 2:17 PM, David Bapst  wrote:

> Hi Gustavo,
>
> I'm paleotree's author and maintainer. Just to be clear that I
> understand your problem, I believe you are saying that when you use
> timeSliceTree, you are getting an error that the internal call to
> dist.nodes is failing? Is that right?
>
> The first thought I have is that maybe the solution here is to avoid
> dist.nodes, as it is somewhat overkill. I use dist.nodes in that code,
> which I wrote in 2011, to get the distance of tips and nodes from the
> root. A better solution may now exist in another R package. I'd have
> to investigate (although maybe someone on the list can suggest one).
>
> The second thought I have is that there might be alternative functions
> that do something lie timeSliceTree in another R package. Off the top
> of my head, I recall that Nick Matzke had a similar, 'chainsaw'
> function, which you can find here and appears not to call dist.nodes:
>
> https://stat.ethz.ch/pipermail/r-sig-phylo/2011-July/001483.html
>
> Again, maybe someone on the list knows of a good alternative function.
>
> I'll try to give this more thought, but for now, maybe see if you can
> use Nick's function succesfully. Overall though, I've discovered the
> use of truly gigantic trees can often run into unexpected problems.
>
> Cheers,
> -Dave
>
>
>
> On Fri, Oct 16, 2015 at 12:47 PM, Gustavo Burin Ferreira
>  wrote:
> > Dear list,
> >
> > I'm trying to perform a time travel in simulated phylogenies with both
> > extant and extinct species using the timeSliceTree function form the
> > paleotree package. My aim is to have the molecular phylogenies derived
> from
> > the complete phylogeny (attached) in different points in time.
> >
> > However, when I try that with big trees (bigger than 2 tips total), I
> > get an error of integer overflow coming from the dist.nodes function.
> After
> > slightly tweaking the dist.nodes function (changing nm from integer to
> > numeric/double), I get the following message:
> >
> > Error in dist.nodes(tree) (from #7) :
> >   long vectors (argument 7) are not supported in .Fortran
> >
> > Since I don't know much about C or Fortran, I couldn't find a way of
> solving
> > this by myself, so any help will be greatly appreciated.
> >
> > I'm sending one tree attached for example.
> >
> > Thank you very much in advance!
> >
> > Best,
> >
> > Gustavo Burin Ferreira, Msc.
> > Instituto de Biociências
> > Universidade de São Paulo
> > Tel: (11) 98525-8948
> >
> > ___
> > R-sig-phylo mailing list - R-sig-phylo@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> > Searchable archive at
> http://www.mail-archive.com/r-sig-phylo@r-project.org/
>
>
>
> --
> David W. Bapst, PhD
> Adjunct Asst. Professor, Geology and Geol. Eng.
> South Dakota School of Mines and Technology
> 501 E. St. Joseph
> Rapid City, SD 57701
>
> http://webpages.sdsmt.edu/~dbapst/
> http://cran.r-project.org/web/packages/paleotree/index.html
>
> ___
> R-sig-phylo mailing list - R-sig-phylo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
> Searchable archive at
> http://www.mail-archive.com/r-sig-phylo@r-project.org/

[[alternative HTML version deleted]]

___
R-sig-phylo mailing list - R-sig-phylo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-phylo
Searchable archive at http://www.mail-archive.com/r-sig-phylo@r-project.org/