Hi, Liam,
Thank you for your suggestions. I tried fitMk in phytools and ace in ape.
Below is what I got.
> fit.phytools<-fitMk(tree,bathy2,model="ER",pi="equal")
> fit.phytools
Object of class "fitMk".
Fitted (or set) value of Q:
1shallow 2bathypelagic
1shallow NaN
Hi Lei.
Looks like an error. The likelihood is actually a very *small* number as
the log-likelihood is a very *large* negative number. Similarly, the
fitted transition rates are very small - near zero.
Have you tried to fit the same model using ace in the ape package or
fitMk in phytools?