Given that the maximum floating point value is:
$double.xmax
[1] 1.797693e+308
and the number you are trying to calculate is 5E323 you are exceeding
the size of numbers you can process.
Have a happy holiday and glad I could help.
On Fri, Dec 23, 2011 at 11:24 AM, Michael Pearmain
<michael.pearm...@gmail.com> wrote:
> Thanks for al your help with Jim,
> I realise this is really a case of debugging (and so falls to the code
> writer) using your help i have traced the NaN to be from the c1 calculation
> in the FindGradient function.
> there is a place where we use the calculation of c1...
>
> which at the start was calculated as
> c1 <- (2*pi) ** (n.cols/2)
>
> when i use
> c1 <- (2*pi) ** (811/2)
>> [1] Inf
>
> So i think i have found a ceiling on the number of cols i can use to make a
> calcuation.
>
> Many thanks again for your help, especially on 23rd Dec!
>
> Mike
>
>
>
> On 23 December 2011 16:01, jim holtman <jholt...@gmail.com> wrote:
>>
>> The next step is to now evaluate the objects used in the calculation;
>> e.g., look at the objects in the statement:
>>
>> change <- sqrt(sum((next.x - x) * (next.x - x)))
>>
>> What do each of them show? Most likely you have something in the data
>> that causing this statement to evaluate to NaN (not a number) and this
>> is caused by the values of one of these as not being what you think it
>> is. From the value of 'i', it looks like the first iteration (or at
>> least when it has the value of 1). Notice that "X" is defined as a
>> matrix with NAs as the initial values and you are picking up the first
>> row (X[i,]) which on the initial pass is an NA. So start by examining
>> the values of all the objects used up to that point and what the
>> values of each expression is.
>>
>> On Fri, Dec 23, 2011 at 10:38 AM, Michael Pearmain
>> <michael.pearm...@gmail.com> wrote:
>> > Thanks for guidance;
>> >
>> > the return from the dump is below:
>> >
>> > So i can see that the NaN is causing NA, as its evaluating to NA in the
>> > ifelse.
>> > So i guess my next question is why this happens for my large data set
>> > but
>> > not he iris dataset?
>> >
>> > Thank you again for your help on this, i really appreciate you taking
>> > the
>> > time to help
>> >
>> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol =
>> > 1e-05,
>> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> > 4: lapply(X, FUN, ...)
>> > 5: FUN(1:9985[[1]], ...)
>> >
>> > Selection: 5
>> > Called from: function ()
>> > {
>> > if (.isMethodsDispatchOn()) {
>> > tState <- tracingState(FALSE)
>> > on.exit(tracingState(tState))
>> > }
>> > calls <- sys.calls()
>> > from <- 0L
>> > n <- length(calls)
>> > if (identical(sys.function(n), recover))
>> > n <- n - 1L
>> > for (i in rev(seq_len(n))) {
>> > calli <- calls[[i]]
>> > fname <- calli[[1L]]
>> > if (!is.na(match(deparse(fname)[1L], c("methods::.doTrace",
>> > ".doTrace")))) {
>> > from <- i - 1L
>> > break
>> > }
>> > }
>> > if (from == 0L)
>> > for (i in rev(seq_len(n))) {
>> > calli <- calls[[i]]
>> > fname <- calli[[1L]]
>> > if (!is.name(fname) || is.na(match(as.character(fname),
>> > c("recover", "stop", "Stop")))) {
>> > from <- i
>> > break
>> > }
>> > }
>> > if (from > 0L) {
>> > if (!interactive()) {
>> > try(dump.frames())
>> > cat(gettext("recover called non-interactively; frames
>> > dumped,
>> > use debugger() to view\n"))
>> > return(NULL)
>> > }
>> > else if (identical(getOption("show.error.messages"),
>> > FALSE))
>> > return(NULL)
>> > calls <- limitedLabels(calls[1L:from])
>> > repeat {
>> > which <- menu(calls, title = "\nEnter a frame number, or 0
>> > to
>> > exit ")
>> > if (which)
>> > eval(substitute(browser(skipCalls = skip), list(skip = 7
>> > -
>> > which)), envir = sys.frame(which))
>> > else break
>> > }
>> > }
>> > else cat(gettext("No suitable frames for recover()\n"))
>> > }()
>> > Browse[1]> ls()
>> > [1] "change" "find.gradient" "i" "iters"
>> > [5] "next.x" "notconverged" "x"
>> > Browse[1]> change
>> > [1] NaN
>> > Browse[1]> i
>> > [1] 1
>> > Browse[1]>
>> >
>> >
>> > On 23 December 2011 15:22, jim holtman <jholt...@gmail.com> wrote:
>> >>
>> >> According to your dump, if you look in section 5 (this is the function
>> >> being call by 'sapply') you should do an 'ls()' and see the object
>> >> 'notconverged' and if you type its name, I am guessing that is that
>> >> the value NA. Also look at value of 'change' and 'tol' to determine
>> >> how the 'ifelse' was evaluated.
>> >>
>> >> Let me know how it goes.
>> >>
>> >> On Fri, Dec 23, 2011 at 9:35 AM, Michael Pearmain
>> >> <michael.pearm...@gmail.com> wrote:
>> >> > Hi Jim,
>> >> >
>> >> > Sorry to email directly, i wasn't sure what i was looking for, can
>> >> > you
>> >> > give
>> >> > me a little guidance then i can post back to the list? Many thanks
>> >> >
>> >> >> density.data <- DensitiesAndAttractors(data = cluster.data,
>> >> > smoothing = 0.34,
>> >> > tol = 0.00001,
>> >> > parallel = FALSE)
>> >> > density.data <- DensitiesAndAttractors(data = cluster.data,
>> >> > + smoothing = 0.34,
>> >> > + tol = 0.00001,
>> >> > + parallel = FALSE)
>> >> > Explicit sfStop() is missing: stop now.
>> >> > snowfall 1.84 initialized: sequential execution, one CPU.
>> >> >
>> >> > Error in while (notconverged) { : missing value where TRUE/FALSE
>> >> > needed
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
>> >> > =
>> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 1
>> >> > Called from: sfSapply(rep(1:n.rows), function(i) {
>> >> > notconverged <- TRUE
>> >> > x <- (X[i, ])
>> >> > iters <- as.integer(1)
>> >> > while (notconverged) {
>> >> > find.gradient <- FindGradient(x)
>> >> > next.x <- x + stepsize * find.gradient$gradient
>> >> > change <- sqrt(sum((next.x - x) * (next.x - x)))
>> >> > notconverged <- ifelse(change > tol, TRUE, FALSE)
>> >> > x <- next.x
>> >> > iters <- iters + 1
>> >> > }
>> >> > return(c(densities[i, ] <- find.gradient$density, attractors[i,
>> >> > ] <- x))
>> >> > })
>> >> > Browse[1]> ls()
>> >> > [1] "attractors" "c1" "c2" "c3"
>> >> > "cores"
>> >> >
>> >> > [6] "data" "densities" "FindGradient" "Kexp"
>> >> > "n.cols"
>> >> >
>> >> > [11] "n.rows" "parallel" "smoothing" "stepsize"
>> >> > "tol"
>> >> >
>> >> > [16] "X"
>> >> > Browse[1]>
>> >> >
>> >> > Enter a frame number, or 0 to exit
>> >> >
>> >> > 1: DensitiesAndAttractors(data = cluster.data, smoothing = 0.34, tol
>> >> > =
>> >> > 1e-05,
>> >> > 2: denclue_par.R#136: sfSapply(rep(1:n.rows), function(i) {
>> >> > 3: sapply(x, fun, ..., simplify = simplify, USE.NAMES = USE.NAMES)
>> >> > 4: lapply(X, FUN, ...)
>> >> > 5: FUN(1:9985[[1]], ...)
>> >> >
>> >> > Selection: 2
>> >> > Called from: sapply(x, fun, ..., simplify = simplify, USE.NAMES =
>> >> > USE.NAMES)
>> >> > Browse[1]> ls()
>> >> > [1] "fun" "simplify" "USE.NAMES" "x"
>> >> > Browse[1]>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 23 December 2011 13:26, jim holtman <jholt...@gmail.com> wrote:
>> >> >>
>> >> >> Does this look similar to the error you are getting:
>> >> >>
>> >> >> > while(NA == TRUE) 1
>> >> >> Error in while (NA == TRUE) 1 : missing value where TRUE/FALSE
>> >> >> needed
>> >> >>
>> >> >> SO 'notconverged' is probably equal to NA. BTW, what is the value
>> >> >> of
>> >> >> 'tol'; I do not see it defined. So when computing 'notconverged'
>> >> >> you
>> >> >> have generated an NA. You can test it to see if this is true.
>> >> >>
>> >> >> You can use the following command:
>> >> >>
>> >> >> options(error=utils::recover)
>> >> >>
>> >> >> and then learn how to use the 'browser' to examine variables when
>> >> >> the
>> >> >> error occurs.
>> >> >>
>> >> >> On Fri, Dec 23, 2011 at 5:44 AM, Michael Pearmain
>> >> >> <michael.pearm...@gmail.com> wrote:
>> >> >> > Merry Xmas to all,
>> >> >> >
>> >> >> > I am writing a function and curiously this runs sometimes on one
>> >> >> > data
>> >> >> > set
>> >> >> > and fails on another and i cannot figure out why.
>> >> >> > Any help much appreciated.
>> >> >> >
>> >> >> > If i run the code below with
>> >> >> > data <- iris[ ,1:4]
>> >> >> > The code runs fine, but if i run on a large dataset i get the
>> >> >> > following
>> >> >> > error (showing data structures as matrix is large)
>> >> >> >
>> >> >> >> str(cluster.data)
>> >> >> > num [1:9985, 1:811] 0 0 0 0 0 0 0 0 0 0 ...
>> >> >> > - attr(*, "dimnames")=List of 2
>> >> >> > ..$ : NULL
>> >> >> > ..$ : chr [1:811] "1073949105" "1073930585" "1073843224"
>> >> >> > "1073792624"
>> >> >> > ...
>> >> >> > #(This is intended to be chr)
>> >> >> >> str(iris)
>> >> >> > 'data.frame': 150 obs. of 5 variables:
>> >> >> > $ Sepal.Length: num 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
>> >> >> > $ Sepal.Width : num 3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
>> >> >> > $ Petal.Length: num 1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
>> >> >> > $ Petal.Width : num 0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
>> >> >> > $ Species : Factor w/ 3 levels "setosa","versicolor",..: 1 1
>> >> >> > 1 1
>> >> >> > 1
>> >> >> > 1 1
>> >> >> > 1 1 1 ...
>> >> >> >> str(as.matrix(iris[,1:4]))
>> >> >> > num [1:150, 1:4] 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
>> >> >> > - attr(*, "dimnames")=List of 2
>> >> >> > ..$ : NULL
>> >> >> > ..$ : chr [1:4] "Sepal.Length" "Sepal.Width" "Petal.Length"
>> >> >> > "Petal.Width"
>> >> >> >
>> >> >> > n.cols <- ncol(data)
>> >> >> > n.rows <- nrow(data)
>> >> >> > X <- as.matrix(data)
>> >> >> > stepsize <- 0.05
>> >> >> > c1 <- (2 * pi) ** (n.cols / 2)
>> >> >> > c2 <- n.rows * (smoothing ** (n.cols + 2))
>> >> >> > c3 <- n.rows * (smoothing ** n.cols)
>> >> >> >
>> >> >> > Kexp <- function(sqs){
>> >> >> > return (exp((-1 * sqs) / (2 * smoothing ** 2)))
>> >> >> > }
>> >> >> >
>> >> >> > FindGradient <- function(x){
>> >> >> > XmY <- t(x - t(X))
>> >> >> > sqsum <- rowSums(XmY * XmY)
>> >> >> > K <- sapply(sqsum, Kexp)
>> >> >> > dens <- ((c1 * c3) ** -1) * sum(K)
>> >> >> > grad <- -1 * ((c1 * c2) ** -1) * colSums(K * XmY)
>> >> >> > return (list(gradient = grad,
>> >> >> > density = dens))
>> >> >> > }
>> >> >> >
>> >> >> > attractors <- matrix(0, n.rows, n.cols)
>> >> >> > densities <- matrix(0, n.rows)
>> >> >> >
>> >> >> >
>> >> >> >> density.attractors <-
>> >> >> > sapply(rep(1:n.rows), function(i) {
>> >> >> > notconverged <- TRUE
>> >> >> > # For each row loop through and find the attractor and
>> >> >> > density
>> >> >> > value.
>> >> >> > x <- (X[i, ])
>> >> >> > iters <- as.integer(1)
>> >> >> > # Run gradient ascent for each point to obtain x*
>> >> >> > while(notconverged == TRUE) {
>> >> >> > find.gradient <- FindGradient(x)
>> >> >> > next.x <- x + stepsize * find.gradient$gradient
>> >> >> > change <- sqrt(sum((next.x - x) * (next.x - x)))
>> >> >> > notconverged <- ifelse(change > tol, TRUE, FALSE)
>> >> >> > x <- next.x
>> >> >> > iters <- iters + 1
>> >> >> > }
>> >> >> >
>> >> >> > # store the attractor and density value
>> >> >> > return(c(densities[i, ] <- find.gradient$density,
>> >> >> > attractors[i, ] <- x))
>> >> >> > })
>> >> >> >
>> >> >> > Error in while (notconverged == TRUE) { :
>> >> >> > missing value where TRUE/FALSE needed
>> >> >> >>
>> >> >> >
>> >> >> > Any help would be great
>> >> >> >
>> >> >> > Mike
>> >> >> >
>> >> >> > [[alternative HTML version deleted]]
>> >> >> >
>> >> >> > ______________________________________________
>> >> >> > R-help@r-project.org mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/r-help
>> >> >> > PLEASE do read the posting guide
>> >> >> > http://www.R-project.org/posting-guide.html
>> >> >> > and provide commented, minimal, self-contained, reproducible code.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jim Holtman
>> >> >> Data Munger Guru
>> >> >>
>> >> >> What is the problem that you are trying to solve?
>> >> >> Tell me what you want to do, not how you want to do it.
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Jim Holtman
>> >> Data Munger Guru
>> >>
>> >> What is the problem that you are trying to solve?
>> >> Tell me what you want to do, not how you want to do it.
>> >
>> >
>>
>>
>>
>> --
>> Jim Holtman
>> Data Munger Guru
>>
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>
>
--
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
