Re: [R] Memory not release when an environment is created

2016-09-22 Thread peter dalgaard 

> On 22 Sep 2016, at 21:24 , luke-tier...@uiowa.edu wrote:
> 
> On Thu, 22 Sep 2016, luke-tier...@uiowa.edu wrote:
> 
>> My preference is to use a top level function in the package or global
>> env that takes as arguments just the variables I want in the parent
>> frame. That avoids the explicit environment manipulations. Here that
>> would be
>> 
>>> makeFunc0 <- function(xmin, xmax)
>> function(y) (y - xmin) / (xmax - xmin)
>> 
> 
> But I do keep forgetting the need to force the parameters if you don't
> want a big value to stick around until the returned function is used
> the first time,

...not to mention the pains that arise if the makeFunc0 gets called with 
argument expressions that involve items that may change before the first call 
of the returned function.

> so a better definition of makeFUnc0 is
> 
> makeFunc0 <- function(xmin, xmax) {
>force(xmin)
>force(xmax)
>function(y) (y - xmin) / (xmax - xmin)
> }
> 
> Best,
> 
> luke
> 
>>> makeFunc1 <- function(x)
>> makeFunc0(min(x), max(x))
>> 
>>> f <- makeFunc1(1:1e8)
>>> ls.str(all=TRUE, environment(f))
>> xmax :  int 1
>> xmin :  int 1
>>> parent.env(environment(f))
>> 
>>> f(c(1234567, 2345678))
>> [1] 0.01234566 0.02345677
>> 
>> Best,
>> 
>> luke
>> 
>> 
>> On Thu, 22 Sep 2016, William Dunlap via R-help wrote:
>> 
>>> I like to have my function-returning functions use new.env(parent=XXX)
>>> to make an environment for the returned function and put into it only
>>> the objects needed by the function.  The 'XXX' should be a an environment
>>> which will hang around anyway.  It could be globalenv(), but if your
>>> function
>>> is in a package, as.environment(paste0("package:", .packageName))
>>> would work well.  The later ensures the your returned function has access
>>> to all the other functions in that package.
>>> E.g.,
 makeFunc1 <- function(x) {
>>>   envir <- new.env(parent = environment(sys.function()))
>>>   envir$xmax <- max(x)
>>>   envir$xmin <- min(x)
>>>   with(envir, function(y) (y - xmin) / (xmax - xmin))
>>> }
 f <- makeFunc1(1:1e8)
 ls.str(all=TRUE, environment(f))
>>> xmax :  int 1
>>> xmin :  int 1
 parent.env(environment(f))
>>> 
 f(c(1234567, 2345678))
>>> [1] 0.01234566 0.02345677
>>> Bill Dunlap
>>> TIBCO Software
>>> wdunlap tibco.com
>>> On Thu, Sep 22, 2016 at 8:41 AM, Olivier Merle 
>>> wrote:
 Dear,
 When I use big data for a temporary use it seems that the memory is not
 released when a function/environement is created nearby.
 Here the reproducible exemple:
 test<-function(){
 x=matrix(0,5,1)
 y=function(nb) nb^2
 return(y)
 }
 xx=test() # 3 Go of Ram is used
 gc() # Memory is not released !! even if x has been destroyed [look into
 software mem used]
 format(object.size(xx),units="auto") # 1.4 KiB => R is worng on the size
 of
 the object
 rm(xx)
 gc() # Memory is released
 ## Classic
 test2<-function(){
 x=matrix(0,5,1)
 y=1
 return(y)
 }
 xx=test2() # Memory is used
 gc() # => Memory is released
 How can I release the data in test without destroying the xx object ? As x
 which is big object is destroyed, I though I could get my memory back but
 it seems that the function y is keeping the x object.
 Best
 
[[alternative HTML version deleted]]
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/
 posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
>>> 
>>> [[alternative HTML version deleted]]
>>> __
>>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>> 
>> 
> 
> -- 
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa  Phone: 319-335-3386
> Department of Statistics andFax:   319-335-3017
>   Actuarial Science
> 241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
> Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A

Re: [R] Memory not release when an environment is created

2016-09-22 Thread luke-tierney 

On Thu, 22 Sep 2016, luke-tier...@uiowa.edu wrote:


My preference is to use a top level function in the package or global
env that takes as arguments just the variables I want in the parent
frame. That avoids the explicit environment manipulations. Here that
would be


makeFunc0 <- function(xmin, xmax)

 function(y) (y - xmin) / (xmax - xmin)



But I do keep forgetting the need to force the parameters if you don't
want a big value to stick around until the returned function is used
the first time, so a better definition of makeFUnc0 is

makeFunc0 <- function(xmin, xmax) {
force(xmin)
force(xmax)
function(y) (y - xmin) / (xmax - xmin)
}

Best,

luke


makeFunc1 <- function(x)

 makeFunc0(min(x), max(x))


f <- makeFunc1(1:1e8)
ls.str(all=TRUE, environment(f))

xmax :  int 1
xmin :  int 1

parent.env(environment(f))



f(c(1234567, 2345678))

[1] 0.01234566 0.02345677

Best,

luke


On Thu, 22 Sep 2016, William Dunlap via R-help wrote:


I like to have my function-returning functions use new.env(parent=XXX)
to make an environment for the returned function and put into it only
the objects needed by the function.  The 'XXX' should be a an environment
which will hang around anyway.  It could be globalenv(), but if your
function
is in a package, as.environment(paste0("package:", .packageName))
would work well.  The later ensures the your returned function has access
to all the other functions in that package.

E.g.,

makeFunc1 <- function(x) {

   envir <- new.env(parent = environment(sys.function()))
   envir$xmax <- max(x)
   envir$xmin <- min(x)
   with(envir, function(y) (y - xmin) / (xmax - xmin))
}

f <- makeFunc1(1:1e8)
ls.str(all=TRUE, environment(f))

xmax :  int 1
xmin :  int 1

parent.env(environment(f))



f(c(1234567, 2345678))

[1] 0.01234566 0.02345677



Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Thu, Sep 22, 2016 at 8:41 AM, Olivier Merle 
wrote:


Dear,

When I use big data for a temporary use it seems that the memory is not
released when a function/environement is created nearby.
Here the reproducible exemple:

test<-function(){
x=matrix(0,5,1)
y=function(nb) nb^2
return(y)
}
xx=test() # 3 Go of Ram is used
gc() # Memory is not released !! even if x has been destroyed [look into
software mem used]
format(object.size(xx),units="auto") # 1.4 KiB => R is worng on the size
of
the object
rm(xx)
gc() # Memory is released

## Classic
test2<-function(){
x=matrix(0,5,1)
y=1
return(y)
}
xx=test2() # Memory is used
gc() # => Memory is released

How can I release the data in test without destroying the xx object ? As x
which is big object is destroyed, I though I could get my memory back but
it seems that the function y is keeping the x object.

Best

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/
posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.






--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
   Actuarial Science
241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory not release when an environment is created

2016-09-22 Thread luke-tierney 

My preference is to use a top level function in the package or global
env that takes as arguments just the variables I want in the parent
frame. That avoids the explicit environment manipulations. Here that
would be


makeFunc0 <- function(xmin, xmax)

  function(y) (y - xmin) / (xmax - xmin)


makeFunc1 <- function(x)

  makeFunc0(min(x), max(x))


f <- makeFunc1(1:1e8)
ls.str(all=TRUE, environment(f))

xmax :  int 1
xmin :  int 1

parent.env(environment(f))



f(c(1234567, 2345678))

[1] 0.01234566 0.02345677

Best,

luke


On Thu, 22 Sep 2016, William Dunlap via R-help wrote:


I like to have my function-returning functions use new.env(parent=XXX)
to make an environment for the returned function and put into it only
the objects needed by the function.  The 'XXX' should be a an environment
which will hang around anyway.  It could be globalenv(), but if your
function
is in a package, as.environment(paste0("package:", .packageName))
would work well.  The later ensures the your returned function has access
to all the other functions in that package.

E.g.,

makeFunc1 <- function(x) {

   envir <- new.env(parent = environment(sys.function()))
   envir$xmax <- max(x)
   envir$xmin <- min(x)
   with(envir, function(y) (y - xmin) / (xmax - xmin))
}

f <- makeFunc1(1:1e8)
ls.str(all=TRUE, environment(f))

xmax :  int 1
xmin :  int 1

parent.env(environment(f))



f(c(1234567, 2345678))

[1] 0.01234566 0.02345677



Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Thu, Sep 22, 2016 at 8:41 AM, Olivier Merle 
wrote:


Dear,

When I use big data for a temporary use it seems that the memory is not
released when a function/environement is created nearby.
Here the reproducible exemple:

test<-function(){
x=matrix(0,5,1)
y=function(nb) nb^2
return(y)
}
xx=test() # 3 Go of Ram is used
gc() # Memory is not released !! even if x has been destroyed [look into
software mem used]
format(object.size(xx),units="auto") # 1.4 KiB => R is worng on the size
of
the object
rm(xx)
gc() # Memory is released

## Classic
test2<-function(){
x=matrix(0,5,1)
y=1
return(y)
}
xx=test2() # Memory is used
gc() # => Memory is released

How can I release the data in test without destroying the xx object ? As x
which is big object is destroyed, I though I could get my memory back but
it seems that the function y is keeping the x object.

Best

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/
posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



--
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
   Actuarial Science
241 Schaeffer Hall  email:   luke-tier...@uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory not release when an environment is created

2016-09-22 Thread William Dunlap via R-help 
I like to have my function-returning functions use new.env(parent=XXX)
to make an environment for the returned function and put into it only
the objects needed by the function.  The 'XXX' should be a an environment
which will hang around anyway.  It could be globalenv(), but if your
function
is in a package, as.environment(paste0("package:", .packageName))
would work well.  The later ensures the your returned function has access
to all the other functions in that package.

E.g.,
> makeFunc1 <- function(x) {
envir <- new.env(parent = environment(sys.function()))
envir$xmax <- max(x)
envir$xmin <- min(x)
with(envir, function(y) (y - xmin) / (xmax - xmin))
}
> f <- makeFunc1(1:1e8)
> ls.str(all=TRUE, environment(f))
xmax :  int 1
xmin :  int 1
> parent.env(environment(f))

> f(c(1234567, 2345678))
[1] 0.01234566 0.02345677



Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Thu, Sep 22, 2016 at 8:41 AM, Olivier Merle 
wrote:

> Dear,
>
> When I use big data for a temporary use it seems that the memory is not
> released when a function/environement is created nearby.
> Here the reproducible exemple:
>
> test<-function(){
> x=matrix(0,5,1)
> y=function(nb) nb^2
> return(y)
> }
> xx=test() # 3 Go of Ram is used
> gc() # Memory is not released !! even if x has been destroyed [look into
> software mem used]
> format(object.size(xx),units="auto") # 1.4 KiB => R is worng on the size
> of
> the object
> rm(xx)
> gc() # Memory is released
>
> ## Classic
> test2<-function(){
> x=matrix(0,5,1)
> y=1
> return(y)
> }
> xx=test2() # Memory is used
> gc() # => Memory is released
>
> How can I release the data in test without destroying the xx object ? As x
> which is big object is destroyed, I though I could get my memory back but
> it seems that the function y is keeping the x object.
>
> Best
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory not release when an environment is created

2016-09-22 Thread David Winsemius 

> On Sep 22, 2016, at 9:45 AM, Ismail SEZEN  wrote:
> 
> 
>> On 22 Sep 2016, at 18:41, Olivier Merle  wrote:
>> 
>> Dear,
>> 
>> When I use big data for a temporary use it seems that the memory is not
>> released when a function/environement is created nearby.
>> Here the reproducible exemple:
>> 
>> test<-function(){
>> x=matrix(0,5,1)
>> y=function(nb) nb^2
>> return(y)
>> }
>> xx=test() # 3 Go of Ram is used
>> gc() # Memory is not released !! even if x has been destroyed [look into
>> software mem used]
> 
> Because y is a function and returns with its own environment.
> 
> ls(environment(xx)) # x and y objects are still there
> 
>> How can I release the data in test without destroying the xx object ? As x
>> which is big object is destroyed, I though I could get my memory back but
>> it seems that the function y is keeping the x object.
> 
> if you do not need the x object in y function then remove it in it’s own 
> environment as follows;
> 
>> test<-function(){
>> x=matrix(0,5,1)
>   rm(x)
>> y=function(nb) nb^2
>> return(y)
>> }
> 
> or if you need to remove it out of the function;
> 
> rm("x", envir = environment(xx))
> ls(environment(xx)) # x has gone

That's much clearer than my `eval(quote(...` approach. I had forgotten that 
`rm` had an 'environment' parameter. I had considered trying:

environment(xx)$x <- NULL   # but after looking at `environment`'s help page I 
was pretty sure it would have failed 

# But there again I was wrong:

test<-function(){
x=matrix(0,500,100)
y=function(nb) nb^2
return(y)
}
xx=test()

environment(xx)$x <- NULL


> object.size( get("x", envir=environment(xx) ) )
0 bytes

I still think rm() is the way to go but this offers another illustration of 
available methods of environment mangling, er, manipulation.

Best;
David.


> 
> If y function uses x somehow, then you will need to live with a big object.
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory not release when an environment is created

2016-09-22 Thread David Winsemius 

> On Sep 22, 2016, at 8:41 AM, Olivier Merle  wrote:
> 
> Dear,
> 
> When I use big data for a temporary use it seems that the memory is not
> released when a function/environement is created nearby.
> Here the reproducible exemple:
> 
> test<-function(){
> x=matrix(0,5,1)
> y=function(nb) nb^2
> return(y)
> }
> xx=test() # 3 Go of Ram is used
> gc() # Memory is not released !! even if x has been destroyed [look into
> software mem used]

Looking at this I would imagine that the unreleased memory is allocated within 
the `xx` objects environment (since functions in R are actually closures that 
carry along their environments of creation.


> format(object.size(xx),units="auto") # 1.4 KiB => R is worng on the size of
> the object
> rm(xx)
> gc() # Memory is released
> 
> ## Classic
> test2<-function(){
> x=matrix(0,5,1)
> y=1
> return(y)
> }
> xx=test2() # Memory is used
> gc() # => Memory is released
> 
> How can I release the data in test without destroying the xx object ? As x
> which is big object is destroyed, I though I could get my memory back but
> it seems that the function y is keeping the x object.

That's how I understand the semantics of R. You would need to replace the 
environment that is attached to `xx`. I madesomwaht smaller object:

test<-function(
x=matrix(0,500,
y=function(nb) 
return(y)
}
xx=test() 

> environment(xx)


> object.size(xx)
1384 bytes
> ls(x, envir=environment(xx) )
[1] "x" "y"
> ?get
> object.size( get("x", envir=environment(xx) ) )
400200 bytes

Even with that perspective in mind it still took me two tries to get rid of the 
xx object, since my abilities to "program on the language" are still fairly 
modest.:

> eval( x <- NULL, envir=environment(xx) )
NULL
> object.size( get("x", envir=environment(xx) ) )
400200 bytes
> eval( quote(x <- NULL), envir=environment(xx) )
> object.size( get("x", envir=environment(xx) ) )
0 bytes

So now I have learned that object.size does not include measurements of the 
size of function environments, a fact about which I was not aware.

Best;
David.

> 
> Best
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Memory not release when an environment is created

2016-09-22 Thread Ismail SEZEN 

> On 22 Sep 2016, at 18:41, Olivier Merle  wrote:
> 
> Dear,
> 
> When I use big data for a temporary use it seems that the memory is not
> released when a function/environement is created nearby.
> Here the reproducible exemple:
> 
> test<-function(){
> x=matrix(0,5,1)
> y=function(nb) nb^2
> return(y)
> }
> xx=test() # 3 Go of Ram is used
> gc() # Memory is not released !! even if x has been destroyed [look into
> software mem used]

Because y is a function and returns with its own environment.

ls(environment(xx)) # x and y objects are still there

> How can I release the data in test without destroying the xx object ? As x
> which is big object is destroyed, I though I could get my memory back but
> it seems that the function y is keeping the x object.

if you do not need the x object in y function then remove it in it’s own 
environment as follows;

> test<-function(){
> x=matrix(0,5,1)
   rm(x)
> y=function(nb) nb^2
> return(y)
> }

or if you need to remove it out of the function;

rm("x", envir = environment(xx))
ls(environment(xx)) # x has gone

 If y function uses x somehow, then you will need to live with a big object.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.