On Oct 24, 2009, at 2:58 PM, Abhijit Bera wrote:
Ok I get it. So everytime it does a alloc and copy.
I haven't finished the design yet. I'm just thinking about how
randomly the data might arrive; its real time data. So I will
allocate a large chunk of memory and keep track of when it fills up,
once the data exceeds I will alloc and copy the data (provided the
size is within system limits). In this manner I should be able to
reduce the number of expensive operations of allocing and copying.
Many smart people have thought about those things before you, it's
worthwhile to read about it --- I would suggest reading a bit about
data structures and programming in C. What you describe is usually
tackled by a allocating additional (usually linked) buffers as you go
since that means you don't have to copy anything (except for that last
step where you create the R object). It's also very trivial to
implement.
Cheers,
Simon
Abhijit Bera
On Sat, Oct 24, 2009 at 10:06 PM, Douglas Bates
<ba...@stat.wisc.edu> wrote:
On Fri, Oct 23, 2009 at 2:02 PM, Abhijit Bera <abhib...@gmail.com>
wrote:
Sorry, I made a mistake while writing the code. The declaration of
Data
should have been first.
I still have some doubts:
Because you are making some sweeping and incorrect assumptions about
the way that the internals of R operate. R allows for arrays to be
dynamically resized but this is accomplished internally by allocating
new storage, copying the current contents to this new location and
installing the values of the new elements. It is an expensive
operation, which is why it is discouraged.
Your design is deeply flawed. Go back to the drawing board.
When you say calloc and realloc are you talking about R's C
interface
Calloc
and Realloc or the regular calloc and realloc?
Either one.
I want to feed data directly into a R matrix and grow it as
required. So
at
one time I might have 100 rows coming in from a data source. The
next
time I
might have 200 rows coming in from a data source. I want to be
able to
expand the R-matrix instead of creating a regular C float matrix
and then
make an R-matrix based on the new size. I just want to have one R
object
and
be able to expand it's size dynamically.
R stores floating-point numbers as the C data type double, not float.
It may seem pedantic to point out distinctions like that but not when
you are writing programs. Compilers are the ultimate pedants - they
are real sticklers for getting the details right.
As I said, it just doesn't work the way that you think it does. The
fact that there is an R object with a certain name before and after
an
operation doesn't mean it is the same R object.
I was reading the language specs. It says that one could declare an
object
in R like this:
m=matrix(nrows=10,ncols=10)
and then one could assign
m[101]=1.00
to expand the object.
but this has one problem when I do a
dim(m)
I get
NULL instead of 10 10
So what is happening here?
I am aware that R matrices are stored in column major order.
Thanks for the tip on using float *dat= REAL(Data);
Regards
Abhijit Bera
On Fri, Oct 23, 2009 at 7:27 PM, Douglas Bates <ba...@stat.wisc.edu>
wrote:
On Fri, Oct 23, 2009 at 9:23 AM, Douglas Bates
<ba...@stat.wisc.edu>
wrote:
On Fri, Oct 23, 2009 at 8:39 AM, Abhijit Bera <abhib...@gmail.com>
wrote:
Hi
I'm having slight confusion.
Indeed.
I plan to grow/realloc a matrix depending on the data available
in a
C
program.
Here is what I'm tried to do:
Data=allocMatrix(REALSXP,3,4);
SEXP Data;
Those lines should be in the other order, shouldn't they?
Also, you need to PROTECT Data or bad things will happen.
REAL(Data)[8]=0.001123;
REAL(Data)[200000]=0.001125;
printf("%f %f\n\n\n\n",REAL(Data)[8],REAL(Data)[200000]);
And I forgot to mention, it is not a good idea to write REAL(Data)
many times like this. REAL is a function, not a macro and you are
calling the same function over and over again unnecessarily. It is
better to write
double *dat = REAL(Data);
and use the dat pointer instead of REAL(Data).
Here is my confusion:
Do I always require to allocate the exact number of data
elements in
a
R
Matrix?
Yes.
In the above code segment I have clearly exceeded the number of
elements that have been allocated but my program doesn't crash.
Remember that when programming in C you have a lot of rope with
which
to hang yourself. You have corrupted a memory location beyond
that
allocated to the array but nothing bad has happened - yet.
I don't find any specific R functions for reallocation incase
my data
set
grows. How do I reallocate?
You allocate a new matrix, copy the contents of the current
matrix to
the new matrix, then release the old one. It gets tricky in
that you
should unprotect the old one and protect the new one but you
need to
watch the order of those operations.
This approach is not a very good one. If you really need to
grow an
array it is better to allocate and reallocate the memory within
your C
code using calloc and realloc then, at the end of the
calculations,
allocate an R matrix and copy the results over.
Also, you haven't said whether you are growing the matrix by row
or by
column or both. If you are adding rows then you can't just
reallocate
storage because R stores matrices in column-major order. The
positions
of the elements in a matrix with n+1 rows are different from
those in
a matrix with n rows.
Is it necessary to reallocate or is R handling
the memory management for the matrix that I have allocated?
Regards
Abhijit Bera
[[alternative HTML version deleted]]
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[[alternative HTML version deleted]]
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
