Ok I get it. So everytime it does a alloc and copy.
I haven't finished the design yet. I'm just thinking about how randomly the
data might arrive; its real time data. So I will allocate a large chunk of
memory and keep track of when it fills up, once the data exceeds I will
alloc and copy the data (provided the size is within system limits). In
this manner I should be able to reduce the number of expensive operations of
allocing and copying.
Thanks for your input.
Regards
Abhijit Bera
On Sat, Oct 24, 2009 at 10:06 PM, Douglas Bates <ba...@stat.wisc.edu> wrote:
> On Fri, Oct 23, 2009 at 2:02 PM, Abhijit Bera <abhib...@gmail.com> wrote:
> > Sorry, I made a mistake while writing the code. The declaration of Data
> > should have been first.
>
> > I still have some doubts:
>
> Because you are making some sweeping and incorrect assumptions about
> the way that the internals of R operate. R allows for arrays to be
> dynamically resized but this is accomplished internally by allocating
> new storage, copying the current contents to this new location and
> installing the values of the new elements. It is an expensive
> operation, which is why it is discouraged.
>
> Your design is deeply flawed. Go back to the drawing board.
>
> > When you say calloc and realloc are you talking about R's C interface
> Calloc
> > and Realloc or the regular calloc and realloc?
>
> Either one.
>
> > I want to feed data directly into a R matrix and grow it as required. So
> at
> > one time I might have 100 rows coming in from a data source. The next
> time I
> > might have 200 rows coming in from a data source. I want to be able to
> > expand the R-matrix instead of creating a regular C float matrix and then
> > make an R-matrix based on the new size. I just want to have one R object
> and
> > be able to expand it's size dynamically.
>
> R stores floating-point numbers as the C data type double, not float.
> It may seem pedantic to point out distinctions like that but not when
> you are writing programs. Compilers are the ultimate pedants - they
> are real sticklers for getting the details right.
>
> As I said, it just doesn't work the way that you think it does. The
> fact that there is an R object with a certain name before and after an
> operation doesn't mean it is the same R object.
>
> > I was reading the language specs. It says that one could declare an
> object
> > in R like this:
> >
> > m=matrix(nrows=10,ncols=10)
> >
> > and then one could assign
> >
> > m[101]=1.00
> >
> > to expand the object.
> >
> > but this has one problem when I do a
> >
> > dim(m)
> >
> > I get
> >
> > NULL instead of 10 10
> >
> > So what is happening here?
> >
> >
> > I am aware that R matrices are stored in column major order.
> >
> > Thanks for the tip on using float *dat= REAL(Data);
> >
> > Regards
> >
> > Abhijit Bera
> >
> >
> >
> > On Fri, Oct 23, 2009 at 7:27 PM, Douglas Bates <ba...@stat.wisc.edu>
> wrote:
> >>
> >> On Fri, Oct 23, 2009 at 9:23 AM, Douglas Bates <ba...@stat.wisc.edu>
> >> wrote:
> >> > On Fri, Oct 23, 2009 at 8:39 AM, Abhijit Bera <abhib...@gmail.com>
> >> > wrote:
> >> >> Hi
> >> >>
> >> >> I'm having slight confusion.
> >> >
> >> > Indeed.
> >> >
> >> >> I plan to grow/realloc a matrix depending on the data available in a
> C
> >> >> program.
> >> >
> >> >> Here is what I'm tried to do:
> >> >
> >> >> Data=allocMatrix(REALSXP,3,4);
> >> >> SEXP Data;
> >> >
> >> > Those lines should be in the other order, shouldn't they?
> >> >
> >> > Also, you need to PROTECT Data or bad things will happen.
> >> >
> >> >> REAL(Data)[8]=0.001123;
> >> >> REAL(Data)[200000]=0.001125;
> >> >> printf("%f %f\n\n\n\n",REAL(Data)[8],REAL(Data)[200000]);
> >>
> >> And I forgot to mention, it is not a good idea to write REAL(Data)
> >> many times like this. REAL is a function, not a macro and you are
> >> calling the same function over and over again unnecessarily. It is
> >> better to write
> >>
> >> double *dat = REAL(Data);
> >>
> >> and use the dat pointer instead of REAL(Data).
> >>
> >> >> Here is my confusion:
> >> >
> >> >> Do I always require to allocate the exact number of data elements in
> a
> >> >> R
> >> >> Matrix?
> >> >
> >> > Yes.
> >> >
> >> >> In the above code segment I have clearly exceeded the number of
> >> >> elements that have been allocated but my program doesn't crash.
> >> >
> >> > Remember that when programming in C you have a lot of rope with which
> >> > to hang yourself. You have corrupted a memory location beyond that
> >> > allocated to the array but nothing bad has happened - yet.
> >> >
> >> >> I don't find any specific R functions for reallocation incase my data
> >> >> set
> >> >> grows. How do I reallocate?
> >> >
> >> > You allocate a new matrix, copy the contents of the current matrix to
> >> > the new matrix, then release the old one. It gets tricky in that you
> >> > should unprotect the old one and protect the new one but you need to
> >> > watch the order of those operations.
> >> >
> >> > This approach is not a very good one. If you really need to grow an
> >> > array it is better to allocate and reallocate the memory within your C
> >> > code using calloc and realloc then, at the end of the calculations,
> >> > allocate an R matrix and copy the results over.
> >> >
> >> > Also, you haven't said whether you are growing the matrix by row or by
> >> > column or both. If you are adding rows then you can't just reallocate
> >> > storage because R stores matrices in column-major order. The positions
> >> > of the elements in a matrix with n+1 rows are different from those in
> >> > a matrix with n rows.
> >> >
> >> >> Is it necessary to reallocate or is R handling
> >> >> the memory management for the matrix that I have allocated?
> >> >>
> >> >> Regards
> >> >>
> >> >> Abhijit Bera
> >> >>
> >> >> [[alternative HTML version deleted]]
> >> >>
> >> >> ______________________________________________
> >> >> R-devel@r-project.org mailing list
> >> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >> >>
> >> >
> >
> >
>
[[alternative HTML version deleted]]
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
