Re: [Rd] data is getting corrupted
On Tue, 30 Nov 2004 08:01:14 -0800 (PST), "Jeff D. Hamann"
<[EMAIL PROTECTED]> wrote :
>I'm not sure I described the problem correctly with my last post.
Or maybe I missed that. Sorry!
>The structure that is being allocated contains fixed length arrays and
>thus, the location of the strings aren't NULL (the contents are however).
I'd suggest using strncpy instead of strcpy; it might be that one of
your strings is longer than expected, and is overwriting something it
shouldn't.
Duncan Murdoch
>
>struct SAMPLE_RECORD
>{
>charforest[HEADER_CHAR_STR]; /* forest identifier*/
>charsubunit[HEADER_CHAR_STR]; /* subunit */
>charstand_name[HEADER_CHAR_STR]; /* stand name */
>charlegal[HEADER_CHAR_STR];/* legal description*/
>longelevation; /* elevation (ft) */
>double acreage; /* number of acres */
>unsigned long age; /* stand age*/
>
>unsigned long sampled_month; /* month of the year*/
>unsigned long sampled_day; /* day of the month */
>unsigned long sampled_year; /* year measured*/
>unsigned long current_year; /* year of forecast */
>
>double x0;
>
>unsigned long n_points;
>struct PLOT_RECORD *plots_ptr;
>
>unsigned long n_plants;
>struct PLANT_RECORD *plants_ptr;
>};
>
>The plots_ptr and plants_ptr are set to NULL because they're not static
>arrays and are allocated.
>
>Since the problem didn't manifest itself within the strings, but rather in
>the arrays that are allocated (plants_ptr) and specifically a single
>column (variable) that occurs within the plants_ptr, I still think I have
>a problem with the PROTECT/UNPROTECT operations because with small
>data.frames (200 rows x 10 cols) the code works fine. With larger
>data.frames (2000 rows x 10 cols), then the corruption begins. COuld this
>be a problem with the sequencing of the allocate
>plants_ptr->PROTECT->assign values to variables->UNPROTECT? Is it possible
>to find out when R is performing garbage collection or moving data around?
>
>Thanks,
>Jeff.
>
>
>Duncan Murdoch said:
>> On Mon, 29 Nov 2004 09:27:42 -0800 (PST), "Jeff D. Hamann"
>> <[EMAIL PROTECTED]> wrote:
>>
>>>/* this function converts the sample list*/
>>>/* from R into the internal structure*/
>>>struct SAMPLE_RECORD *build_sample_from_sexp( SEXP sample )
>>>{
>>
>> [ declarations deleted ]
>>
>>>
>>> chartemp_sp_code[16];
>>> struct SAMPLE_RECORD *s_ptr;
>>> struct SPECIES_RECORD *sp_ptr;
>>>
>>> s_ptr = (struct SAMPLE_RECORD *)calloc( 1, sizeof( struct
>>> SAMPLE_RECORD
>>>) );
>>
>> This allocates a structure, initialized to all zeros.
>>
>>>/*s_ptr = (struct SAMPLE_RECORD *)Calloc( 1, struct SAMPLE_RECORD );
>>> */
>>>
>>> /* *fill in the header info */
>>> strcpy( s_ptr->forest,
>>>CHAR(STRING_ELT(get_list_element(sample,"forest"), 0)) ) ;
>>
>> This copies the string element to the address s_ptr->forest points to,
>> which is address 0, since you didn't change it from the initial NULL.
>> I'm surprised you didn't get a bigger error than the one you saw.
>>
>> [ more deletions ]
>>
>>>I'm sure there's something I don't understand about the PROTECT/UNPROTECT
>>>sequence as this seems to work on smaller data.frames
>>
>> Maybe you were just lucky that the overwriting at address 0 didn't
>> trash anything in those cases?
>>
>> Duncan Murdoch
>>
__
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] data is getting corrupted
I'm not sure I described the problem correctly with my last post.
The structure that is being allocated contains fixed length arrays and
thus, the location of the strings aren't NULL (the contents are however).
struct SAMPLE_RECORD
{
charforest[HEADER_CHAR_STR]; /* forest identifier*/
charsubunit[HEADER_CHAR_STR]; /* subunit */
charstand_name[HEADER_CHAR_STR]; /* stand name */
charlegal[HEADER_CHAR_STR];/* legal description*/
longelevation; /* elevation (ft) */
double acreage; /* number of acres */
unsigned long age; /* stand age*/
unsigned long sampled_month; /* month of the year*/
unsigned long sampled_day; /* day of the month */
unsigned long sampled_year; /* year measured*/
unsigned long current_year; /* year of forecast */
double x0;
unsigned long n_points;
struct PLOT_RECORD *plots_ptr;
unsigned long n_plants;
struct PLANT_RECORD *plants_ptr;
};
The plots_ptr and plants_ptr are set to NULL because they're not static
arrays and are allocated.
Since the problem didn't manifest itself within the strings, but rather in
the arrays that are allocated (plants_ptr) and specifically a single
column (variable) that occurs within the plants_ptr, I still think I have
a problem with the PROTECT/UNPROTECT operations because with small
data.frames (200 rows x 10 cols) the code works fine. With larger
data.frames (2000 rows x 10 cols), then the corruption begins. COuld this
be a problem with the sequencing of the allocate
plants_ptr->PROTECT->assign values to variables->UNPROTECT? Is it possible
to find out when R is performing garbage collection or moving data around?
Thanks,
Jeff.
Duncan Murdoch said:
> On Mon, 29 Nov 2004 09:27:42 -0800 (PST), "Jeff D. Hamann"
> <[EMAIL PROTECTED]> wrote:
>
>>/* this function converts the sample list */
>>/* from R into the internal structure */
>>struct SAMPLE_RECORD *build_sample_from_sexp( SEXP sample )
>>{
>
> [ declarations deleted ]
>
>>
>> chartemp_sp_code[16];
>> struct SAMPLE_RECORD *s_ptr;
>> struct SPECIES_RECORD *sp_ptr;
>>
>> s_ptr = (struct SAMPLE_RECORD *)calloc( 1, sizeof( struct
>> SAMPLE_RECORD
>>) );
>
> This allocates a structure, initialized to all zeros.
>
>>/*s_ptr = (struct SAMPLE_RECORD *)Calloc( 1, struct SAMPLE_RECORD );
>> */
>>
>> /* *fill in the header info */
>> strcpy( s_ptr->forest,
>>CHAR(STRING_ELT(get_list_element(sample,"forest"), 0)) ) ;
>
> This copies the string element to the address s_ptr->forest points to,
> which is address 0, since you didn't change it from the initial NULL.
> I'm surprised you didn't get a bigger error than the one you saw.
>
> [ more deletions ]
>
>>I'm sure there's something I don't understand about the PROTECT/UNPROTECT
>>sequence as this seems to work on smaller data.frames
>
> Maybe you were just lucky that the overwriting at address 0 didn't
> trash anything in those cases?
>
> Duncan Murdoch
>
--
Jeff D. Hamann
Forest Informatics, Inc.
PO Box 1421
Corvallis, Oregon 97339-1421
phone 541-754-1428
fax 541-752-0288
[EMAIL PROTECTED]
http://www.forestinformatics.com
__
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] data is getting corrupted
On Mon, 29 Nov 2004 09:27:42 -0800 (PST), "Jeff D. Hamann"
<[EMAIL PROTECTED]> wrote:
>/* this function converts the sample list */
>/* from R into the internal structure */
>struct SAMPLE_RECORD *build_sample_from_sexp( SEXP sample )
>{
[ declarations deleted ]
>
> chartemp_sp_code[16];
> struct SAMPLE_RECORD *s_ptr;
> struct SPECIES_RECORD *sp_ptr;
>
> s_ptr = (struct SAMPLE_RECORD *)calloc( 1, sizeof( struct SAMPLE_RECORD
>) );
This allocates a structure, initialized to all zeros.
>/*s_ptr = (struct SAMPLE_RECORD *)Calloc( 1, struct SAMPLE_RECORD ); */
>
> /* *fill in the header info */
> strcpy( s_ptr->forest,
>CHAR(STRING_ELT(get_list_element(sample,"forest"), 0)) ) ;
This copies the string element to the address s_ptr->forest points to,
which is address 0, since you didn't change it from the initial NULL.
I'm surprised you didn't get a bigger error than the one you saw.
[ more deletions ]
>I'm sure there's something I don't understand about the PROTECT/UNPROTECT
>sequence as this seems to work on smaller data.frames
Maybe you were just lucky that the overwriting at address 0 didn't
trash anything in those cases?
Duncan Murdoch
__
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] data is getting corrupted
I've been attempting to perform some analysis on a model that was
interfaced with R (R calls a library that takes SEXPs and converts the
data frames into the internal structures of data), and I notice that for
small data.frames the vectors don't get corrupt (n<200-ish). When I pass
in larger data.frames, the vectors will become corrupt. I've been
PROTECTING the heck out of everything (as best as I can from the examples)
to make sure that something is not overlooked. I know the code in my
library works fine becuase when I attempt to do the same thing (with much
larger data arrays) none of this behaviour occurs.
An example of the corruption is,
1 1714 ARPA 0.00 0. 0.00 0. 3.64 0.000 1
20.00 0.00 0.00 0
1 1715 ARPA 0.00 0. 0.00 0. 3.14 0.000 1
20.00 0.00 0.00 0
1 1716 ARPA
97538806975312948000.00
0. 0.00 0. 4.68 0.000 120.00 0.00 0.00
0
1 1717 ARPA 0.00 0. 0.00 0. 2.50 0.000 1
20.00 0.00 0.00 0
1 1718 ARPA 0.00 0. 0.00 0. 4.78 0.000 1
20.00 0.00 0.00 0
1 1719 ARPA 0.00 0. 0.00 0. 4.04 0.000 1
20.00 0.00 0.00 0
1 1720 ARPA 0.00 0. 0.00 0. 2.60 0.000 1
20.00 0.00 0.00 0
1 1721 ARPA
11415665383569361000.00
0. 0.00 0. 3.57 0.000 120.00 0.00 0.00
0
1 1722 ARPA 0.00 0. 0.00 0. 2.29 0.000 1
20.00 0.00 0.00 0
And it's the same columns that become corrupt. I wanted to make sure I'm
using PROTECT correctly (the examples in the docs don't appear very
thourough) and included a snippet below,
SEXP r_write_sample_to_file( SEXP sample_in,
SEXP filename )
{
unsigned long return_code;
struct SAMPLE_RECORD *sample_ptr;
SEXP ans;
PROTECT( filename = AS_CHARACTER( filename ) );
PROTECT(ans = allocVector(INTSXP, 1));
PROTECT( sample_in = AS_LIST( sample_in ) );
sample_ptr = build_sample_from_sexp( sample_in );
write_sample_to_file(
&return_code,
CHAR(STRING_ELT(filename, 0)),
sample_ptr,
N_SPECIES,
SPECIES_PTR );
if( return_code != CONIFERS_SUCCESS )
{
Rprintf( "unable to write %s\n", CHAR(STRING_ELT(filename, 0)) );
INTEGER(ans)[0] = -1;
UNPROTECT(2);
UNPROTECT( 1 );
return ans;
}
INTEGER(ans)[0] = 0;
UNPROTECT(2);
UNPROTECT( 1 );
return ans;
}
which calls this rather lengthy function (but I thought I should include
the entire function for completeness),
/* this function converts the sample list */
/* from R into the internal structure */
struct SAMPLE_RECORD *build_sample_from_sexp( SEXP sample )
{
int i;
/* plots variables */
SEXP plot_list;
SEXP plot_plot_sexp;
SEXP plot_lat_sexp;
SEXP plot_long_sexp;
SEXP plot_elev_sexp;
SEXP plot_slp_sexp;
SEXP plot_asp_sexp;
SEXP plot_h20_sexp;
SEXP plot_map_sexp;
/* plants variables */
SEXP plant_list;
SEXP plant_plot_sexp;
SEXP plant_plant_sexp;
SEXP plant_sp_code_sexp;
SEXP plant_d6_sexp;
SEXP plant_d6_area_sexp;
SEXP plant_dbh_sexp;
SEXP plant_basal_area_sexp;
SEXP plant_tht_sexp;
SEXP plant_cr_sexp;
SEXP plant_n_stems_sexp;
SEXP plant_expf_sexp;
SEXP plant_crown_width_sexp;
SEXP plant_crown_area_sexp;
SEXP plant_user_code_sexp;
chartemp_sp_code[16];
struct SAMPLE_RECORD *s_ptr;
struct SPECIES_RECORD *sp_ptr;
s_ptr = (struct SAMPLE_RECORD *)calloc( 1, sizeof( struct SAMPLE_RECORD
) );
/*s_ptr = (struct SAMPLE_RECORD *)Calloc( 1, struct SAMPLE_RECORD ); */
/* *fill in the header info */
strcpy( s_ptr->forest,
CHAR(STRING_ELT(get_list_element(sample,"forest"), 0)) ) ;
strcpy( s_ptr->subunit,
CHAR(STRING_ELT(get_list_element(sample,"subunit"), 0)) );
strcpy( s_ptr->stand_name,
CHAR(STRING_ELT(get_list_element(sample,"stand.name"), 0)) );
strcpy( s_ptr->legal, CHAR(STRING_ELT(get_list_element(sample,"legal"),
0)) );
s_ptr->elevation = asInteger( get_list_element( sample, "elevation" ) );
s_ptr->acreage = asReal( get_list_element( sample, "acreage" ) );
s_ptr->age = asInteger( get_list_element( sample, "age" ) );
s_ptr->sampled_month = asInteger( get_list_element( sample,
"sampled.month" ) );
s_ptr->sampled_day = asInteger( get_list_element( sample, "sampled.day"
) );
s_ptr->sampled_year = asInteger( get_list_element
