i've got a text file with the following format:
-----------------------------------------------
int
double double
double double
double double
END
int
double double
double double
END
int
double double
double double
double double
double double
END
END
-----------------------------------------------
which i want to read into an array of the following structures
typedef struct
{
double x;
double y;
} POINT;
struct
{
int ID;
int numPoints;
POINT *pasPoints;
} MYOBJECT;
ie in a MYOBJECT struct with reference to the input file, the int is the
ID, and the double's are x and y pairs that make up the array of POINT
structs. There can be an arbitrary number of ID's in a file hence an
arbitrary sized array of MYOBJECT structs. For each ID there can be an
arbitrary number of double pairs therefore an arbitrary sized array of
POINT structs in each MYOBJECT struct.
Since the input files are often large (10meg, 50meg, 100meg not unusual) i
need to optimize the way in which i read in this data.
Possible methods that come to mind are:
- simply malloc an arbitrary sized array of MYOBJECT's, each with
an arbitrary sized array of POINTS, then use fgets() and
sscanf() to read in the data, realloc()'ing when memory
allocated is insufficient.
- use a binary fread() and memchr() to work out how many 'N' chars
and hence how many "END"'s and hence how many MYOBJECTS there
are. Use fread() and memchr() again to work out how many '\n'
chars there are between each "END" hence how many POINT structs
to allocate for each MYOBJECT. Then use fgets() and sscanf() to
read the data into the structs.
Both of these methods seem to be very inefficient, either do to constant
realloc()'ing or re-reading the same (large) file for different info. Also
are fgets() and sscanf() the best way to read in data of this form ? I'd
like to be able to do everything using fread(), memchr() with no
realloc()'ing, perhap's with a getnextword() function in combination with
atod() - is this possible and if so is it any quicker ?
thanx in advance (and for reading this far :),
Novak.