i've got a text file with the following format:

-----------------------------------------------
int
        double  double
        double  double
        double  double
END
int
        double  double
        double  double
END
int
        double  double
        double  double
        double  double
        double  double
END
END
-----------------------------------------------

which i want to read into an array of the following structures

typedef struct
{
        double  x;
        double  y;
} POINT;

struct
{
        int     ID;
        int     numPoints;
        POINT   *pasPoints;
} MYOBJECT;

ie in a MYOBJECT struct with reference to the input file, the int is the
ID, and the double's are x and y pairs that make up the array of POINT
structs. There can be an arbitrary number of ID's in a file hence an
arbitrary sized array of MYOBJECT structs. For each ID there can be an
arbitrary number of double pairs therefore an arbitrary sized array of
POINT structs in each MYOBJECT struct.

Since the input files are often large (10meg, 50meg, 100meg not unusual) i
need to optimize the way in which i read in this data.

Possible methods that come to mind are:

        - simply malloc an arbitrary sized array of MYOBJECT's, each with
          an arbitrary sized array of POINTS, then use fgets() and
          sscanf() to read in the data, realloc()'ing when memory
          allocated is insufficient.

        - use a binary fread() and memchr() to work out how many 'N' chars
          and hence how many "END"'s and hence how many MYOBJECTS there
          are. Use fread() and memchr() again to work out how many '\n'
          chars there are between each "END" hence how many POINT structs
          to allocate for each MYOBJECT. Then use fgets() and sscanf() to
          read the data into the structs.

Both of these methods seem to be very inefficient, either do to constant
realloc()'ing or re-reading the same (large) file for different info. Also
are fgets() and sscanf() the best way to read in data of this form ? I'd
like to be able to do everything using fread(), memchr() with no
realloc()'ing, perhap's with a getnextword() function in combination with
atod() - is this possible and if so is it any quicker ?

thanx in advance (and for reading this far :),
Novak.

Reply via email to