On 30 Jan 2002 10:06:35 -0800, Jeff Zucker wrote:
>Given a web context and a small dataset without mod_perl or other
>persistance scheme, the connection time when using an RDBMS will likely
>be as slow as the opening/parsing time of the flatfile.
>Using DBD::CSV or DBD::AnyData you can stick with SQL,
>have keys in any column and have 0 connection time.
You forget to mention that DBD::CSV and DBD::AnyData need time to parse
the data, because this uses a flatfile. No way these DBD::* modules
would be any faster than parsing a flatfile yourself.
>From the OP:
>> > So I'm wondering if it will be faster if I cache this data to a flat
>> > file & read
>> > the flat file instead of connecting to Oracle (via DBI), fetching the
>> > data, & dropping the DB connection.
If you only need a small portion of this data at once, you might uses an
presorted and/or indexed flatfile. That way it might be possible to
quickly find and extract the chunk of data you want, without reading all
of the file.
Also, there's no need to have just *one* datafile. If your data is
logically grouped, for example per product group and you only need to
access data from one group at a time, you can make one datafile per
group.
--
Bart.
