That's not a bad idea and we thought about it,
but...
With large number of attributes to be compared (over 100 in some
cases),
possible numeric comparison (value between 10 and 50),
joins to other tables (comparisons might include indirect
comparisons like ratings Aaa, Bbb that require a separate value look up
table),
several pass updates (outputs will have to be determined in
several passes through update statements since many rules need to be
applied before an output is made with each rule altering a different
component of the output)
and a relatively large data set (100s of attributes, dozens of
output columns, and over 20000 rows to which rules need to be applied)
it feels like we'll just end up rewriting a db engine in parts.
Do you still think your suggestion would still work? If not, would
SQLite support the memory-only lock-free mode?
MR
-----Original Message-----
From: John Stanton [mailto:[EMAIL PROTECTED]
Sent: Monday, September 26, 2005 3:00 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] Running SQLite in memory-only mode
You might do better not to use any database manager and to place your
rules in a flat file which you memory map into your program, then use s
REGEX regular expression search (or simpler string search) to pop out a
list of pointers to the rules. That would be blindingly fast and memory
resident (or at least virtual memory resident).
Simpler can be better.
JS
Radomyselskiy, Mikhail (IT) wrote:
> Hi,
>
> I would like to use SQLite for processing of rules that look like
> WHERE clauses (so if i fill a table with attributes and outputs i can
> run "SELECT <OUTPUTS> WHERE <ATTRIBUTE MATCHING RULES>")
>
> Looking at the speed comparison (and i realize its not uptodate but
> for the lack of better info) it seems a lot of time is spent on the
> file access and synchronization. We need an engine that runs
> completely in memory without disk use and that doesn't have to be
> thread-safe (FastDB seems to do this but I wanted to see if there is a
> more main-stream alternative).
>
> From various comments it looks like this is possible and I just
> wanted to make sure and maybe get some idea of how to do this.
>
> Thank You,
>
>
> Mike Radomyselskiy
> [EMAIL PROTECTED]
> --------------------------------------------------------
>
> NOTICE: If received in error, please destroy and notify sender.
Sender does not waive confidentiality or privilege, and use is
prohibited.
>
--------------------------------------------------------
NOTICE: If received in error, please destroy and notify sender. Sender does
not waive confidentiality or privilege, and use is prohibited.
