Hallo! I'm not using compressed data while SQLite is creating the database. I create it in the standard way, and AFTER it is created and closed I compress the just generated file and I transfer it. The compression is used only for performance reasons during the file transfer.
I cannot use a memory database because it is lost when it is closed. Your suggestion to rewrite the file access functions is surely nice, but a bit too complex. bye -----Messaggio originale----- Da: Teg [mailto:[EMAIL PROTECTED] Inviato: martedì, 7. febbraio 2006 14:52 A: Clinco, Michele Oggetto: Re: [sqlite] R: [sqlite] Snapshot database creation performance Hello Michele, Perhaps replacing the "os_win.c" (or whatever your OS) with functions the simulate disk IO through your compressed stream functions. The biggest problem I see is random access. Typically there is no quick seeking within compressed data, you just have to "Read" to the point you want and then do this repeatedly every time you seek. Why can't you use a plain "memory" database? My application uses compressed data (gzip) but, the tradeoff to small data files is exceptionally heavy CPU usage when the data is decompressed/compressed. The datafiles are sized so, seeking isn't too painful. C Tuesday, February 7, 2006, 8:09:47 AM, you wrote: CM> You understood perfectly. CM> The ramdisk is not the goal, but just a mean to obtain the CM> 'best performance' possible goal. CM> I don't need the ramdisk at all, just a 'all in memory processing'. CM> The solution to this problem should help many users because I CM> think we are many that use a SQLite database to store data in CM> offline mode. CM> Bye, Michele CM> -----Messaggio originale----- CM> Da: Andrew Piskorski [mailto:[EMAIL PROTECTED] CM> Inviato: martedì, 7. febbraio 2006 13:54 CM> A: sqlite-users@sqlite.org CM> Oggetto: Re: [sqlite] Snapshot database creation performance CM> On Tue, Feb 07, 2006 at 07:43:44AM -0500, Andrew Piskorski wrote: >> On Tue, Feb 07, 2006 at 12:52:08PM +0100, Clinco, Michele wrote: >> > My program is written in .Net and the compression routines I'm using are >> > stream based, so I need to create a memory stream from the internal >> > buffers that can be used by the compression routine... >> >> Do you mean you wish to directly access SQLite's in-memory data >> structures, rather than using a SQL query to get the data? Why? CM> Oh, never mind, I wrote too soon. I realize now that you want an CM> in-memory representation of a SQLite database file, which you than CM> gzip or whatever, all still in memory, and then transfer over the CM> network to whomever wants that file. And you want to know if there's CM> some way to do that which offers more programatic control than the CM> ramdisk filesystem you're using now. CM> Essentially, you want a way to dynamically control the size of the RAM CM> disk. But it sounds like you don't necessarily need ALL the CM> facilities of a normal file-system, so you're wondering if perhaps you CM> could implement something more purpose-specific yourself rather than CM> using that ramdisk driver. CM> Interesting, but I don't know the answer. -- Best regards, Teg mailto:[EMAIL PROTECTED]
