On 21 Dec 2017, at 7:06pm, Nelson, Erik - 2 <erik.l.nel...@bankofamerica.com> wrote:
> I'm running on linux with the OS page size configured to 4096 and ~380 GB of > ram (much more than required for the table so I think I'm not swapping) and > haven't altered the sqlite page size. I am using sqlite version 3.13.0 and > these pragmas immediately after database creation. > > pragma temp_store = MEMORY > pragma journal_mode = off > > With these settings I'm seeing nonlinear (in a bad way) times for the insert. > Is that expected? Thank you for your detailed description of your setup, which saved a lot of back-and-forth. Your blog page shows a good understanding of SQLite and I can find no obvious errors in your code or your reasoning. Your Blog post includes a very nice report of comparative testing. The inconsistency you’re seeing is the result of caching. Once the amount of data busts the amount of cache memory available, the operations have to wait for storage access, which takes a lot longer than RAM access. I’m not talking about the cache under control of SQLite, but the amount of memory allocated to the program by the OS. The differences you show for the various PRAGMA changes are not unexpected and don’t suggest any fault with your setup. I’m not familiar with Windows as a dev platform, but from what I see there, Windows is busting a cache (OS process memory ? storage ? both ?) near the upper end of your testing range. As you say in your comments, your macOS and Windows platforms are not comparable because the hardware isn’t identical. I thought it worth repeating that here before someone gets hooked up on the faster figures for the Mac. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
