Re: [Interest] SQLite database: avoid high memory usage?
Sorry, I was wrong. It is an issue of our app.
On 1/24/2022 11:40 AM, Alexander Dyagilev wrote:
Hello,
Let's suppose we have a database file of 16GB in size.
We open it using this code:
m_db = QSqlDatabase::addDatabase(
"QSQLITE",
dbConnectionName());
m_db.setConnectOptions("PRAGMA jounal_mode=WAL;");
m_db.setDatabaseName(m_dbPath);
m_db.open();
After this, our process will use 16GB of memory (at least, under
Windows OS). I.e. the whole database file is loaded into the memory.
Is there a way to avoid this?
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] SQLite database: avoid high memory usage?
On Monday, 24 January 2022 02:41:38 PST Alexander Dyagilev wrote: > Sorry, I don't really know (and I do not even care). But this is what > Windows Task Manager shows to our users, and they are concerned about this. Easy way to answer this: try to open a database that is bigger than the virtual memory total for your system. If you can, then it's just virtual address for the memory-mapped file, not committed memory. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel DPG Cloud Engineering ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] SQLite database: avoid high memory usage?
On 1/24/2022 12:39 PM, Hamish Moffatt via Interest wrote: Did you mean to misspell journal_mode? LOL. OK, fine... ROFL... No, of course no... Are you sure it's real memory usage and not just address space used by mmap? Sorry, I don't really know (and I do not even care). But this is what Windows Task Manager shows to our users, and they are concerned about this. ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] SQLite database: avoid high memory usage?
On Mon, Jan 24, 2022 at 11:40 AM Hamish Moffatt via Interest
wrote:
>
> On 24/1/22 19:40, Alexander Dyagilev wrote:
> > Hello,
> >
> > Let's suppose we have a database file of 16GB in size.
> >
> > We open it using this code:
> >
> > m_db = QSqlDatabase::addDatabase(
> > "QSQLITE",
> > dbConnectionName());
> > m_db.setConnectOptions("PRAGMA jounal_mode=WAL;");
> > m_db.setDatabaseName(m_dbPath);
> > m_db.open();
> >
> > After this, our process will use 16GB of memory (at least, under
> > Windows OS). I.e. the whole database file is loaded into the memory.
> >
> > Is there a way to avoid this?
>
>
> Did you mean to misspell journal_mode?
>
> Are you sure it's real memory usage and not just address space used by mmap?
>
>
> Hamish
>
Indeed. Doubts if on Windows it will be all really mapped:
https://docs.microsoft.com/en-us/windows/win32/memory/memory-limits-for-windows-releases
Besides that, to keep large areas mapped in RAM,
one may need to lock the pages in memory with their own limits.
Consider inspecting the sources of the SQL-db concerned.
Kind regards,
Robert
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
Re: [Interest] SQLite database: avoid high memory usage?
On 24/1/22 19:40, Alexander Dyagilev wrote:
Hello,
Let's suppose we have a database file of 16GB in size.
We open it using this code:
m_db = QSqlDatabase::addDatabase(
"QSQLITE",
dbConnectionName());
m_db.setConnectOptions("PRAGMA jounal_mode=WAL;");
m_db.setDatabaseName(m_dbPath);
m_db.open();
After this, our process will use 16GB of memory (at least, under
Windows OS). I.e. the whole database file is loaded into the memory.
Is there a way to avoid this?
Did you mean to misspell journal_mode?
Are you sure it's real memory usage and not just address space used by mmap?
Hamish
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
[Interest] SQLite database: avoid high memory usage?
Hello,
Let's suppose we have a database file of 16GB in size.
We open it using this code:
m_db = QSqlDatabase::addDatabase(
"QSQLITE",
dbConnectionName());
m_db.setConnectOptions("PRAGMA jounal_mode=WAL;");
m_db.setDatabaseName(m_dbPath);
m_db.open();
After this, our process will use 16GB of memory (at least, under Windows
OS). I.e. the whole database file is loaded into the memory.
Is there a way to avoid this?
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
