Hello again ! After all my tests I conclude that the actuall implementation of sqlite "wall" mode needs active manual management to not grow unbounded and with multiuser/long running proesses sharing the same database sqlite will eat all our disk space, because the chance of a process get the sqlite database without any active reader blocking it to do a checkpoint will diminish inversaly proportional to the number of simultaneos users.
ls -l -rw-r--r-- 1 xxxx xxxx 6185697280 Mar 23 21:34 hacker-news-items.db // 5.7GB -rw-r--r-- 1 xxxx xxxx???? 786432 Mar 23 21:34 hacker-news-items.db-shm // 786KB -rw-r--r-- 1 xxxx xxxx? 397299872 Mar 23 21:34 hacker-news-items.db-wal //387MB stable at this size manually checkpoint every 1000 users -rw-r--r-- 1 xxxx xxxx?? 42370112 Mar 23 22:42 hacker-news-items.db-wal //41MB stable at this size manually checkpoint every 100 users -rw-r--r-- 1 xxxx xxxx?? 16686032 Mar 23 22:56 hacker-news-items.db-wal //16MB stable at this size manually checkpoint every 10 users To be able to compare I dumped all users and users_submitted on a separated database to measure it's total size -rw-r--r-- 1 xxxx xxxx? 562626560 Mar 23 22:26 hacker-news-items-diffdb //550MB db with all users and users_submitted
