Summary: Postgres database exceeded its 300 MB memory limit, causing the kernel to kill the process. Upon restart, the database failed to recover from corruption. Resolved by: Tearing down and restarting demos. This caused the databases to reset. To consider: Perhaps the 300 MB limit for the database process is too low? Or perhaps some run-away process in object caused excessive database use?
Hi Jacques, I took a look at the standard output and standard error logs for ofbiz-trunk and found there was a huge amount, suggesting some sort of cycle. Filtering the logs for the Postgres database, I used the command: docker compose logs db | less Extract from the logs: db-1 | 2026-10-17 02:35:56.952 UTC [64] LOG: database system was shut down at 2026-10-17 02:35:56 UTC db-1 | 2026-10-17 02:35:56.962 UTC [1] LOG: database system is ready to accept connections db-1 | 2026-10-17 11:47:24.253 UTC [1] LOG: server process (PID 1747) was terminated by signal 9: Killed db-1 | 2026-10-17 11:47:24.253 UTC [1] DETAIL: Failed process was running: SELECT PTYACCPREF.PARTY_ID, PTYACCPREF.BASE_CURRENCY_UOM_ID, PTYGROUP.GROUP_NAME, PTYROLE.ROLE_TYPE_ID FROM (public.PARTY_ACCTG_PREFERENCE PTYACCPREF INNER JOIN public.PARTY_GROUP PTYGROUP ON PTYACCPREF.PARTY_ID = PTYGROUP.PARTY_ID) INNER JOIN public.PARTY_ROLE PTYROLE ON PTYACCPREF.PARTY_ID = PTYROLE.PARTY_ID WHERE (((PTYROLE.ROLE_TYPE_ID = $1))) db-1 | 2026-10-17 11:47:24.253 UTC [1] LOG: terminating any other active server processes db-1 | 2026-10-17 11:47:24.253 UTC [2369] WARNING: terminating connection because of crash of another server process at character 245 db-1 | 2026-10-17 11:47:24.253 UTC [2369] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. db-1 | 2026-10-17 11:47:24.253 UTC [2369] HINT: In a moment you should be able to reconnect to the database and repeat your command. db-1 | 2026-10-17 11:47:24.253 UTC [2357] WARNING: terminating connection because of crash of another server process db-1 | 2026-10-17 11:47:24.253 UTC [2357] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. It appears that the database was killed at 11:47:24.253 UTC. The question is why.... Searching in /var/log/syslog for 2026-10-17T11:47 we find: 2026-10-17T11:47:24.235788+00:00 ofbiz-ec2-or kernel: postgres invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0 2026-10-17T11:47:24.235808+00:00 ofbiz-ec2-or kernel: CPU: 2 UID: 100999 PID: 1648549 Comm: postgres Not tainted 7.0.0-1014-aws #14~24.04.1-Ubuntu PREEMPTLAZY 2026-10-17T11:47:24.235813+00:00 ofbiz-ec2-or kernel: Hardware name: Xen HVM domU, BIOS 4.11.amazon 08/24/2006 ... ... 2026-10-17T11:47:24.239755+00:00 ofbiz-ec2-or kernel: memory: usage 307200kB, limit 307200kB, failcnt 1879 2026-10-17T11:47:24.239771+00:00 ofbiz-ec2-or kernel: swap: usage 0kB, limit 0kB, failcnt 0 2026-10-17T11:47:24.239773+00:00 ofbiz-ec2-or kernel: Memory cgroup stats for /system.slice/docker-59d2f448671fcb358419dfe26eced00b69cebe3e7edfbc9e16f3fb3a0f0746b9.scope: In the syslog snippet above we see mention of the 'oom-killer'. This is the kernel killing processes when Out Of Memory. The log shows that the process was using 307200KB (approx 300MB) which was at the limit, triggering the process to be killed. I have run the /home/ofbizdocker/pull-and-restart.sh script as a way to tear down and restart the demos. The demos appear to have restarted and are operational again. If we see this type of problem occuring again we will need to decide if the DB memory limit is too low, or whether we should be investigating some other aspect of OFBiz. Thanks Dan. On Sat, 17 Oct 2026 at 18:34, Daniel Watford <[email protected]> wrote: > Hi Jaques, > > I'm also taking a look. There appear to be a huge number of logs. So many > that it is difficult to find the relevant text. > > There doesn't appear to be any issues with lack of disk space. > > I'll investigate by shutting the demos down and then bringing them up one > by one, checking logs as I go. > > Thanks > > Dan. > > On Sat, 17 Oct 2026 at 17:31, Jacques Le Roux via dev < > [email protected]> wrote: > >> Hi, >> >> Between 16:45 UTC and "now", OFBiz demos are slow (or even very slow) or >> down. >> Trunk is by far the much affected, which is weird... I'll have a look... >> >> Jacques >> >> >> > > -- > Daniel Watford > -- Daniel Watford
