Thomas Munro <thomas.mu...@gmail.com> writes: > Here's another crash like that.
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=cavefish&dt=2019-07-13%2003%3A49%3A38 > 2019-07-13 04:01:23.437 UTC [9365:70] LOG: server process (PID 12951) > was terminated by signal 11: Segmentation fault > 2019-07-13 04:01:23.437 UTC [9365:71] DETAIL: Failed process was > running: select infinite_recurse(); It occurred to me to scrape the buildfarm database for these failures, and what I got was sysname | branch | snapshot | stage | data | architecture --------------+---------------+---------------------+-----------------+---------------------------------------------------------------------------------------------------------------+------------------ demoiselle | HEAD | 2019-04-27 14:55:52 | pg_upgradeCheck | 2019-04-27 15:00:42.736 UTC [1457:66] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) buri | HEAD | 2019-04-27 23:54:46 | Check | 2019-04-28 00:01:49.794 UTC [3041:66] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) takin | HEAD | 2019-05-01 08:16:48 | pg_upgradeCheck | 2019-05-01 08:23:27.159 UTC [32303:59] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le bonito | HEAD | 2019-05-01 23:05:36 | Check | 2019-05-01 23:11:00.145 UTC [13933:66] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) shoveler | HEAD | 2019-05-10 14:04:34 | Check | 2019-05-10 14:11:26.833 UTC [13456:73] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER8) demoiselle | HEAD | 2019-05-19 14:22:23 | pg_upgradeCheck | 2019-05-19 14:26:17.002 UTC [23275:80] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) vulpes | HEAD | 2019-06-15 09:16:45 | pg_upgradeCheck | 2019-06-15 09:22:22.268 UTC [4885:77] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le ayu | HEAD | 2019-06-19 22:13:23 | pg_upgradeCheck | 2019-06-19 22:18:16.805 UTC [2708:71] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER8) quokka | HEAD | 2019-07-10 14:20:13 | pg_upgradeCheck | 2019-07-10 15:24:06.102 BST [5d25f4fb.2644:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64 cavefish | HEAD | 2019-07-13 03:49:38 | pg_upgradeCheck | 2019-07-13 04:01:23.437 UTC [9365:71] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) pintail | REL_12_STABLE | 2019-07-13 19:36:51 | Check | 2019-07-13 19:39:29.013 UTC [31086:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) bonito | HEAD | 2019-07-19 23:13:01 | Check | 2019-07-19 23:16:33.330 UTC [24191:70] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) takin | HEAD | 2019-07-24 08:24:56 | Check | 2019-07-24 08:28:01.735 UTC [16366:75] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le quokka | HEAD | 2019-07-31 02:00:07 | pg_upgradeCheck | 2019-07-31 03:04:04.043 BST [5d40f709.776a:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64 elasmobranch | HEAD | 2019-08-01 03:13:38 | Check | 2019-08-01 03:19:05.394 UTC [22888:62] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) buri | HEAD | 2019-08-02 00:10:23 | Check | 2019-08-02 00:17:11.075 UTC [28222:73] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) urocryon | HEAD | 2019-08-02 05:43:46 | Check | 2019-08-02 05:51:51.944 UTC [2724:64] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le batfish | HEAD | 2019-08-04 19:02:36 | pg_upgradeCheck | 2019-08-04 19:08:11.728 UTC [23899:79] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le buri | REL_12_STABLE | 2019-08-07 00:03:29 | pg_upgradeCheck | 2019-08-07 00:11:24.500 UTC [1405:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) quokka | REL_12_STABLE | 2019-08-08 02:43:45 | pg_upgradeCheck | 2019-08-08 03:47:38.115 BST [5d4b8d3f.cdd7:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64 quokka | HEAD | 2019-08-08 14:00:08 | Check | 2019-08-08 15:02:59.770 BST [5d4c2b88.cad9:5] DETAIL: Failed process was running: select infinite_recurse(); | ppc64 mereswine | REL_11_STABLE | 2019-08-11 02:10:12 | InstallCheck-C | 2019-08-11 02:36:10.159 PDT [5004:4] DETAIL: Failed process was running: select infinite_recurse(); | ARMv7 takin | HEAD | 2019-08-11 08:02:48 | Check | 2019-08-11 08:05:57.789 UTC [11500:67] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le mereswine | REL_12_STABLE | 2019-08-11 09:52:46 | pg_upgradeCheck | 2019-08-11 04:21:16.756 PDT [6804:5] DETAIL: Failed process was running: select infinite_recurse(); | ARMv7 mereswine | HEAD | 2019-08-11 11:29:27 | pg_upgradeCheck | 2019-08-11 07:15:28.454 PDT [9954:76] DETAIL: Failed process was running: select infinite_recurse(); | ARMv7 demoiselle | HEAD | 2019-08-11 14:51:38 | pg_upgradeCheck | 2019-08-11 14:57:29.422 UTC [9436:70] DETAIL: Failed process was running: select infinite_recurse(); | ppc64le (POWER9) (26 rows) This is from a scan going back 9 months (to mid-December), so the lack of any matches before late April is pretty notable: it seems highly probable that some change we made during April is related. A cursory scan of commits during April finds only one that seems conceivably related (though perhaps I just lack enough imagination): Author: Tom Lane <t...@sss.pgh.pa.us> Branch: master Release: REL_12_BR [798070ec0] 2019-04-11 18:16:50 -0400 Re-order some regression test scripts for more parallelism. Move the strings, numerology, insert, insert_conflict, select and errors tests to be parts of nearby parallel groups, instead of executing by themselves. So that leads to the thought that "the infinite_recurse test is fine if it runs by itself, but it tends to fall over if there are concurrently-running backends". I have absolutely no idea how that would happen on anything that passes for a platform built in this century. Still, it's a place to start, which we hadn't before. Also notable is that we now have a couple of hits on ARM, not only ppc64. Don't know what to make of that. regards, tom lane
