"Andrey M. Borodin" <x4...@yandex-team.ru> writes: >> On 10 Dec 2024, at 08:31, Andy Fan <zhihuifan1...@163.com> wrote: >> >> I want to know if we have some existing testing framework for >> this area (design, code, licence etc). > > I think isolation tests [0] are what you are looking for. These tests are > designed to test concurrent execution of various queries. > More subtle race conditions require coordination of injection points > [1] or are tested with stochastic tests [2].
Thanks, I always think the isolation tests under t/ directory is valuable and it is a great way prove a bug is really fixed under the known case. However it is not perfect because: (a). All the cases are the known cases, kind of different from high concurrently case which may find some bugs we are not aware of. (b). every tap test needs to create a new instance and run the test, it is hard to think this is a effective way for lots of tests. So for testing some unknown bug, I am imaging the testing framework should contains the following module at least. (a). Workload. Produce kind of queries with different character, like hot update, epq update and so on. (b). Testing enginer with pgbench. run these queries with high concurrey. (c). error detection. error, core, wrong result and so on. Some more modules like: relkind module: the same column definitions with different relkind, like (plain table, partition table etc.), so that the same workload can hit different knowledge. different index definitions and so on. parallel module: different server configuration (debug_paralle_query etc.). Common GUCS module: different enable_xxx configuration, compare the result under Read Repeatable isolation level. In the past 2 ~ 0.7 years, we internally develop such testing framework and found out some unkonwn bug in the community version. like [1] [2]. But that project has stopped and I thought the project is kind of heavy in implementation and test case organization, so I want to redesign the v2 and for the *MVCC & vacuum (freeze) & heap_page_prune* part only(which is more related to my current work). [1] https://www.postgresql.org/message-id/tencent_A3CE810F59132D8E230475A5F0F7A08C8307%40qq.com [2] https://www.postgresql.org/message-id/202312121711.tzjyb5yeb3fa%40alvherre.pgsql -- Best Regards Andy Fan
