Hi Hannu, Thank you for patches. I'm interested in the pgbench refactoring.
The current series includes code movement, changes to internal interfaces, and new features. To make review easier, could we focus this thread on moving the existing code into separate modules while preserving existing behavior? For v6, patches 0001-0007 seem like a suitable scope for that review. I have left 0008-0012 out of this scope because they go beyond code movement and include changes to internal interfaces and structures. Those changes could be discussed separately, along with the new features. I'd like to help review the code movement in 0001-0007. Regards, Koshi Shibagaki FUJITSU LIMITED https://www.fujitsu.com/ ________________________________________ 差出人: Hannu Krosing <[email protected]> 送信日時: 2026年10月22日 2:43 宛先: Zsolt Parragi CC: [email protected] 件名: Re: [PATCH] Refactor pgbench to make future improvements easier From b102b11350e0ab77499f9249b2fd27097865088c Mon Sep 17 00:00:00 2001 From: Hannu Krosing <[email protected]> Date: Wed, 21 Oct 2026 17:20:20 +0000 Subject: [PATCH v6 00/28] Modularize pgbench, extract shared PRNG/math to src/common, add contrib/pgbench, TPC/HTAP/LDBC-Graph/VecBench suites, JSON configs, --log-db telemetry, and WASM viewer Hi hackers, Attached is v6 of the patch series to modularize pgbench, extract its core PRNG/hashing/distribution routines into src/common for reuse by a new server-side contrib/pgbench extension, and extend pgbench with the scripting primitives, concurrency/dataset pillars, declarative JSON/JSONC configurations, database telemetry logging (--log-db), modular WebAssembly viewer (src/bin/pgbench/viewer/), and bundled workload suites needed to run complex multi-transaction OLTP, DSS, HTAP, Graph, and Vector workloads natively without external middleware. Changes in v6 (compared to v5): ------------------------------- 1. Rebased onto latest master (c00a97b646c). 2. Expanded the patch series from 22 patches (v5) to 28 patches (v6) with six cohesive feature commits (Patches 0023-0028): - Patch 0023 (Common Horizontal Pillars Tiers 1-3): * Read-only mmap dataset loader (dataset.c/h) for binary vector files (.fvecs, .bvecs, .ivecs) and columnar CSV/TSV files, plus native PGBT_VECTOR and PGBT_BYTEA value types and \gset_col. * Lock-free shared reservoirs (--reservoir, \reservoir_push, \reservoir_sample) and causal consistency watermarks (\wait_until, \advance_watermark) for cross-stream producer/consumer coordination. * Non-blocking mid-script connection switching (\session_connect, \session_disconnect), subroutine calls (\call), Recall@K / NDCG@K telemetry (\record_recall), schedule lag SLAs, and deterministic trace replay (--replay). - Patch 0024 (LDBC-Inspired Graph Benchmark Suites): * Correlated power-law graph generators (ldbc_correlated_edge, ldbc_flashmob_ts), dependent write tracking (\begin_dependent_write, \end_dependent_write), phased execution barriers (\phase_barrier), and in-script result assertions (\validate_result). * LDBC-inspired graph suites across three SQL execution models: standard relational SQL (pgbench-graph), recursive CTEs with UNION DISTINCT ON (pgbench-graph-sql-ud), and ISO SQL/PGQ GRAPH_TABLE / CREATE PROPERTY GRAPH (pgbench-graph-pgq) covering LSQB (Q1-Q9), SNB Interactive v1/v2 (IC1-IC13, IS1-IS7, IU1-IU8, ID1-ID8), SNB Business Intelligence (Power -> \phase_barrier -> Batch Update -> \phase_barrier -> Throughput), FinBench, and Graphalytics, with BIGINT key alignment across vertex/edge tables. - Patch 0025 (VecBench-Inspired Vector ANN Suite & Documentation): * VecBench-inspired vector ANN benchmark suite (pgbench-vec Modes 1-6: unfiltered search, pre-filtered search, post-filtered oversampling, streaming insert, multi-vector late interaction, and continuous aging/index degradation tracking Q1 vs Q4 recall), named per-stream recall SLAs (--sla="total:<metric>:recall>=X"), --vector-base, --ground-truth-table, and comprehensive user documentation in src/bin/pgbench/workloads/README. - Patch 0026 (Declarative JSON/JSONC Workload Configuration): * Adds --config=<file> and --dump-config=<file> (config.c/h), multi-stage initialization pipelines (init_steps[]), direct replay of --json-report files via --config, and 23 ready-to-run .pgbench.json workload definitions. - Patch 0027 (Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs): * Adds --log-db, --log-db-unlogged[=stream|all|off], --log-db-mode={stream|deferred}, --run-label, --log-db-schema, and --log-db-index (logdb.c/h) to stream run metadata, HDR latency percentiles, Recall@K metrics, SLA compliance, and UNLOGGED interval/transaction samples into PostgreSQL via COPY, plus 5 overloaded SQL inspection/comparison SRFs (pgbench_show_run and pgbench_compare_runs) and pgbench_viewer_payload_json(). - Patch 0028 (Modular bottle.py + Water.css + WebAssembly Viewer): * Adds a standalone zero-dependency web telemetry viewer and two-stage metatemplate SQL Data Explorer (src/bin/pgbench/viewer/) organized into html/ (10 SimpleTemplate .html.stpl views), css/ (water.css + viewer.css), wasm/ (439-byte pgbench_chart.wasm SVG charting kernel), and src/ (C/WAT sources and builder), documented in viewer/README.md. 3. Cross-platform portability & CI fixes (Windows MSVC, MinGW, Meson): - Added a WIN32 portable temporary file stream fallback (open_temp_json_stream() / finish_temp_json_stream()) for serializeConfigJsonToString() and serializeReportJsonToString() in src/bin/pgbench/logdb.c since POSIX open_memstream() is unavailable on Windows (MinGW / MSVC). - Replaced GCC __attribute__((aligned(64))) struct field attributes in src/bin/pgbench/dataset.h (PgBenchReservoir) and src/bin/pgbench/dataset.c (PgBenchCausalClock) with explicit cache-line padding arrays, and added MSVC (_MSC_VER) _Interlocked* and _ReadWriteBarrier() shims in src/bin/pgbench/pgbench.h for GCC __atomic_* builtins used in dataset.c, commands.c, engine.c, and script.c. - Updated POLL_USING_SELECT in src/bin/pgbench/poller.c to track write_fds alongside read_fds so non-blocking PGRES_POLLING_WRITING states wake immediately on Windows select(). - Updated dataset/vector/ground-truth spec parsers in dataset.c to recognize Windows drive-letter colons (C:\...) and backslash path separators. - Pinned deterministic cluster seed 42 in pgbench-vec SQL workloads and increased Phase 2 multi-script transaction sample count in 001_pgbench_with_server.pl for deterministic CI runs. Architecture & Series Overview (28 Patches): -------------------------------------------- The series is organized into ten logical parts across 28 self-contained patches so each layer can be reviewed and tested independently: Part 1: Modularize src/bin/pgbench (Patches 0001-0012) Splits the monolithic ~8,000-line src/bin/pgbench/pgbench.c into focused compilation units with clean headers and minimal global coupling: - 0001, 0008: stats.c/h (SimpleStats, StatsData, LatencyHist, mergeStats) - 0002, 0009: variable.c/h (PgBenchValue, Variable, scoped VariableSpace) - 0003, 0010: poller.c/h (socket_set abstraction over poll/select) - 0004, 0011: script.c/h (ParsedScript, expression evaluation, chooseScript) - 0005, 0012: commands.c/h (meta-command execution & pluggable handler table) - 0006: init.c/h (database initialization -i) - 0007: engine.c/h (CState/TState client state machine & thread loop) Part 2: Shared PRNG/Math Library & contrib/pgbench (Patches 0013-0014) - 0013: Extracts pgbench's deterministic PRNG (pg_erand48 / Xoshiro), 64-bit FNV/Murmur hashing, uniform/exponential/gaussian/zipfian random distributions, permute(), TPC-C NURand, syllable generation, random strings, Pareto/lognormal distributions, and correlated graph/vector generators into src/common/pgbench_funcs.c (and src/include/common/pgbench_funcs.h), compiled into both libpgcommon.a and libpgcommon_shlib.a. - 0014: Introduces contrib/pgbench (CREATE EXTENSION pgbench), installing SQL-callable C functions into SCHEMA pgbench so server-side data generation (INSERT ... SELECT FROM generate_series(...)) produces values using the exact same algorithms and distributions as the pgbench client. Part 3: Rich Scripting & Client State Machine Primitives (Patches 0015-0019) - 0015: Extends PgBenchValue with text (PGBT_TEXT), timestamp (PGBT_TIMESTAMP), and 1D array (PGBT_ARRAY) types, with automatic SQL literal quoting and array subscripting (:arr[:i]). - 0016: Adds built-in expression functions: nurand(), c_last(), random_string(), random_alnum(), random_date(), random_timestamp(), array_length(), array_append(), and format(). - 0017: Adds structured control-flow meta-commands (\for / \endfor, \while / \endwhile) and block-scoped exception handling (\try / \catch / \endtry with automatic SAVEPOINT / ROLLBACK TO SAVEPOINT and serialization/deadlock retry support, plus \expect_error). - 0018: Adds \gset_meta (capturing rowcount, sqlstate, and backend_pid) and PostgreSQL array text parsing in \gset and \aset. - 0019: Extends \sleep to evaluate arbitrary arithmetic expressions and adds \keying_time and \think_time (log-normal user think times that sleep without inflating active transaction latency). Part 4: Multi-Group Orchestration, Telemetry & Standard Suites (Patches 0020-0022) - 0020: Adds heterogeneous client groups (--group name:clients=N, threads=M,rate=R,scripts=...) and sequential execution streams for concurrent mixed OLTP + DSS/OLAP workloads. - 0021: Adds power-of-two sub-bucket HDR latency histograms (p50, p90, p95, p99, p99.9, max), --sla threshold validation, composite throughput metrics (tpmC, QphH), and machine-readable --json-report output. - 0022: Adds six self-contained benchmark workload suites under src/bin/pgbench/workloads/: * pgbench-b (enhanced TPCB-like banking workload) * pgbench-c (TPC-C-inspired wholesale supplier OLTP workload) * pgbench-e (TPC-E-inspired brokerage OLTP workload) * pgbench-h (TPC-H-inspired decision support workload) * pgbench-ds (TPC-DS-inspired multi-channel retail DSS workload) * pgbench-htap (hybrid concurrent OLTP + analytical OLAP workload) Part 5: Common Horizontal Pillars Tiers 1-3 (Patch 0023) - 0023: Implements in-memory mmap datasets (dataset.c/h) for zero-copy loading of .fvecs/.bvecs/.ivecs and .csv/.tsv datasets, PGBT_VECTOR and PGBT_BYTEA value types, \gset_col, lock-free shared reservoirs (--reservoir, \reservoir_push, \reservoir_sample), causal watermarks (\wait_until, \advance_watermark), non-blocking \session_connect / \session_disconnect, \call / \return subroutines, Recall@K / NDCG@K accuracy telemetry (\record_recall), schedule lag SLAs, and deterministic trace replay (--replay), along with reference documentation in doc/src/sgml/ref/pgbench.sgml and doc/src/sgml/pgbench-ext.sgml. Part 6: LDBC-Inspired Graph Benchmark Suites (Patch 0024) - 0024: Implements correlated power-law graph generators (ldbc_correlated_edge, ldbc_flashmob_ts), dependent write tracking (\begin_dependent_write, \end_dependent_write), phased execution barriers (\phase_barrier), result validation (\validate_result), SGML reference documentation (doc/src/sgml/ref/pgbench.sgml and doc/src/sgml/pgbench-ext.sgml), and the unified BIGINT-aligned LDBC-inspired schema and query suites for LSQB (Q1-Q9), SNB Interactive v1/v2, SNB BI, FinBench, and Graphalytics across standard relational SQL (workloads/pgbench-graph/), recursive UNION DISTINCT ON (workloads/pgbench-graph-sql-ud/), and native ISO SQL/PGQ GRAPH_TABLE (workloads/pgbench-graph-pgq/). Part 7: VecBench-Inspired Vector Benchmark Suite & Docs (Patch 0025) - 0025: Implements the VecBench-inspired vector ANN benchmark suite under workloads/pgbench-vec/ across 6 operational modes (unfiltered search, pre-filtered search, post-filtered adaptive oversampling, streaming ingestion, multi-vector MaxSim late interaction, and continuous aging/index degradation comparing initial Q1 vs mature Q4 recall), plus --vector-base, --ground-truth-table, named per-stream recall SLAs, and comprehensive user documentation and trademark/naming guardrails in src/bin/pgbench/workloads/README. Part 8: Declarative JSON/JSONC Configuration (Patch 0026) - 0026: Adds --config=<file> and --dump-config=<file> (src/bin/pgbench/ config.c/h) using PostgreSQL's native src/common/jsonapi.c parser with a C/C++ comment stripper for JSONC support, multi-stage init_steps, direct replay of --json-report output files, and 23 ready-to-run .pgbench.json workload definitions. Part 9: Direct PostgreSQL Telemetry Logging & SQL Comparison SRFs (Patch 0027) - 0027: Adds --log-db, --log-db-unlogged[=stream|all|off], --log-db-mode={stream|deferred}, --run-label, --log-db-schema, and --log-db-index (src/bin/pgbench/logdb.c/h) to stream run metadata, per-group/per-script HDR percentiles, Recall@K metrics, SLA compliance, and high-throughput UNLOGGED per-transaction/per-window samples into a 3-tier, 9-table PostgreSQL schema using COPY, plus 5 overloaded SQL comparison SRFs (pgbench_show_run, pgbench_compare_runs) and pgbench_viewer_payload_json(). Part 10: Modular WebAssembly Telemetry Viewer & SQL Data Explorer (Patch 0028) - 0028: Adds a standalone zero-dependency bottle.py + Water.css + 439-byte WebAssembly SVG chart viewer and two-stage metatemplate SQL Data Explorer (src/bin/pgbench/viewer/ organized into html/, css/, wasm/, and src/, with documentation in viewer/README.md and src/bin/pgbench/README). Testing: -------- - `make -C contrib/pgbench check` passes all SQL regression tests for `SCHEMA pgbench` functions and generators. - `make -C src/bin/pgbench check` passes all TAP tests in `t/001_pgbench_with_server.pl` and `t/002_pgbench_no_server.pl`. Hannu Krosing (28): Refactor pgbench: extract statistics structures and functions to stats.c/h Refactor pgbench: extract variable management and value types to variable.c/h Refactor pgbench: extract socket polling abstraction to poller.c/h Refactor pgbench: extract script parsing, built-in scripts, and expression evaluation to script.c/h Refactor pgbench: extract meta-command execution and prepared statement helpers to commands.c/h Refactor pgbench: extract database initialization (-i) to init.c/h Refactor pgbench: extract benchmark execution engine and client state machine to engine.c/h Refactor pgbench stats: add mergeStats() and decouple global flags Refactor pgbench variables: introduce scoped variable store Refactor pgbench poller: encapsulate socket_set and add poller API aliases Refactor pgbench script: decouple chooseScript() from TState Refactor pgbench commands: introduce pluggable meta-command dispatch Extract pgbench low-level functions into src/common Add contrib/pgbench extension exposing benchmark functions in SCHEMA pgbench pgbench: Extend PgBenchValue with text, timestamp, and array types pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array functions pgbench: Add script loops (\for, \while) and exception handling (\try, \catch, \expect_error) pgbench: Add \gset_meta and array column parsing in \gset and \aset pgbench: Support dynamic \sleep expressions, \keying_time, and \think_time pgbench: Add heterogeneous client groups (--group) and sequential streams pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH metrics, and --json-report pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds, and pgbench-htap workload suites pgbench: add mmap datasets, vector/bytea types, reservoirs, watermarks, \call, Recall@K, and --replay pgbench: add LDBC-inspired graph suites (SQL, SQL_UD, SQL/PGQ), generators, and barriers pgbench: add VecBench-inspired vector ANN suite (Modes 1-6), recall SLAs, and docs pgbench: add --config and --dump-config JSON/JSONC workload configuration support pgbench: add --log-db PostgreSQL telemetry logging, UNLOGGED stream tables, and SQL comparison SRFs pgbench: add modular bottle.py + Water.css + WebAssembly telemetry viewer contrib/Makefile | 1 + contrib/meson.build | 1 + contrib/pgbench/.gitignore | 7 + contrib/pgbench/Makefile | 29 + contrib/pgbench/README | 34 + contrib/pgbench/expected/pgbench.out | 306 + contrib/pgbench/meson.build | 35 + contrib/pgbench/pgbench--1.0.sql | 427 + contrib/pgbench/pgbench.c | 1034 +++ contrib/pgbench/pgbench.control | 6 + contrib/pgbench/sql/pgbench.sql | 111 + doc/src/sgml/contrib.sgml | 2 + doc/src/sgml/filelist.sgml | 1 + doc/src/sgml/pgbench-ext.sgml | 580 ++ doc/src/sgml/ref/pgbench.sgml | 1062 ++- src/bin/pgbench/.gitignore | 1 + src/bin/pgbench/Makefile | 13 + src/bin/pgbench/README | 217 + src/bin/pgbench/commands.c | 2057 +++++ src/bin/pgbench/commands.h | 87 + src/bin/pgbench/config.c | 3052 +++++++ src/bin/pgbench/config.h | 55 + src/bin/pgbench/dataset.c | 2894 ++++++ src/bin/pgbench/dataset.h | 159 + src/bin/pgbench/engine.c | 4143 +++++++++ src/bin/pgbench/engine.h | 253 + src/bin/pgbench/exprparse.y | 234 +- src/bin/pgbench/exprscan.l | 18 + src/bin/pgbench/init.c | 828 ++ src/bin/pgbench/init.h | 51 + src/bin/pgbench/logdb.c | 2735 ++++++ src/bin/pgbench/logdb.h | 77 + src/bin/pgbench/meson.build | 10 + src/bin/pgbench/pgbench.c | 8042 ++--------------- src/bin/pgbench/pgbench.h | 470 +- src/bin/pgbench/poller.c | 287 + src/bin/pgbench/poller.h | 47 + src/bin/pgbench/script.c | 4435 +++++++++ src/bin/pgbench/script.h | 198 + src/bin/pgbench/stats.c | 753 ++ src/bin/pgbench/stats.h | 220 + src/bin/pgbench/t/001_pgbench_with_server.pl | 1544 ++++ src/bin/pgbench/t/002_pgbench_no_server.pl | 557 ++ src/bin/pgbench/variable.c | 1772 ++++ src/bin/pgbench/variable.h | 177 + src/bin/pgbench/viewer/README.md | 133 + src/bin/pgbench/viewer/bottle.py | 3829 ++++++++ src/bin/pgbench/viewer/css/viewer.css | 202 + src/bin/pgbench/viewer/css/water.css | 81 + .../viewer/html/compare_matrix.html.stpl | 43 + .../viewer/html/compare_pair.html.stpl | 49 + .../viewer/html/explorer_body.html.stpl | 51 + .../viewer/html/explorer_table_meta.html.stpl | 87 + src/bin/pgbench/viewer/html/layout.html.stpl | 46 + .../viewer/html/live_monitor.html.stpl | 100 + .../pgbench/viewer/html/phase1_body.html.stpl | 30 + .../viewer/html/run_selector.html.stpl | 120 + .../pgbench/viewer/html/single_run.html.stpl | 146 + .../viewer/html/wasm_chart_box.html.stpl | 5 + .../viewer/html/wasm_chart_module.html.stpl | 246 + src/bin/pgbench/viewer/pgbench_viewer.py | 704 ++ src/bin/pgbench/viewer/src/build_wasm.py | 216 + src/bin/pgbench/viewer/src/pgbench_chart.c | 66 + src/bin/pgbench/viewer/src/pgbench_chart.wat | 105 + .../pgbench/viewer/wasm/pgbench_chart.wasm | Bin 0 -> 439 bytes src/bin/pgbench/workloads/README | 511 ++ .../pgbench-b/pgbench_b.pgbench.json | 37 + .../pgbench-b/pgbench_b_constraints.sql | 17 + .../workloads/pgbench-b/pgbench_b_init.sql | 32 + .../workloads/pgbench-b/pgbench_b_schema.sql | 100 + .../pgbench-b/pgbench_b_select_only.sql | 9 + .../pgbench-b/pgbench_b_simple_update.sql | 14 + .../workloads/pgbench-b/pgbench_b_tx.sql | 14 + .../pgbench-c/pgbench_c.pgbench.json | 53 + .../pgbench-c/pgbench_c_constraints.sql | 11 + .../pgbench-c/pgbench_c_delivery.sql | 15 + .../workloads/pgbench-c/pgbench_c_init.sql | 206 + .../pgbench-c/pgbench_c_new_order.sql | 41 + .../pgbench-c/pgbench_c_order_status.sql | 19 + .../workloads/pgbench-c/pgbench_c_payment.sql | 31 + .../workloads/pgbench-c/pgbench_c_schema.sql | 470 + .../pgbench-c/pgbench_c_stock_level.sql | 11 + .../pgbench-ds/pgbench_ds.pgbench.json | 63 + .../pgbench-ds/pgbench_ds_constraints.sql | 22 + .../workloads/pgbench-ds/pgbench_ds_init.sql | 306 + .../pgbench-ds/pgbench_ds_maintenance.sql | 35 + .../pgbench-ds/pgbench_ds_q_cross_channel.sql | 61 + .../pgbench-ds/pgbench_ds_q_inventory.sql | 32 + .../pgbench-ds/pgbench_ds_q_promotion.sql | 31 + .../pgbench-ds/pgbench_ds_q_store_sales.sql | 47 + .../pgbench-ds/pgbench_ds_schema.sql | 214 + .../pgbench-e/pgbench_e.pgbench.json | 57 + .../pgbench-e/pgbench_e_broker_volume.sql | 8 + .../pgbench-e/pgbench_e_constraints.sql | 20 + .../pgbench-e/pgbench_e_customer_position.sql | 19 + .../workloads/pgbench-e/pgbench_e_init.sql | 202 + .../pgbench-e/pgbench_e_market_watch.sql | 12 + .../workloads/pgbench-e/pgbench_e_schema.sql | 509 ++ .../pgbench-e/pgbench_e_trade_order.sql | 24 + .../pgbench-e/pgbench_e_trade_result.sql | 14 + .../pgbench-e/pgbench_e_trade_status.sql | 14 + .../pgbench-graph-pgq/finbench.pgbench.json | 37 + .../graphalytics.pgbench.json | 37 + .../pgbench-graph-pgq/lsqb.pgbench.json | 37 + .../pgbench_graph_pgq_1hop.sql | 40 + .../pgbench_graph_pgq_2hop.sql | 38 + .../pgbench_graph_pgq_add_edge.sql | 14 + .../pgbench_graph_pgq_constraints.sql | 151 + .../pgbench_graph_pgq_ddl.sql | 93 + .../pgbench_graph_pgq_init.sql | 203 + .../pgbench_graph_pgq_schema.sql | 273 + .../pgbench_ldbc_pgq_finbench.sql | 178 + .../pgbench_ldbc_pgq_graphalytics.sql | 186 + .../pgbench_ldbc_pgq_lsqb.sql | 138 + .../pgbench_ldbc_pgq_snb_bi.sql | 255 + .../pgbench_ldbc_pgq_snb_ic.sql | 151 + .../pgbench_ldbc_pgq_snb_id.sql | 83 + .../pgbench_ldbc_pgq_snb_is.sql | 141 + .../pgbench_ldbc_pgq_snb_iu.sql | 87 + .../pgbench-graph-pgq/snb_bi.pgbench.json | 37 + .../snb_interactive.pgbench.json | 66 + .../finbench.pgbench.json | 37 + .../graphalytics.pgbench.json | 37 + .../pgbench-graph-sql-ud/lsqb.pgbench.json | 37 + .../pgbench_graph_sql_ud_1hop.sql | 28 + .../pgbench_graph_sql_ud_2hop.sql | 39 + .../pgbench_graph_sql_ud_add_edge.sql | 14 + .../pgbench_graph_sql_ud_constraints.sql | 151 + .../pgbench_graph_sql_ud_init.sql | 203 + .../pgbench_graph_sql_ud_schema.sql | 181 + .../pgbench_ldbc_sql_ud_finbench.sql | 135 + .../pgbench_ldbc_sql_ud_graphalytics.sql | 152 + .../pgbench_ldbc_sql_ud_lsqb.sql | 109 + .../pgbench_ldbc_sql_ud_snb_bi.sql | 219 + .../pgbench_ldbc_sql_ud_snb_ic.sql | 125 + .../pgbench_ldbc_sql_ud_snb_id.sql | 76 + .../pgbench_ldbc_sql_ud_snb_is.sql | 91 + .../pgbench_ldbc_sql_ud_snb_iu.sql | 87 + .../pgbench-graph-sql-ud/snb_bi.pgbench.json | 37 + .../snb_interactive.pgbench.json | 66 + .../pgbench-graph/finbench.pgbench.json | 37 + .../pgbench-graph/graphalytics.pgbench.json | 37 + .../workloads/pgbench-graph/lsqb.pgbench.json | 37 + .../pgbench-graph/pgbench_graph_1hop.sql | 28 + .../pgbench-graph/pgbench_graph_2hop.sql | 39 + .../pgbench-graph/pgbench_graph_add_edge.sql | 14 + .../pgbench_graph_constraints.sql | 151 + .../pgbench-graph/pgbench_graph_init.sql | 203 + .../pgbench-graph/pgbench_graph_pgq_ddl.sql | 93 + .../pgbench-graph/pgbench_graph_schema.sql | 181 + .../pgbench-graph/pgbench_ldbc_finbench.sql | 146 + .../pgbench_ldbc_graphalytics.sql | 190 + .../pgbench-graph/pgbench_ldbc_lsqb.sql | 108 + .../pgbench-graph/pgbench_ldbc_snb_bi.sql | 225 + .../pgbench-graph/pgbench_ldbc_snb_ic.sql | 130 + .../pgbench-graph/pgbench_ldbc_snb_id.sql | 76 + .../pgbench-graph/pgbench_ldbc_snb_is.sql | 91 + .../pgbench-graph/pgbench_ldbc_snb_iu.sql | 87 + .../pgbench_ldbc_sql_ud_finbench.sql | 135 + .../pgbench_ldbc_sql_ud_graphalytics.sql | 152 + .../pgbench_ldbc_sql_ud_lsqb.sql | 109 + .../pgbench_ldbc_sql_ud_snb_bi.sql | 219 + .../pgbench_ldbc_sql_ud_snb_ic.sql | 125 + .../pgbench_ldbc_sql_ud_snb_id.sql | 76 + .../pgbench_ldbc_sql_ud_snb_is.sql | 91 + .../pgbench_ldbc_sql_ud_snb_iu.sql | 87 + .../pgbench-graph/snb_bi.pgbench.json | 37 + .../snb_interactive.pgbench.json | 66 + .../pgbench-h/pgbench_h.pgbench.json | 52 + .../pgbench-h/pgbench_h_constraints.sql | 20 + .../workloads/pgbench-h/pgbench_h_init.sql | 187 + .../workloads/pgbench-h/pgbench_h_q1.sql | 28 + .../workloads/pgbench-h/pgbench_h_q10.sql | 40 + .../workloads/pgbench-h/pgbench_h_q12.sql | 36 + .../workloads/pgbench-h/pgbench_h_q14.sql | 21 + .../workloads/pgbench-h/pgbench_h_q19.sql | 46 + .../workloads/pgbench-h/pgbench_h_q3.sql | 32 + .../workloads/pgbench-h/pgbench_h_q5.sql | 34 + .../workloads/pgbench-h/pgbench_h_q6.sql | 21 + .../workloads/pgbench-h/pgbench_h_queries.sql | 37 + .../workloads/pgbench-h/pgbench_h_refresh.sql | 54 + .../workloads/pgbench-h/pgbench_h_schema.sql | 105 + .../pgbench-htap/pgbench_htap.pgbench.json | 70 + .../pgbench-htap/pgbench_htap_constraints.sql | 15 + .../pgbench-htap/pgbench_htap_init.sql | 233 + .../pgbench-htap/pgbench_htap_olap_q1.sql | 19 + .../pgbench-htap/pgbench_htap_olap_q18.sql | 36 + .../pgbench-htap/pgbench_htap_olap_q3.sql | 34 + .../pgbench-htap/pgbench_htap_olap_q5.sql | 39 + .../pgbench-htap/pgbench_htap_olap_q6.sql | 10 + .../pgbench-htap/pgbench_htap_oltp.sql | 46 + .../pgbench-htap/pgbench_htap_schema.sql | 396 + .../pgbench-vec/pgbench_vec_aging.sql | 79 + .../pgbench-vec/pgbench_vec_constraints.sql | 26 + .../pgbench-vec/pgbench_vec_filtered.sql | 19 + .../pgbench-vec/pgbench_vec_gt_gen.sql | 51 + .../pgbench-vec/pgbench_vec_init.sql | 34 + .../pgbench-vec/pgbench_vec_insert.sql | 22 + .../pgbench-vec/pgbench_vec_multivec.sql | 39 + .../pgbench-vec/pgbench_vec_postfiltered.sql | 36 + .../pgbench-vec/pgbench_vec_schema.sql | 91 + .../pgbench-vec/pgbench_vec_search.sql | 28 + .../pgbench-vec/vec_aging.pgbench.json | 45 + .../pgbench-vec/vec_hybrid.pgbench.json | 64 + src/common/Makefile | 1 + src/common/meson.build | 1 + src/common/pgbench_funcs.c | 1687 ++++ src/include/common/pgbench_funcs.h | 117 + 208 files changed, 52414 insertions(+), 7138 deletions(-) create mode 100644 contrib/pgbench/.gitignore create mode 100644 contrib/pgbench/Makefile create mode 100644 contrib/pgbench/README create mode 100644 contrib/pgbench/expected/pgbench.out create mode 100644 contrib/pgbench/meson.build create mode 100644 contrib/pgbench/pgbench--1.0.sql create mode 100644 contrib/pgbench/pgbench.c create mode 100644 contrib/pgbench/pgbench.control create mode 100644 contrib/pgbench/sql/pgbench.sql create mode 100644 doc/src/sgml/pgbench-ext.sgml create mode 100644 src/bin/pgbench/README create mode 100644 src/bin/pgbench/commands.c create mode 100644 src/bin/pgbench/commands.h create mode 100644 src/bin/pgbench/config.c create mode 100644 src/bin/pgbench/config.h create mode 100644 src/bin/pgbench/dataset.c create mode 100644 src/bin/pgbench/dataset.h create mode 100644 src/bin/pgbench/engine.c create mode 100644 src/bin/pgbench/engine.h create mode 100644 src/bin/pgbench/init.c create mode 100644 src/bin/pgbench/init.h create mode 100644 src/bin/pgbench/logdb.c create mode 100644 src/bin/pgbench/logdb.h create mode 100644 src/bin/pgbench/poller.c create mode 100644 src/bin/pgbench/poller.h create mode 100644 src/bin/pgbench/script.c create mode 100644 src/bin/pgbench/script.h create mode 100644 src/bin/pgbench/stats.c create mode 100644 src/bin/pgbench/stats.h create mode 100644 src/bin/pgbench/variable.c create mode 100644 src/bin/pgbench/variable.h create mode 100644 src/bin/pgbench/viewer/README.md create mode 100644 src/bin/pgbench/viewer/bottle.py create mode 100644 src/bin/pgbench/viewer/css/viewer.css create mode 100644 src/bin/pgbench/viewer/css/water.css create mode 100644 src/bin/pgbench/viewer/html/compare_matrix.html.stpl create mode 100644 src/bin/pgbench/viewer/html/compare_pair.html.stpl create mode 100644 src/bin/pgbench/viewer/html/explorer_body.html.stpl create mode 100644 src/bin/pgbench/viewer/html/explorer_table_meta.html.stpl create mode 100644 src/bin/pgbench/viewer/html/layout.html.stpl create mode 100644 src/bin/pgbench/viewer/html/live_monitor.html.stpl create mode 100644 src/bin/pgbench/viewer/html/phase1_body.html.stpl create mode 100644 src/bin/pgbench/viewer/html/run_selector.html.stpl create mode 100644 src/bin/pgbench/viewer/html/single_run.html.stpl create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_box.html.stpl create mode 100644 src/bin/pgbench/viewer/html/wasm_chart_module.html.stpl create mode 100755 src/bin/pgbench/viewer/pgbench_viewer.py create mode 100644 src/bin/pgbench/viewer/src/build_wasm.py create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.c create mode 100644 src/bin/pgbench/viewer/src/pgbench_chart.wat create mode 100644 src/bin/pgbench/viewer/wasm/pgbench_chart.wasm create mode 100644 src/bin/pgbench/workloads/README create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_select_only.sql create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_simple_update.sql create mode 100644 src/bin/pgbench/workloads/pgbench-b/pgbench_b_tx.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_delivery.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_new_order.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_order_status.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_payment.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-c/pgbench_c_stock_level.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_maintenance.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_cross_channel.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_inventory.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_promotion.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_q_store_sales.sql create mode 100644 src/bin/pgbench/workloads/pgbench-ds/pgbench_ds_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_broker_volume.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_customer_position.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_market_watch.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_order.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_result.sql create mode 100644 src/bin/pgbench/workloads/pgbench-e/pgbench_e_trade_status.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/finbench.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/graphalytics.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/lsqb.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_1hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_2hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_add_edge.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_ddl.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_graph_pgq_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_finbench.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_graphalytics.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_lsqb.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_bi.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_ic.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_id.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_is.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/pgbench_ldbc_pgq_snb_iu.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/snb_bi.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-pgq/snb_interactive.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/finbench.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/graphalytics.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/lsqb.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_1hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_2hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_add_edge.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_graph_sql_ud_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_finbench.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_graphalytics.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_lsqb.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_bi.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_ic.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_id.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_is.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/pgbench_ldbc_sql_ud_snb_iu.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_bi.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph-sql-ud/snb_interactive.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph/finbench.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph/graphalytics.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph/lsqb.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_1hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_2hop.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_add_edge.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_pgq_ddl.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_graph_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_finbench.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_graphalytics.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_lsqb.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_bi.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_ic.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_id.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_is.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_snb_iu.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_finbench.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_graphalytics.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_lsqb.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_bi.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_ic.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_id.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_is.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/pgbench_ldbc_sql_ud_snb_iu.sql create mode 100644 src/bin/pgbench/workloads/pgbench-graph/snb_bi.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-graph/snb_interactive.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q1.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q10.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q12.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q14.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q19.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q3.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q5.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_q6.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_queries.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_refresh.sql create mode 100644 src/bin/pgbench/workloads/pgbench-h/pgbench_h_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q1.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q18.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q3.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q5.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_olap_q6.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_oltp.sql create mode 100644 src/bin/pgbench/workloads/pgbench-htap/pgbench_htap_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_aging.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_constraints.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_filtered.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_gt_gen.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_init.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_insert.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_multivec.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_postfiltered.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_schema.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/pgbench_vec_search.sql create mode 100644 src/bin/pgbench/workloads/pgbench-vec/vec_aging.pgbench.json create mode 100644 src/bin/pgbench/workloads/pgbench-vec/vec_hybrid.pgbench.json create mode 100644 src/common/pgbench_funcs.c create mode 100644 src/include/common/pgbench_funcs.h -- 2.56.0 On Sun, Oct 18, 2026 at 5:16 PM Hannu Krosing <[email protected]> wrote: > > [PATCH v5 00/22] > * Modular pgbench refactoring, > * contrib/pgbench extension, > * advanced benchmark capabilities, > * and standard workload suites (pgbench-b, pgbench-c, pgbench-e, > pgbench-h, pgbench-ds, pgbench-htap) > > Hi hackers, > > This patch series presents a comprehensive modernization of pgbench, > transforming > it from a monolithic single-script driver into a modern, modular, > multi-workload > benchmarking engine with native support for complex OLTP, OLAP, and > HTAP benchmarks. > > See https://wiki.postgresql.org/wiki/Pgbench-modernization-2026 for > more background and rationale. > > The series is organized into five logical parts across 22 clean, > self-contained > commits rebased cleanly on latest master: > > ================================================================================ > Part 1: Modular pgbench Refactoring (Patches 0001 - 0012) > ================================================================================ > pgbench.c had grown to over 7,400 lines combining distinct responsibilities: > statistics accumulation, variable management, event polling, script parsing, > meta-command execution, database initialization (-i), and the multi-threaded > state machine. > > Patches 0001-0007 extract these modules without altering behavior or > public APIs: > - stats.c / stats.h: StatsData, latency tally, rate-limiting logic. > - variable.c / variable.h: Scoped variable store and value evaluation. > - poller.c / poller.h: Socket set polling abstraction (select/poll). > - script.c / script.h: Script parsing, AST nodes, expression evaluator. > - commands.c / commands.h: Meta-command handlers and prepared > statement helpers. > - init.c / init.h: Database initialization (-i) logic and data generators. > - engine.c / engine.h: Multi-threaded client state machine and event loop. > > Patches 0008-0012 decouple remaining global dependencies cleanly: > - stats: add mergeStats() and decouple global flags. > - variable: introduce scoped variable store (VariableSpace). > - poller: encapsulate socket_set and provide poller API wrappers. > - script: decouple chooseScript() from TState. > - commands: introduce pluggable meta-command dispatch table. > > ================================================================================ > Part 2: Code Extraction & In-Database Functions (Patches 0013 - 0014) > ================================================================================ > Patch 0013 extracts pgbench's mathematical and pseudo-random > generators (Gaussian, > Exponential, Zipfian, Permute, MurmurHash64) into src/common/pgbench_funcs.c, > making them reusable across both client and server binaries. > > Patch 0014 introduces contrib/pgbench, a server-side extension that > exposes these > pseudo-random generators and helper functions directly to SQL under > SCHEMA pgbench. > This allows high-performance parallel in-database synthetic data > generation using > standard SQL (e.g. generate_series()) without relying on slow client > roundtrips. > > ================================================================================ > Part 3: Advanced Types, Functions, Control Flow & Pacing (Patches 0015 - 0019) > ================================================================================ > Patch 0015 extends PgBenchValue with text, timestamp, and array types, > along with > array operations (array_append, array_length, array_get, array_generate), text > concatenation, and timestamp arithmetic. > > Patch 0016 adds standard benchmark helper functions to pgbench expressions and > contrib/pgbench: > - nurand(A, s, x, y): Non-uniform random distribution generator. > - c_last(num): Syllable-based customer last name generator. > - random_astring(min, max), random_nstring(min, max): Alphanumeric > and numeric strings. > - format_string(fmt, ...), date_add(ts, amt, unit), int_to_date(int). > > Patch 0017 adds script control flow and exception handling: > - Loops: \for <var> <start> .. <end> [BY <step>], \while <expr>, > \break, \continue. > - Exception handling: \try ... \catch [<sqlstate>] ... \endtry, > \expect_error <sqlstate>. > > Patch 0018 adds result set metadata inspection and array column parsing: > - \gset_meta <prefix>: Extracts query result status, error code, > rows affected, and timing. > - \aset and array-aware \gset: Maps multi-row query results into > client-side array variables. > > Patch 0019 introduces dynamic client pacing: > - Dynamic \sleep expressions (e.g. \sleep random_exponential(1, 1000, 10) > ms). > - \keying_time and \think_time commands for realistic transaction > pacing per specification. > > ================================================================================ > Part 4: Heterogeneous Groups, Latency Percentiles & Reporting (Patches > 0020 - 0021) > ================================================================================ > Patch 0020 introduces heterogeneous client groups via --group: > - Run concurrent mixes of different scripts with distinct client > counts and rates > (e.g., --group name=oltp,clients=10,weight=100 --group > name=olap,clients=2,sequential=true). > - Supports sequential stream execution for multi-query analytical streams. > > Patch 0021 implements High Dynamic Range (HDR) latency histogram tracking: > - Configurable percentiles (p50, p90, p95, p99, p99.9). > - SLA validation via --sla target=<metric>,threshold=<value>. > - Metric computation for tpmC and QphH@SF. > - Structured machine-readable JSON output via --json-report=<path>. > > ================================================================================ > Part 5: Standard Workload Suites (Patch 0022) > ================================================================================ > > <THIS WAS LEFT OUT FROM THIS EMAIL> > > * Trademark & Naming Rationale: > These workloads are inspired by and modeled after industry-standard > specifications > (TPC-C, TPC-E, TPC-H, TPC-DS, and CH-benCHmark). To respect and > avoid infringing > upon Transaction Processing Performance Council (TPC) trademarks, > and to clearly > distinguish these community engineering tools from official > council-audited benchmarks, > they are explicitly named pgbench-b, pgbench-c, pgbench-e, > pgbench-h, pgbench-ds, and pgbench-htap. > Full background and guidelines are documented in > src/bin/pgbench/workloads/README. > > * In-Database Parallel Loading: > Each workload includes an initialization script > (<workload>_init.sql) designed to > run in parallel across multiple worker sessions (-c N -j N -t 1 -D > num_workers=N). > Data generation is executed entirely within the database engine > using generate_series() > and contrib/pgbench SQL functions, achieving rapid parallel load > rates without external > file generation or ETL binaries. > > Comprehensive documentation is provided in doc/src/sgml/ref/pgbench.sgml and > doc/src/sgml/pgbench-ext.sgml, and full end-to-end TAP coverage is added in > src/bin/pgbench/t/001_pgbench_with_server.pl (passing all tests across > all platforms). > > Feedback, testing, and reviews are warmly welcomed! > > Hannu Krosing (22): > Refactor pgbench: extract statistics structures and functions to > stats.c/h > Refactor pgbench: extract variable management and value types to > variable.c/h > Refactor pgbench: extract socket polling abstraction to poller.c/h > Refactor pgbench: extract script parsing, built-in scripts, and > expression evaluation to script.c/h > Refactor pgbench: extract meta-command execution and prepared > statement helpers to commands.c/h > Refactor pgbench: extract database initialization (-i) to init.c/h > Refactor pgbench: extract benchmark execution engine and client state > machine to engine.c/h > Refactor pgbench stats: add mergeStats() and decouple global flags > Refactor pgbench variables: introduce scoped variable store > Refactor pgbench poller: encapsulate socket_set and add poller API > aliases > Refactor pgbench script: decouple chooseScript() from TState > Refactor pgbench commands: introduce pluggable meta-command dispatch > Extract pgbench low-level functions into src/common > Add contrib/pgbench extension exposing benchmark functions in SCHEMA > pgbench > pgbench: Extend PgBenchValue with text, timestamp, and array types > pgbench: Add TPC-C NURand, syllable, string, date/timestamp, and array > functions > pgbench: Add script loops (\for, \while) and exception handling (\try, > \catch, \expect_error) > pgbench: Add \gset_meta and array column parsing in \gset and \aset > pgbench: Support dynamic \sleep expressions, \keying_time, and > \think_time > pgbench: Add heterogeneous client groups (--group) and sequential > streams > pgbench: Add HDR latency percentiles, --sla validation, tpmC/QphH > metrics, and --json-report > pgbench: Add pgbench-b, pgbench-c, pgbench-e, pgbench-h, pgbench-ds, > and pgbench-htap workload suites > > --- > Hannu
