On Fri, Mar 14, 2025 at 3:43 PM Andres Freund <and...@anarazel.de> wrote:
>
> Open items:
>
> - Right now effective_io_concurrency cannot be set > 0 on Windows and other
> platforms that lack posix_fadvise. But with AIO we can read ahead without
> posix_fadvise().
>
> It'd not really make anything worse than today to not remove the limit, but
> it'd be pretty weird to prevent windows etc from benefiting from AIO. Need
> to look around and see whether it would require anything other than doc
> changes.
I've attached a patch that removes the limit for
effective_io_concurrency and maintenance_io_concurrency. I tested both
GUCs with fadvise manually disabled on my system and I think it is
working for those read stream users I tried (vacuum and BHS).
I checked around to make sure no one was using only the value of the
guc to guard prefetches, and it seems like we're safe.
The one thing I am wondering about with the docs is whether or not we
need to make it more clear that only a subset of the "simultaneous
I/O" behavior controlled by eic/mic is available if your system
doesn't have fadvise. I tried to do that a bit, but I avoided getting
into too many details.
- Melanie
From 398282293c38f9dfb4f03839a4a7962887c9c0c9 Mon Sep 17 00:00:00 2001
From: Melanie Plageman <melanieplage...@gmail.com>
Date: Mon, 17 Mar 2025 15:17:21 -0400
Subject: [PATCH v1] Enable IO concurrency on all systems
Previously effective_io_concurrency and maintenance_io_concurrency could
not be set above 0 on machines without fadvise support. AIO enables
IO concurrency without such support.
Currently only subsystems using the read stream API will take advantage
of this. Other users of maintenance_io_concurrency (like recovery
prefetching) which leverage OS advice directly will not benefit from
this change. In those cases, maintenance_io_concurrency will have no
effect on I/O behavior.
---
doc/src/sgml/config.sgml | 15 +++++-----
doc/src/sgml/ref/alter_tablespace.sgml | 2 +-
doc/src/sgml/ref/create_tablespace.sgml | 2 +-
src/backend/access/common/reloptions.c | 8 -----
src/backend/commands/variable.c | 30 -------------------
src/backend/utils/misc/guc_tables.c | 4 +--
src/backend/utils/misc/postgresql.conf.sample | 5 ++--
src/bin/initdb/initdb.c | 5 ----
src/include/storage/bufmgr.h | 6 ----
src/include/utils/guc_hooks.h | 4 ---
10 files changed, 14 insertions(+), 67 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 69638d7c1f9..2a7587de320 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2581,8 +2581,7 @@ include_dir 'conf.d'
session attempts to initiate in parallel. The allowed range is
<literal>1</literal> to <literal>1000</literal>, or
<literal>0</literal> to disable issuance of asynchronous I/O requests.
- The default is <literal>16</literal> on supported systems, otherwise
- <literal>0</literal>.
+ The default is <literal>16</literal>.
</para>
<para>
@@ -2593,8 +2592,8 @@ include_dir 'conf.d'
</para>
<para>
- On systems without prefetch advice support, attempting to configure
- any value other than <literal>0</literal> will error out.
+ On systems with prefetch advice support,
+ <varname>effective_io_concurrency</varname> also controls the prefetch distance.
</para>
<para>
@@ -2617,10 +2616,10 @@ include_dir 'conf.d'
for maintenance work that is done on behalf of many client sessions.
</para>
<para>
- The default is 10 on supported systems, otherwise 0. This value can
- be overridden for tables in a particular tablespace by setting the
- tablespace parameter of the same name (see
- <xref linkend="sql-altertablespace"/>).
+ The default is <literal>10</literal>. This value can be overridden
+ for tables in a particular tablespace by setting the tablespace
+ parameter of the same name (see <xref
+ linkend="sql-altertablespace"/>).
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/alter_tablespace.sgml b/doc/src/sgml/ref/alter_tablespace.sgml
index 6ec863400d1..d0e08089ddb 100644
--- a/doc/src/sgml/ref/alter_tablespace.sgml
+++ b/doc/src/sgml/ref/alter_tablespace.sgml
@@ -88,7 +88,7 @@ ALTER TABLESPACE <replaceable>name</replaceable> RESET ( <replaceable class="par
and <varname>maintenance_io_concurrency</varname>.
Setting these values for a particular tablespace will override the
planner's usual estimate of the cost of reading pages from tables in
- that tablespace, and the executor's prefetching behavior, as established
+ that tablespace, and how many concurrent I/Os are issued, as established
by the configuration parameters of the
same name (see <xref linkend="guc-seq-page-cost"/>,
<xref linkend="guc-random-page-cost"/>,
diff --git a/doc/src/sgml/ref/create_tablespace.sgml b/doc/src/sgml/ref/create_tablespace.sgml
index 9d5ab025261..b77e774c53f 100644
--- a/doc/src/sgml/ref/create_tablespace.sgml
+++ b/doc/src/sgml/ref/create_tablespace.sgml
@@ -110,7 +110,7 @@ CREATE TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
and <varname>maintenance_io_concurrency</varname>.
Setting these values for a particular tablespace will override the
planner's usual estimate of the cost of reading pages from tables in
- that tablespace, and the executor's prefetching behavior, as established
+ that tablespace, and how many concurrent I/Os are issued, as established
by the configuration parameters of the
same name (see <xref linkend="guc-seq-page-cost"/>,
<xref linkend="guc-random-page-cost"/>,
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 59fb53e7707..3e43ad064c3 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -361,11 +361,7 @@ static relopt_int intRelOpts[] =
RELOPT_KIND_TABLESPACE,
ShareUpdateExclusiveLock
},
-#ifdef USE_PREFETCH
-1, 0, MAX_IO_CONCURRENCY
-#else
- 0, 0, 0
-#endif
},
{
{
@@ -374,11 +370,7 @@ static relopt_int intRelOpts[] =
RELOPT_KIND_TABLESPACE,
ShareUpdateExclusiveLock
},
-#ifdef USE_PREFETCH
-1, 0, MAX_IO_CONCURRENCY
-#else
- 0, 0, 0
-#endif
},
{
{
diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c
index 4ad6e236d69..7188afd2860 100644
--- a/src/backend/commands/variable.c
+++ b/src/backend/commands/variable.c
@@ -1145,7 +1145,6 @@ check_cluster_name(char **newval, void **extra, GucSource source)
void
assign_maintenance_io_concurrency(int newval, void *extra)
{
-#ifdef USE_PREFETCH
/*
* Reconfigure recovery prefetching, because a setting it depends on
* changed.
@@ -1153,7 +1152,6 @@ assign_maintenance_io_concurrency(int newval, void *extra)
maintenance_io_concurrency = newval;
if (AmStartupProcess())
XLogPrefetchReconfigure();
-#endif
}
@@ -1231,34 +1229,6 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
-bool
-check_effective_io_concurrency(int *newval, void **extra, GucSource source)
-{
-#ifndef USE_PREFETCH
- if (*newval != 0)
- {
- GUC_check_errdetail("\"%s\" must be set to 0 on platforms that lack support for issuing read-ahead advice.",
- "effective_io_concurrency");
- return false;
- }
-#endif /* USE_PREFETCH */
- return true;
-}
-
-bool
-check_maintenance_io_concurrency(int *newval, void **extra, GucSource source)
-{
-#ifndef USE_PREFETCH
- if (*newval != 0)
- {
- GUC_check_errdetail("\"%s\" must be set to 0 on platforms that lack support for issuing read-ahead advice.",
- "maintenance_io_concurrency");
- return false;
- }
-#endif /* USE_PREFETCH */
- return true;
-}
-
bool
check_ssl(bool *newval, void **extra, GucSource source)
{
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 7290793c9a1..2c1ca98a3e5 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3224,7 +3224,7 @@ struct config_int ConfigureNamesInt[] =
&effective_io_concurrency,
DEFAULT_EFFECTIVE_IO_CONCURRENCY,
0, MAX_IO_CONCURRENCY,
- check_effective_io_concurrency, NULL, NULL
+ NULL, NULL, NULL
},
{
@@ -3238,7 +3238,7 @@ struct config_int ConfigureNamesInt[] =
&maintenance_io_concurrency,
DEFAULT_MAINTENANCE_IO_CONCURRENCY,
0, MAX_IO_CONCURRENCY,
- check_maintenance_io_concurrency, assign_maintenance_io_concurrency,
+ NULL, assign_maintenance_io_concurrency,
NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ea71e9075ff..7bd7479190d 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -198,8 +198,9 @@
# - I/O -
#backend_flush_after = 0 # measured in pages, 0 disables
-#effective_io_concurrency = 16 # 1-1000; 0 disables prefetching
-#maintenance_io_concurrency = 10 # 1-1000; 0 disables prefetching
+#effective_io_concurrency = 16 # 1-1000; 0 disables issuing multiple simultaneous IO requests
+ # values > 0 enable prefetching on systems supporting it
+#maintenance_io_concurrency = 10 # 1-1000; same as effective_io_concurrency
#io_combine_limit = 128kB # usually 1-32 blocks (depends on OS)
#io_method = worker # worker, io_uring, sync
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 21a0fe3ecd9..b9d75799ee0 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1401,11 +1401,6 @@ setup_config(void)
repltok, true);
#endif
-#ifndef USE_PREFETCH
- conflines = replace_guc_value(conflines, "effective_io_concurrency",
- "0", true);
-#endif
-
#ifdef WIN32
conflines = replace_guc_value(conflines, "update_process_title",
"off", true);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 30d965ec069..c425cc816e8 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -156,14 +156,8 @@ extern PGDLLIMPORT int bgwriter_lru_maxpages;
extern PGDLLIMPORT double bgwriter_lru_multiplier;
extern PGDLLIMPORT bool track_io_timing;
-/* only applicable when prefetching is available */
-#ifdef USE_PREFETCH
#define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16
#define DEFAULT_MAINTENANCE_IO_CONCURRENCY 10
-#else
-#define DEFAULT_EFFECTIVE_IO_CONCURRENCY 0
-#define DEFAULT_MAINTENANCE_IO_CONCURRENCY 0
-#endif
extern PGDLLIMPORT int effective_io_concurrency;
extern PGDLLIMPORT int maintenance_io_concurrency;
diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h
index a3eba8fbe21..74de7a6abdf 100644
--- a/src/include/utils/guc_hooks.h
+++ b/src/include/utils/guc_hooks.h
@@ -61,8 +61,6 @@ extern bool check_default_text_search_config(char **newval, void **extra, GucSou
extern void assign_default_text_search_config(const char *newval, void *extra);
extern bool check_default_with_oids(bool *newval, void **extra,
GucSource source);
-extern bool check_effective_io_concurrency(int *newval, void **extra,
- GucSource source);
extern bool check_huge_page_size(int *newval, void **extra, GucSource source);
extern void assign_io_method(int newval, void *extra);
extern bool check_io_max_concurrency(int *newval, void **extra, GucSource source);
@@ -83,8 +81,6 @@ extern bool check_log_stats(bool *newval, void **extra, GucSource source);
extern bool check_log_timezone(char **newval, void **extra, GucSource source);
extern void assign_log_timezone(const char *newval, void *extra);
extern const char *show_log_timezone(void);
-extern bool check_maintenance_io_concurrency(int *newval, void **extra,
- GucSource source);
extern void assign_maintenance_io_concurrency(int newval, void *extra);
extern bool check_max_slot_wal_keep_size(int *newval, void **extra,
GucSource source);
--
2.34.1
