Hi,
I was recently reminded of my previous desire to allow setting the segment
size to less than 1GB. It's pretty painful to test large amount of segments
with a segment size of 1GB, certainly our regression test don't cover anything
with multiple segments.
This likely wouldn't have detected the issue fixed in 0e758ae89a2, but it make
it easier to validate that the fix doesn't break anything badly.
In the attached patch I renamed --with-segsize= to --with-segsize-mb= /
-Dsegsize= to -Dsegsize_mb=, to avoid somebody building with --with-segsize=2
or such suddenly ending up with an incompatible build.
Greetings,
Andres Freund
>From e8f67e3c27ff553f6a3a26947cac8df91136e8f5 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Mon, 7 Nov 2022 09:05:35 -0800
Subject: [PATCH] allow-smaller-segsize
---
doc/src/sgml/installation.sgml | 6 +++---
doc/src/sgml/storage.sgml | 2 +-
.cirrus.yml | 1 +
configure | 27 ++++++++++++++-------------
configure.ac | 12 ++++++------
meson.build | 6 ++++--
meson_options.txt | 4 ++--
7 files changed, 31 insertions(+), 27 deletions(-)
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 319c7e69660..20a98c3887c 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1458,13 +1458,13 @@ build-postgresql:
</varlistentry>
<varlistentry>
- <term><option>--with-segsize=<replaceable>SEGSIZE</replaceable></option></term>
+ <term><option>--with-segsize-mb=<replaceable>SEGSIZE</replaceable></option></term>
<listitem>
<para>
- Set the <firstterm>segment size</firstterm>, in gigabytes. Large tables are
+ Set the <firstterm>segment size</firstterm>, in megabytes. Large tables are
divided into multiple operating-system files, each of size equal
to the segment size. This avoids problems with file size limits
- that exist on many platforms. The default segment size, 1 gigabyte,
+ that exist on many platforms. The default segment size, 1024 megabytes,
is safe on all supported platforms. If your operating system has
<quote>largefile</quote> support (which most do, nowadays), you can use
a larger segment size. This can be helpful to reduce the number of
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index e5b9f3f1ffa..03e0bb0ecbf 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -236,7 +236,7 @@ When a table or index exceeds 1 GB, it is divided into gigabyte-sized
filenode; subsequent segments are named filenode.1, filenode.2, etc.
This arrangement avoids problems on platforms that have file size limitations.
(Actually, 1 GB is just the default segment size. The segment size can be
-adjusted using the configuration option <option>--with-segsize</option>
+adjusted using the configuration option <option>--with-segsize-mb</option>
when building <productname>PostgreSQL</productname>.)
In principle, free space map and visibility map forks could require multiple
segments as well, though this is unlikely to happen in practice.
diff --git a/.cirrus.yml b/.cirrus.yml
index 9f2282471a9..b0543f20175 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -377,6 +377,7 @@ task:
-Dextra_lib_dirs=${brewpath}/lib \
-Dcassert=true \
-Dssl=openssl -Duuid=e2fs -Ddtrace=auto \
+ -Dsegsize_mb=1 \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
build
diff --git a/configure b/configure
index 3966368b8d9..fb55792f51b 100755
--- a/configure
+++ b/configure
@@ -843,7 +843,7 @@ enable_coverage
enable_dtrace
enable_tap_tests
with_blocksize
-with_segsize
+with_segsize_mb
with_wal_blocksize
with_CC
with_llvm
@@ -1553,7 +1553,8 @@ Optional Packages:
--with-pgport=PORTNUM set default port number [5432]
--with-blocksize=BLOCKSIZE
set table block size in kB [8]
- --with-segsize=SEGSIZE set table segment size in GB [1]
+ --with-segsize-mb=SEGSIZE
+ set table segment size in MB [1024]
--with-wal-blocksize=BLOCKSIZE
set WAL block size in kB [8]
--with-CC=CMD set compiler (deprecated)
@@ -3740,32 +3741,32 @@ $as_echo_n "checking for segment size... " >&6; }
-# Check whether --with-segsize was given.
-if test "${with_segsize+set}" = set; then :
- withval=$with_segsize;
+# Check whether --with-segsize-mb was given.
+if test "${with_segsize_mb+set}" = set; then :
+ withval=$with_segsize_mb;
case $withval in
yes)
- as_fn_error $? "argument required for --with-segsize option" "$LINENO" 5
+ as_fn_error $? "argument required for --with-segsize-mb option" "$LINENO" 5
;;
no)
- as_fn_error $? "argument required for --with-segsize option" "$LINENO" 5
+ as_fn_error $? "argument required for --with-segsize-mb option" "$LINENO" 5
;;
*)
- segsize=$withval
+ segsize_mb=$withval
;;
esac
else
- segsize=1
+ segsize_mb=1024
fi
# this expression is set up to avoid unnecessary integer overflow
# blocksize is already guaranteed to be a factor of 1024
-RELSEG_SIZE=`expr '(' 1024 / ${blocksize} ')' '*' ${segsize} '*' 1024`
+RELSEG_SIZE=`expr '(' 1024 / ${blocksize} ')' '*' ${segsize_mb}`
test $? -eq 0 || exit 1
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ${segsize}GB" >&5
-$as_echo "${segsize}GB" >&6; }
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: ${segsize_mb}MB" >&5
+$as_echo "${segsize_mb}MB" >&6; }
cat >>confdefs.h <<_ACEOF
@@ -15555,7 +15556,7 @@ _ACEOF
# If we don't have largefile support, can't handle segsize >= 2GB.
-if test "$ac_cv_sizeof_off_t" -lt 8 -a "$segsize" != "1"; then
+if test "$ac_cv_sizeof_off_t" -lt 8 -a "$segsize_mb" -gt "1024"; then
as_fn_error $? "Large file support is not enabled. Segment size cannot be larger than 1GB." "$LINENO" 5
fi
diff --git a/configure.ac b/configure.ac
index f76b7ee31fc..6abefa0de58 100644
--- a/configure.ac
+++ b/configure.ac
@@ -286,14 +286,14 @@ AC_DEFINE_UNQUOTED([BLCKSZ], ${BLCKSZ}, [
# Relation segment size
#
AC_MSG_CHECKING([for segment size])
-PGAC_ARG_REQ(with, segsize, [SEGSIZE], [set table segment size in GB [1]],
- [segsize=$withval],
- [segsize=1])
+PGAC_ARG_REQ(with, segsize-mb, [SEGSIZE], [set table segment size in MB [1024]],
+ [segsize_mb=$withval],
+ [segsize_mb=1024])
# this expression is set up to avoid unnecessary integer overflow
# blocksize is already guaranteed to be a factor of 1024
-RELSEG_SIZE=`expr '(' 1024 / ${blocksize} ')' '*' ${segsize} '*' 1024`
+RELSEG_SIZE=`expr '(' 1024 / ${blocksize} ')' '*' ${segsize_mb}`
test $? -eq 0 || exit 1
-AC_MSG_RESULT([${segsize}GB])
+AC_MSG_RESULT([${segsize_mb}MB])
AC_DEFINE_UNQUOTED([RELSEG_SIZE], ${RELSEG_SIZE}, [
RELSEG_SIZE is the maximum number of blocks allowed in one disk file.
@@ -1737,7 +1737,7 @@ dnl Check for largefile support (must be after AC_SYS_LARGEFILE)
AC_CHECK_SIZEOF([off_t])
# If we don't have largefile support, can't handle segsize >= 2GB.
-if test "$ac_cv_sizeof_off_t" -lt 8 -a "$segsize" != "1"; then
+if test "$ac_cv_sizeof_off_t" -lt 8 -a "$segsize_mb" -gt "1024"; then
AC_MSG_ERROR([Large file support is not enabled. Segment size cannot be larger than 1GB.])
fi
diff --git a/meson.build b/meson.build
index ce2f223a409..eb242d24665 100644
--- a/meson.build
+++ b/meson.build
@@ -418,7 +418,9 @@ meson_bin = find_program(meson_binpath, native: true)
cdata.set('USE_ASSERT_CHECKING', get_option('cassert') ? 1 : false)
-cdata.set('BLCKSZ', get_option('blocksize').to_int() * 1024, description:
+blocksize = get_option('blocksize').to_int() * 1024
+segsize = (get_option('segsize_mb') * 1024 * 1024) / blocksize
+cdata.set('BLCKSZ', blocksize, description:
'''Size of a disk block --- this also limits the size of a tuple. You can set
it bigger if you need bigger tuples (although TOAST should reduce the need
to have large tuples, since fields can be spread across multiple tuples).
@@ -428,7 +430,7 @@ cdata.set('BLCKSZ', get_option('blocksize').to_int() * 1024, description:
Changing BLCKSZ requires an initdb.''')
cdata.set('XLOG_BLCKSZ', get_option('wal_blocksize').to_int() * 1024)
-cdata.set('RELSEG_SIZE', get_option('segsize') * 131072)
+cdata.set('RELSEG_SIZE', segsize)
cdata.set('DEF_PGPORT', get_option('pgport'))
cdata.set_quoted('DEF_PGPORT_STR', get_option('pgport').to_string())
cdata.set_quoted('PG_KRB_SRVNAM', get_option('krb_srvnam'))
diff --git a/meson_options.txt b/meson_options.txt
index c7ea57994dc..b0bd3f4cbe8 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -10,8 +10,8 @@ option('wal_blocksize', type : 'combo',
value: '8',
description : '''WAL block size, in kilobytes''')
-option('segsize', type : 'integer', value : 1,
- description : '''Segment size, in gigabytes''')
+option('segsize_mb', type : 'integer', value : 1024,
+ description : '''Segment size, in megabytes''')
# Miscellaneous options
--
2.38.0
