On Thu, Aug 30, 2018 at 9:35 PM, Magnus Hagander <mag...@hagander.net>
wrote:
> On Thu, Aug 30, 2018 at 4:39 PM, Tom Lane <t...@sss.pgh.pa.us> wrote:
>
>> Fabien COELHO <coe...@cri.ensmp.fr> writes:
>> >> If I add -fno-strict-aliasing to $CFLAGS, the problem goes away.
>> >> Is this something to worry about, or just pilot error cause I am not
>> >> using the same $CFLAGS as for the rest of the build? I originally
>> >> noticed this problem with my external fork of pg_verify_checksums.
>>
>> > It looks like the code is using aliasing where the standard says it
>> should
>> > not, which breaks compiler optimization, and the added options tells
>> the
>> > compiler to not assume that the code conforms to the standard...
>>
>> Actually, this code is just plain broken:
>>
>> char buf[BLCKSZ];
>> PageHeader header = (PageHeader) buf;
>>
>> There is no guarantee that a local char[] array is aligned on anything
>> stronger than a byte boundary, and if it isn't, word-sized accesses to
>> *header are going to fail on machines with strict alignment rules.
>> I suppose that that is really what Michael's compiler is moaning about.
>>
>> I rather suspect that this hasn't been tested on anything but Intel
>> hardware, which is famously misalignment-tolerant. The lack of any
>> apparent regression test infrastructure for it isn't leaving a warm
>> feeling about how much the buildfarm is testing it.
>>
>
> I know I certainly didn't test it on non-intel.
>
> We did have that in the online verify checksum patch, but it came out as
> part of the revert of that part.
>
> Given that we also recently found bugs in the actual hash index
> implementation that would've gotten caught by this, perhaps we should add a
> TAP test for this.
>
> Should we make it a separate test in pg_verify_checksums, or should we
> piggyback on the pg_basebackup tests (which AFAICT is the only ones that
> create a cluster with checksums enabled at all, and thus is the only
> codepath that actually uses the backend checksum code at all, which I think
> is an even worse thing to have no tests of)
>
PFA some *very* basic tests for pg_verify_checksums, which should at least
be enough to catch the kind of errors we had now in the tool itself.
Does not at this point try to solve the fact that the backend checksum code
is not tested.
(The right fix, of course, is to malloc the work buffer rather than
>> put it on the stack.)
>
>
> So if I get you right, you're saying the attached patch should be all
> that's needed?
>
>
--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>
diff --git a/src/bin/pg_verify_checksums/Makefile b/src/bin/pg_verify_checksums/Makefile
index d16261571f..cfe4ab1b8b 100644
--- a/src/bin/pg_verify_checksums/Makefile
+++ b/src/bin/pg_verify_checksums/Makefile
@@ -34,3 +34,9 @@ uninstall:
clean distclean maintainer-clean:
rm -f pg_verify_checksums$(X) $(OBJS)
rm -rf tmp_check
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/bin/pg_verify_checksums/t/010_pg_verify_checksums.pl b/src/bin/pg_verify_checksums/t/010_pg_verify_checksums.pl
new file mode 100644
index 0000000000..b6d535d76b
--- /dev/null
+++ b/src/bin/pg_verify_checksums/t/010_pg_verify_checksums.pl
@@ -0,0 +1,33 @@
+use strict;
+use warnings;
+use Cwd;
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 10;
+
+program_help_ok('pg_verify_checksums');
+program_version_ok('pg_verify_checksums');
+program_options_handling_ok('pg_verify_checksums');
+
+my $node = get_new_node('main');
+
+# Set umask so test directories and files are created with default permissions
+umask(0077);
+
+# Initialize node without replication settings
+$node->init(extra => ['--data-checksums']);
+
+# Verify checksums of fresh node
+$node->command_ok(['pg_verify_checksums', '-D', $node->data_dir()]);
+
+# Cause some corruption in pg_proc
+open my $file, '+<', $node->data_dir() . "/base/1/1255";
+seek($file, 131, 0);
+syswrite($file, '\0\0\0\0\0\0\0\0\0');
+seek($file, 15721, 0);
+syswrite($file, '\0\0\0\0\0\0\0\0\0');
+close $file;
+
+# Verify that checksums are now broken
+$node->command_fails(['pg_verify_checksums', '-D', $node->data_dir()]);
