Re: [HACKERS] Revisiting Re: BUG #8532: postgres fails to start with timezone-data =2013e
Ian Stakenvicius a...@gentoo.org writes: Hey all -- so I know that Gentoo Linux is likely the only platform this bug occurs under, but i got annoyed enough with it that I decided to write a patch to fix this issue once and for all (or at least, help keep it from happening). That thread in question actually dealt with crashing on startup in postgresql-9.1 and earlier, but all versions including the latest still suffer from the inability to load timezone data via the pg_timezone_* tables if /usr/share/zoneinfo contains filesystem loops. To that end, the following helps resolve this issue by ensuring single-level filesystem loops are detected and skipped. The top half of the patch only applies to postgresql-9.1 and earlier, while the second half applies to all versions. This patch doesn't look right at all. In the first place, tzdirsub is the entire subpath, not necessarily just one filename. In the second place, limiting the strncmp comparison to strlen(direntry-d_name) exposes you to false matches --- for example, a current d_name of foo would be thought to match a tzdirsub path of foobar. In the third place, relying on basename(1) does not seem advisable for portability reasons --- use something from src/port/path.c instead. It would probably be a good idea to be more specific about what sorts of loops you are hoping to catch, because this surely won't detect all possible loops as-is. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW oddity
Andrew Dunstan and...@dunslane.net writes: I have just noticed something slightly odd. The traces (obtained by setting client_min_messages to debug1) from the blackhole FDW show that the handler function is called each time an INSERT is performed. This is not the case for SELECT, UPDATE or DELETE. It looks at first glance like a buglet. Can anyone suggest why this should be so? What do you mean by the handler function, and for that matter what do you define as each time? The ExecForeignInsert method certainly ought to be called once per row inserted, but the same is true of ExecForeignUpdate and ExecForeignDelete. The setup methods such as BeginForeignModify should only be called once per query though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] FDW oddity
I have just noticed something slightly odd. The traces (obtained by setting client_min_messages to debug1) from the blackhole FDW show that the handler function is called each time an INSERT is performed. This is not the case for SELECT, UPDATE or DELETE. It looks at first glance like a buglet. Can anyone suggest why this should be so? cheers andrerw -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] improving speed of make check-world
On Sat, Apr 11, 2015 at 4:35 AM, Peter Eisentraut pete...@gmx.net wrote:
On 3/9/15 2:51 AM, Michael Paquier wrote:
On Sun, Mar 8, 2015 at 10:46 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
Speaking of which, attached is a patch rewritten in-line with those
comments, simplifying a bit the whole at the same time. Note this
patch changes ecpgcheck as it should be patched, but as ecpgcheck test
is broken even on HEAD, I'll raise a separate thread about that with a
proper fix (for example bowerbird skips this test).
Correction: HEAD is fine, but previous patch was broken because it
tried to use --top-builddir. Also for ecpgcheck it looks that tricking
PATH is needed to avoid dll loading errors related to libecpg.dll and
libecpg_compat.dll. Updated patch is attached.
To clarify: Are you of the opinion that your patch (on top of my base
patch) is sufficient to make all of this work on Windows?
Things will work. I just tested again each test target with the patch
attached while wrapping again my mind on this stuff (actually found
one bug in plcheck where --bindir was not correct, and removed
tmp_install in upgradecheck). Now, what this patch does is enforcing
the temporary install for each *check target of vcregress.pl. This has
the disadvantage of making the benefits of MAKELEVEL=0 seen for build
methods using the Makefiles go away for MSVC, but it minimizes the use
of ENV variables which is a good thing to me by setting --bindir to a
value as much as possible (ecpgcheck being an exception), and it makes
the set of tests more consistent with each other in the way they run.
Another thing to know that this patch changes is that vcregress does
not rely anymore on the contents of Release/$projects, aka the build
structure of MS stuff, but just fetches binaries from the temporary
installation. That's more consistent with Makefile builds, now perhaps
some people have a different opinion.
What we could add later on a new target, allcheck, that would kick all
the tests at once and installs the temporary installation just once.
It would be straight-forward to write a patch, but this is a separate
discussion as installcheck would need to be skipped. isolationcheck
should as well be changed to be more self-dependent as even of HEAD it
needs to have an instance of PG running to work.
Regards,
--
Michael
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index bd3dd2c..d9ff7ec 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -16,6 +16,7 @@ my $startdir = getcwd();
chdir ../../.. if (-d ../../../src/tools/msvc);
my $topdir = getcwd();
+my $tmp_installdir = $topdir/tmp_install;
require 'src/tools/msvc/config_default.pl';
require 'src/tools/msvc/config.pl' if (-f 'src/tools/msvc/config.pl');
@@ -94,7 +95,7 @@ sub installcheck
my @args = (
../../../$Config/pg_regress/pg_regress,
--dlpath=.,
- --psqldir=../../../$Config/psql,
+ --bindir=../../../$Config/psql,
--schedule=${schedule}_schedule,
--encoding=SQL_ASCII,
--no-locale);
@@ -106,15 +107,19 @@ sub installcheck
sub check
{
+ chdir $startdir;
+
+ InstallTemp();
+ chdir ${topdir}/src/test/regress;
+
my @args = (
- ../../../$Config/pg_regress/pg_regress,
+ ${tmp_installdir}/bin/pg_regress,
--dlpath=.,
- --psqldir=../../../$Config/psql,
+ --bindir=${tmp_installdir}/bin,
--schedule=${schedule}_schedule,
--encoding=SQL_ASCII,
--no-locale,
- --temp-install=./tmp_check,
- --top-builddir=\$topdir\);
+ --temp-instance=./tmp_check);
push(@args, $maxconn) if $maxconn;
push(@args, $temp_config) if $temp_config;
system(@args);
@@ -128,18 +133,20 @@ sub ecpgcheck
system(msbuild ecpg_regression.proj /p:config=$Config);
my $status = $? 8;
exit $status if $status;
+ InstallTemp();
chdir $topdir/src/interfaces/ecpg/test;
+
+ $ENV{PATH} = ${tmp_installdir}/bin;${tmp_installdir}/lib;$ENV{PATH};
$schedule = ecpg;
my @args = (
- ../../../../$Config/pg_regress_ecpg/pg_regress_ecpg,
- --psqldir=../../../$Config/psql,
+ ${tmp_installdir}/bin/pg_regress_ecpg,
+ --bindir=,
--dbname=regress1,connectdb,
--create-role=connectuser,connectdb,
--schedule=${schedule}_schedule,
--encoding=SQL_ASCII,
--no-locale,
- --temp-install=./tmp_chk,
- --top-builddir=\$topdir\);
+ --temp-instance=./tmp_chk);
push(@args, $maxconn) if $maxconn;
system(@args);
$status = $? 8;
@@ -148,12 +155,14 @@ sub ecpgcheck
sub isolationcheck
{
- chdir ../isolation;
- copy(../../../$Config/isolationtester/isolationtester.exe,
- ../../../$Config/pg_isolation_regress);
+ chdir $startdir;
+
+ InstallTemp();
+ chdir ${topdir}/src/test/isolation;
+
my @args = (
- ../../../$Config/pg_isolation_regress/pg_isolation_regress,
- --psqldir=../../../$Config/psql,
+ ${tmp_installdir}/bin/pg_isolation_regress,
+ --bindir=${tmp_installdir}/bin,
--inputdir=.,
--schedule=./isolation_schedule);
push(@args, $maxconn) if $maxconn;
@@ -164,7 +173,10 @@ sub
Re: [HACKERS] PATCH: Spinlock Documentation
Hello, Thank you again for your feedback. I have improved the patch with your suggestions. Please let me know what you think and if I can do anything else. Current CommitFest link for this patch is: https://commitfest.postgresql.org/5/208/ Respectfully, Artem Luzyanin On Sunday, April 5, 2015 5:59 PM, Artem Luzyanin lisyono...@yahoo.ca wrote: Hello, Thank you very much for your feedback! I will work on the changes as soon as I can. Respectfully, Artem Luzyanin On Sunday, April 5, 2015 5:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: David Fetter da...@fetter.org writes: One issue with this patch is that it is not localized. If someone goes and changes the S_LOCK implementation for one of the platforms below, or adds a new platform, etc., without changing this comment too, this comment becomes confusingly obsolete. Indeed. Moreover, this header comment is supposed to be an overview and specification of the macros that need to be provided. I think it's an actively bad idea to clutter it with platform-by-platform details; that will create a can't see the forest for the trees problem. If we need more info here, I think a comment block before each section of the file would make more sense. But the patch as provided seems like it would just be redundant if it were refactored in that form. What would possibly be useful that's not there now is a paragraph or two describing the overall layout of the file (eg gcc then non gcc, or whatever can be said at more or less that level of detail). But please don't stick that into the middle of the specification part. regards, tom lane spinlock-docsV2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] moving from contrib to bin
On 3/11/15 8:21 PM, Michael Paquier wrote: Attached is a series of patch rebased on current HEAD, there were some conflicts after perl-tidying the refactoring patch for MSVC. Note that this series still uses PGXS in the Makefiles, I am fine to update them if necessary once this matter is set (already did this stuff upthread with a previous version). OK, here we go. I have committed the pg_archivecleanup move, with a complete makefile and your Windows help. Let's see how it builds. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP Patch for GROUPING SETS phase 1
Dear: I am using postgesql 9.4.0. Thanks for your great work on grouping sets patch effort. I am now compiling postgresql from source code 9.4.0 on Linux platform with [gsp-all.patch] successed and grouping function well, but failed on window platform(windows 2003 or window 7). It shows unrecognized keywords: grouping, cube, etc. Good suggestions or tips? thanks a lot! henry
[HACKERS] Patch to improve a few appendStringInfo* calls
I've attached a small patch which just fixes a few appendStringInfo* calls that are not quite doing things the way that it was intended. Regards David Rowley appendstringinfo_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW oddity
On 04/11/2015 05:10 PM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
I have just noticed something slightly odd. The traces (obtained by
setting client_min_messages to debug1) from the blackhole FDW show that
the handler function is called each time an INSERT is performed. This is
not the case for SELECT, UPDATE or DELETE. It looks at first glance like
a buglet. Can anyone suggest why this should be so?
What do you mean by the handler function, and for that matter what do
you define as each time? The ExecForeignInsert method certainly ought
to be called once per row inserted, but the same is true of
ExecForeignUpdate and ExecForeignDelete. The setup methods such as
BeginForeignModify should only be called once per query though.
I mean this function:
Datum
blackhole_fdw_handler(PG_FUNCTION_ARGS)
{
FdwRoutine *fdwroutine = makeNode(FdwRoutine);
elog(DEBUG1,entering function %s,__func__);
/* assign the handlers for the FDW */
...
}
And it is called at the beginning of every INSERT statement, before
blackholePlanForeignModify() is called
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] improving speed of make check-world
On Sat, Apr 11, 2015 at 8:48 PM, Michael Paquier wrote: Now, what this patch does is enforcing the temporary install for each *check target of vcregress.pl. This has the disadvantage of making the benefits of MAKELEVEL=0 seen for build methods using the Makefiles go away for MSVC A trick that we could use here is to store a timestamp when running up to completion build and the temporary installation in vcregress, and skip the temporary installation if timestamp of vcregress' temporary installation is newer than the one of build. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Replication identifiers, take 4
On 10/04/15 18:03, Andres Freund wrote: On 2015-04-07 17:08:16 +0200, Andres Freund wrote: I'm starting benchmarks now. What I'm benchmarking here is the WAL overhead, since that's what we're debating. The test setup I used was a pgbench scale 10 instance. I've run with full_page_write=off to have more reproducible results. This of course over-emphasizes the overhead a bit. But for a long checkpoint interval and a memory resident working set it's not that unrealistic. I ran 50k transactions in a signle b baseline: - 20445024 - 20437128 - 20436864 - avg: 20439672 extern 2byte identifiers: - 23318368 - 23148648 - 23128016 - avg: 23198344 - avg overhead: 13.5% padding 2byte identifiers: - 21160408 - 21319720 - 21164280 - avg: 21214802 - avg overhead: 3.8% extern 4byte identifiers: - 23514216 - 23540128 - 23523080 - avg: 23525808 - avg overhead: 15.1% To me that shows pretty clearly that a) reusing the padding is worthwhile b) even without that using 2byte instead of 4 byte identifiers is beneficial. My opinion is that 10% of WAL size difference is quite high price to pay so that we can keep the padding for some other, yet unknown feature that hasn't come up in several years, which would need those 2 bytes. But if we are willing to pay it then we can really go all the way and just use Oids... -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Remove some duplicated words in comments
Attached is a small patch which removes some duplicated words that have crept into some comments. Regards David Rowley duplicate_words_in_comments_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW oddity
On 04/11/2015 07:30 PM, Andrew Dunstan wrote:
On 04/11/2015 05:10 PM, Tom Lane wrote:
Andrew Dunstan and...@dunslane.net writes:
I have just noticed something slightly odd. The traces (obtained by
setting client_min_messages to debug1) from the blackhole FDW show that
the handler function is called each time an INSERT is performed.
This is
not the case for SELECT, UPDATE or DELETE. It looks at first glance
like
a buglet. Can anyone suggest why this should be so?
What do you mean by the handler function, and for that matter what do
you define as each time? The ExecForeignInsert method certainly ought
to be called once per row inserted, but the same is true of
ExecForeignUpdate and ExecForeignDelete. The setup methods such as
BeginForeignModify should only be called once per query though.
I mean this function:
Datum
blackhole_fdw_handler(PG_FUNCTION_ARGS)
{
FdwRoutine *fdwroutine = makeNode(FdwRoutine);
elog(DEBUG1,entering function %s,__func__);
/* assign the handlers for the FDW */
...
}
And it is called at the beginning of every INSERT statement, before
blackholePlanForeignModify() is called
FYI, The FDW is available here:
https://bitbucket.org/adunstan/blackhole_fdw
And here are the traces (notice the second call to blackhole_fdw_handler):
[andrew@emma inst.bhtest.5709]$ bin/psql
psql (9.5devel)
Type help for help.
andrew=# create extension blackhole_fdw;
CREATE EXTENSION
andrew=# create server blackhole foreign data wrapper blackhole_fdw;
CREATE SERVER
andrew=# create foreign table bh(a text, b int) server blackhole;
CREATE FOREIGN TABLE
andrew=# set client_min_messages = debug1;
SET
andrew=# insert into bh select 'a',x from generate_series(1,5) x;
DEBUG: entering function blackhole_fdw_handler
DEBUG: entering function blackholePlanForeignModify
DEBUG: entering function blackhole_fdw_handler
DEBUG: entering function blackholeBeginForeignModify
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeEndForeignModify
INSERT 0 5
andrew=# insert into bh select 'a',x from generate_series(1,5) x;
DEBUG: entering function blackhole_fdw_handler
DEBUG: entering function blackholePlanForeignModify
DEBUG: entering function blackholeBeginForeignModify
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeExecForeignInsert
DEBUG: entering function blackholeEndForeignModify
INSERT 0 5
andrew=#
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
