Re: [HACKERS] Test code is worth the space
On 13 August 2015 at 00:31, Robert Haas robertmh...@gmail.com wrote: On Wed, Aug 12, 2015 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: FWIW, I've objected in the past to tests that would significantly increase the runtime of make check, unless I thought they were especially valuable (which enumerating every minor behavior of a feature patch generally isn't IMO). I still think that that's an important consideration: every second you add to make check is multiplied many times over when you consider how many developers run that how many times a day. We've talked about having some sort of second rank of tests that people wouldn't necessarily run before committing, and that would be allowed to eat more time than the core regression tests would. I think that might be a valuable direction to pursue if people start submitting very bulky tests. Maybe. Adding a whole new test suite is significantly more administratively complex, because the BF client has to get updated to run it. And if expected outputs in that test suite change very often at all, then committers will have to run it before committing anyway. The value of a core regression suite that takes less time to run has to be weighed against the possibility that a better core regression suite might cause us to find more bugs before committing. That could easily be worth the price in runtime. Seems like a simple fix. We maintain all regression tests in full, but keep slow tests in separate files accessed only by a different schedule. make check == fast-parallel_schedule make check-full == parallel_schedule Probably easier to make one schedule call the other, so we don't duplicate anything. Tom gets his fast schedule, others get their full schedule. -- Simon Riggshttp://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services
Re: [HACKERS] TAP tests are badly named
On Thu, Aug 13, 2015 at 11:17:40AM -0400, Andrew Dunstan wrote: here's what I propose. This changes more than the tapcheck name and the suites it could run. Would you write up the changes you chose to include? That will help guide review. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] replication slot restart_lsn initialization
On Wed, Aug 12, 2015 at 8:20 AM, Andres Freund and...@anarazel.de wrote: On 2015-08-11 15:59:59 -0700, Gurjeet Singh wrote: In your version, I don't see a comment that refers to the fact that it works on the currently active (global variable) slot. Hm? /* * Reserve WAL for the currently active slot. * * Compute and set restart_lsn in a manner that's appropriate for the type of * the slot and concurrency safe. */ I moved values[0] = NameGetDatum(MyReplicationSlot-data.name); nulls[0] = false; to outside the conditional block, there seems no reason to have it in there? The assignment to values[0] is being done twice. We can do away with the one in the else part of the if condition. Ugh, that's a mistake. [1]: I altered you to it in a personal email, but probably it fell through the cracks. I usually just check the lists for newer patch versions, sorry... Commit 6fcd8851, which is the result of this thread, is not touching the replication protocol at all. This looks like an oversight to me: we should be a maximum consistent between the SQL interface and the replication protocol if possible, and it looks useful to me to be able to set restart_lsn when creating the slot as well when using a replication connection. Most of the work has visibly been done with the refactoring that created ReplicationSlotReserveWal, hence how would we expose this option in CREATE_REPLICATION_SLOT? For now we can do that: CREATE_REPLICATION_SLOT IDENT K_PHYSICAL We could append a keyword like RESERVE on this query. Or go through more fancy things like (slot_options) where slot_options is a list of option items, reserve = on/off. Thoughts? -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] replication slot restart_lsn initialization
On 2015-08-14 16:44:44 +0900, Michael Paquier wrote: Commit 6fcd8851, which is the result of this thread, is not touching the replication protocol at all. This looks like an oversight to me: we should be a maximum consistent between the SQL interface and the replication protocol if possible, and it looks useful to me to be able to set restart_lsn when creating the slot as well when using a replication connection. It wasn't, at least not from my side. You can relatively easily do nearly the same just by connecting to the slot and sending a feedback message. The complaint upthread (and/or a related thread) was that it's not possible to do the same from SQL. It'd be a good additional to offer the same facility to the replication protocol. For now we can do that: CREATE_REPLICATION_SLOT IDENT K_PHYSICAL We could append a keyword like RESERVE on this query. Or go through more fancy things like (slot_options) where slot_options is a list of option items, reserve = on/off. Thoughts? -- Michael I'd name it RESERVE_WAL. My feeling is that the options for the logical case are geared towards the output plugin, not the walsender. I think it'd be confusing to use (slot_options) differently for physical slots. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TAP tests are badly named
On 08/14/2015 03:32 AM, Noah Misch wrote:
On Thu, Aug 13, 2015 at 11:17:40AM -0400, Andrew Dunstan wrote:
here's what I propose.
This changes more than the tapcheck name and the suites it could run. Would
you write up the changes you chose to include? That will help guide review.
I don't think it changes anything other than what was discussed. The
code is rearranged a little, and an incorrect piece of code setting
$ENV{PERL5LIB} is fixed (in the case where it was previously empty it
would have added a spurious ; and possibly caused a warning as well).
Instead of looking everywhere in the tree for /t directories, the new
bincheck function only looks for them in src/bin. And the tests would
die on the first failure, as we would also expect the checks run under
make to do.
The effect is to remove the target tapcheck for which there is no
make equivalent, and replace it with the target bincheck, which is
the equivalent of make -C src/bin installcheck, which happens to be
what the buildfarm runs.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] buildfarm does not test make check
On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: So I added the brin isolation test back. Because it needs the pageinspect module, it can only work under make check, not installcheck. The problem is that this means buildfarm will not run it, because it only runs installcheck :-( Aren't make installcheck and make check supposed to test the same set of things? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] buildfarm does not test make check
On 08/14/2015 09:27 AM, Robert Haas wrote: On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: So I added the brin isolation test back. Because it needs the pageinspect module, it can only work under make check, not installcheck. The problem is that this means buildfarm will not run it, because it only runs installcheck :-( Aren't make installcheck and make check supposed to test the same set of things? There are some cases that don't provide installcheck targets. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive
Alexander Korotkov aekorot...@gmail.com writes: On Thu, Aug 6, 2015 at 1:01 PM, Ildus Kurbangaliev i.kurbangal...@postgrespro.ru wrote: This is why I think we shoudn't place wait event into PgBackendStatus. It could be placed into PGPROC or even separate data structure with different concurrency model which would be most suitable for monitoring. +1 for tracking wait events not only for backends Ildus, could you do following? 1) Extract LWLocks refactoring into separate patch. 2) Make a patch with storing current wait event information in PGPROC. What will this accomplish exactly, other than making it more complicated to make a copy of the information when we capture an activity snapshot? You'll have to get data out of two places, which do not have any synchronization protocol defined between them. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive
On Thu, Aug 6, 2015 at 1:01 PM, Ildus Kurbangaliev i.kurbangal...@postgrespro.ru wrote: On 08/05/2015 09:33 PM, Robert Haas wrote: On Wed, Aug 5, 2015 at 1:10 PM, Ildus Kurbangaliev i.kurbangal...@postgrespro.ru wrote: About `memcpy`, PgBackendStatus struct already have a bunch of multi-byte variables, so it will be not consistent anyway if somebody will want to copy it in that way. On the other hand two bytes in this case give less overhead because we can avoid the offset calculations. And as I've mentioned before the class of wait will be useful when monitoring of waits will be extended. You're missing the point. Those multi-byte fields have additional synchronization requirements, as I explained in some detail in my previous email. You can't just wave that away. I see that now. Thank you for the point. I've looked deeper and I found PgBackendStatus to be not a suitable place for keeping information about low level waits. Really, PgBackendStatus is used to track high level information about backend. This is why auxiliary processes don't have PgBackendStatus, because they don't have such information to expose. But when we come to the low level wait events then auxiliary processes are as useful for monitoring as backends are. WAL writer, checkpointer, bgwriter etc are using LWLocks as well. This is certainly unclear why they can't be monitored. This is why I think we shoudn't place wait event into PgBackendStatus. It could be placed into PGPROC or even separate data structure with different concurrency model which would be most suitable for monitoring. +1 for tracking wait events not only for backends Ildus, could you do following? 1) Extract LWLocks refactoring into separate patch. 2) Make a patch with storing current wait event information in PGPROC. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
Re: [HACKERS] why can the isolation tester handle only one waiting process?
Robert Haas wrote: I had the idea that it would be useful to have some regression tests that verify that the deadlock checker works as advertised, because we currently have just about zero test coverage for it. And it's easy enough to write a regression test that causes a simple 2-way deadlock. But you can't test anything more interesting than that, because of this limitation described in the README: Currently, at most one step can be waiting at a time. As long as one step is waiting, subsequent steps are run to completion synchronously. Is there any reason not to try to lift that restriction? (And do we think it's hard?) It's just what we needed at the time, so we didn't press any further. Feel free to lift the restriction if you're so inclined. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TAP tests are badly named
On 08/14/2015 09:27 AM, Andrew Dunstan wrote:
On 08/14/2015 03:32 AM, Noah Misch wrote:
On Thu, Aug 13, 2015 at 11:17:40AM -0400, Andrew Dunstan wrote:
here's what I propose.
This changes more than the tapcheck name and the suites it could
run. Would
you write up the changes you chose to include? That will help guide
review.
I don't think it changes anything other than what was discussed. The
code is rearranged a little, and an incorrect piece of code setting
$ENV{PERL5LIB} is fixed (in the case where it was previously empty it
would have added a spurious ; and possibly caused a warning as
well). Instead of looking everywhere in the tree for /t directories,
the new bincheck function only looks for them in src/bin. And the
tests would die on the first failure, as we would also expect the
checks run under make to do.
The effect is to remove the target tapcheck for which there is no
make equivalent, and replace it with the target bincheck, which is
the equivalent of make -C src/bin installcheck, which happens to be
what the buildfarm runs.
I've just noticed a small error in the patch. Revised version attached.
cheers
andrew
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index d3d736b..e4fa888 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -34,7 +34,7 @@ if (-e src/tools/msvc/buildenv.pl)
my $what = shift || ;
if ($what =~
-/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|tapcheck)$/i
+/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck)$/i
)
{
$what = uc $what;
@@ -61,7 +61,14 @@ unless ($schedule)
$schedule = parallel if ($what eq 'CHECK' || $what =~ /PARALLEL/);
}
-$ENV{PERL5LIB} = $topdir/src/tools/msvc;$ENV{PERL5LIB};
+if ($ENV{PERL5LIB})
+{
+ $ENV{PERL5LIB} = $topdir/src/tools/msvc;$ENV{PERL5LIB};
+}
+else
+{
+ $ENV{PERL5LIB} = $topdir/src/tools/msvc;
+}
my $maxconn = ;
$maxconn = --max_connections=$ENV{MAX_CONNECTIONS}
@@ -81,7 +88,7 @@ my %command = (
CONTRIBCHECK = \contribcheck,
MODULESCHECK = \modulescheck,
ISOLATIONCHECK = \isolationcheck,
- TAPCHECK = \tapcheck,
+ BINCHECK = \bincheck,
UPGRADECHECK = \upgradecheck,);
my $proc = $command{$what};
@@ -168,44 +175,46 @@ sub isolationcheck
exit $status if $status;
}
-sub tapcheck
+sub tap_check
{
- InstallTemp();
+ die Tap tests not enabled in configuration
+ unless $config-{tap_tests};
+
+ my $dir = shift;
+ chdir $dir;
my @args = ( prove, --verbose, t/*.pl);
- my $mstat = 0;
+ # Reset those values, they may have been changed by another test.
+ # XXX is this true?
+ local %ENV = %ENV;
$ENV{PERL5LIB} = $topdir/src/test/perl;$ENV{PERL5LIB};
$ENV{PG_REGRESS} = $topdir/$Config/pg_regress/pg_regress;
+ $ENV{TESTDIR} = $dir;
+
+ system(@args);
+ my $status = $? 8;
+ return $status;
+}
+
+sub bincheck
+{
+ InstallTemp();
+
+ my $mstat = 0;
+
# Find out all the existing TAP tests by looking for t/ directories
# in the tree.
- my $tap_dirs = [];
- my @top_dir = ($topdir);
- File::Find::find(
- { wanted = sub {
-/^t\z/s
- push(@$tap_dirs, $File::Find::name);
- }
- },
- @top_dir);
+ my @bin_dirs = glob($topdir/src/bin/*);
# Process each test
- foreach my $test_path (@$tap_dirs)
+ foreach my $dir (@$bin_dirs)
{
- # Like on Unix make check-world, don't run the SSL test suite
- # automatically.
- next if ($test_path =~ /\/src\/test\/ssl\//);
-
- my $dir = dirname($test_path);
- chdir $dir;
- # Reset those values, they may have been changed by another test.
- $ENV{TESTDIR} = $dir;
- system(@args);
- my $status = $? 8;
- $mstat ||= $status;
+ next unless -d $dir/t;
+ my $status = tap_check($dir);
+ exit $status if $status;
}
- exit $mstat if $mstat;
}
sub plcheck
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] replication slot restart_lsn initialization
On Fri, Aug 14, 2015 at 9:54 AM, Andres Freund and...@anarazel.de wrote: On 2015-08-14 16:44:44 +0900, Michael Paquier wrote: Commit 6fcd8851, which is the result of this thread, is not touching the replication protocol at all. This looks like an oversight to me: we should be a maximum consistent between the SQL interface and the replication protocol if possible, and it looks useful to me to be able to set restart_lsn when creating the slot as well when using a replication connection. It wasn't, at least not from my side. You can relatively easily do nearly the same just by connecting to the slot and sending a feedback message. The complaint upthread (and/or a related thread) was that it's not possible to do the same from SQL. It'd be a good additional to offer the same facility to the replication protocol. For now we can do that: CREATE_REPLICATION_SLOT IDENT K_PHYSICAL We could append a keyword like RESERVE on this query. Or go through more fancy things like (slot_options) where slot_options is a list of option items, reserve = on/off. Thoughts? -- Michael I'd name it RESERVE_WAL. My feeling is that the options for the logical case are geared towards the output plugin, not the walsender. I think it'd be confusing to use (slot_options) differently for physical slots. Yes, but the options list you pass to START_REPLICATION ... LOGICAL, not to CREATE_REPLICATION_SLOT. 2c -- Alex
Re: [HACKERS] replication slot restart_lsn initialization
On 2015-08-14 11:09:38 +0200, Shulgin, Oleksandr wrote: Yes, but the options list you pass to START_REPLICATION ... LOGICAL, not to CREATE_REPLICATION_SLOT. I know, but we might want to extend that at some point. - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Configure checks and optimizations/crc32 tests
Hi, when building with optimizations clang-3.7, probably some older compilers as well, fail with a funny error: fatal error: error in backend: Cannot select: intrinsic %llvm.x86.sse42.crc32.64.64 turns out that is because configure chose to use the crc32 instruction, without the according compiler -msse4.2 flag. Why you ask? Because the whole configure test is optimized away. Nothing depends on it's output... We can easily fix the issue by adding /* return values dependent on crc, to prevent optimizations */ return crc == 0; or so to the end of PGAC_SSE42_CRC32_INTRINSICS. But we've played that whack-a-mole game in a bunch of configure tests now, and it seems likely that more are coming with compilers getting better. I'd not be surprised if some tests actually always succeed, we just don't notice it because it's only on new compilers that have the tested feature anyway... One way trying to fix this would be to explicitly disable optimizations during tests. I'm not sure that's a good idea, but it's one. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] why can the isolation tester handle only one waiting process?
I had the idea that it would be useful to have some regression tests that verify that the deadlock checker works as advertised, because we currently have just about zero test coverage for it. And it's easy enough to write a regression test that causes a simple 2-way deadlock. But you can't test anything more interesting than that, because of this limitation described in the README: Currently, at most one step can be waiting at a time. As long as one step is waiting, subsequent steps are run to completion synchronously. Is there any reason not to try to lift that restriction? (And do we think it's hard?) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] How to compile, link and use a C++ extension
Hello, I want to write a BackgroundWorker in C++ (have lots of C++ code and libs to reuse). I found35.9.13 Using C++ for Extensibility in the 9.3/4 documentation who says it's possible and gives some guidelines for source-code writing. Compiling is not too difficult if one understands this chapter. However I coudn't find any doc. about how to solve the problem of linking C++ code and libs into the .so of my extension, and nothing to solve the runtime-loading problem of the c++ specific .so files ( for ex. to make work a simple usage std::string ) I started my tests by cloning the contrib/worker_spi code, and when transforming the code into C++, I could only note that C++ is not supported in the provided Makefiles. I also found some rather old (2008) message in this mailing list which addresses the same subject: http://www.postgresql.org/message-id/4938e5ed.60...@acm.org: Mostly Harmless: Welcoming our C++ friends with patches that didn't reach their way into the source-distrib , I guess. Is there something more actual about this subject that could help ?. Jacques K.
Re: [HACKERS] why can the isolation tester handle only one waiting process?
Robert Haas wrote: Thanks for the reply. It appears that this is to some degree a semantically relevant restriction. We assume that a step never unblocks except when we run a future step, which is false in the case of the deadlock detector. However, once one step is waiting, we run further steps to completion, which means that there's time for the deadlock detector to kick in after all. To make this useful for deadlock testing, something further is needed. The simplest thing to do seems to be to allow for a way to configure the maximum number of processes that are expected to wait during a particular test. That could default to 1, the current behavior. That's pretty coarse-grained, but I think it would be sufficient for what I want to do. Alternatively, we could try to provide a way to attach information to the step, or within the permutation, indicating the points at which we expect waiters to accumulate and to be released. I'm not sure exactly what that would look like, though. Hmm, clearly you couldn't attach the info to the step itself, because a step that blocks in one permutation doesn't necessarily block in every permutation. You could attach it to each step that needed it in the permutation, but then it wouldn't work to leave permutation specification out for such a test. Maybe that's an acceptable restriction if you cause the test to fail with a specific error instead of stalling forever (which is what happens currently IIRC). -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Raising our compiler requirements for 9.6
Hi, On 2015-08-06 12:43:06 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: Ah, but that's because you cheated and didn't remove the include from namespace.h ... Well, it's not included from frontend code, so I didn't see the need? Going through all the backend code and replacing lock.h by lockdefs.h and some other includes doesn't seem particularly beneficial to me. FWIW, removing it from namespace.h is relatively easy. It starts to get a lot more noisy when you want to touch heapam.h. Ah, heapam.h is the granddaddy of all header messes, isn't it. (Actually it's execnodes.h or something like that.) diff --git a/src/include/storage/lockdefs.h b/src/include/storage/lockdefs.h new file mode 100644 index 000..bfbcdba --- /dev/null +++ b/src/include/storage/lockdefs.h @@ -0,0 +1,56 @@ +/*- + * + * lockdefs.h + *Frontend exposed parts of postgres' low level lock mechanism + * + * The split between lockdefs.h and lock.h is not very principled. No kidding! Do you have a good suggestion about the split? I wanted to expose the minimal amount necessary, and those were the ones. Nope, nothing useful, sorry. To pick up on the general discussion and on the points you made here: I actually think the split came out to work far better than I'd anticipated. Having a slimmed-down version of lock.h for code that just needs to declare/pass lockmode parameters seems to improve our layering considerably. I got round to Alvaro's and Roberts position that removing lock.h from namespace.h and heapam.h would be a really nice improvemen on that front. Attached is a WIP patch to that end. After the further changes only few headers still have to include lock.h and they're pretty firmly in the backend-only territory. It also allows to remove the uglyness of passing around LOCKMODE as an int in parserOpenTable(). Imo lockdefs.h should be updated to describe that it contains the part of the lock infrastructure needed by indirect users of lock.h infrastructure, and that that currently unfortunately may include some frontend programs. I *also* think that removing lwlock.h from lock.h would be a good idea. In my opinion it's a bad idea to pointlessly expose so much low-level things to the wider world, even if it's only needed by relatively low level things. Given that dependent macros are in lwlock.h, it seems pretty sane to move LockHash* there too. We could additionally move all but LOCKMETHODID, LockTagType, LockAcquire*(), LockRelease() and probably one or two more to lock_internals.h (although I'd maybe rather name it lock_details?). I think it'd be an improvement, although possibly not a tremendous one given how many places it's likely going to be included from. Greetings, Andres Freund From f9a74a50c5c02b1a3276385468ae41359741f9fa Mon Sep 17 00:00:00 2001 From: Andres Freund and...@anarazel.de Date: Fri, 14 Aug 2015 19:36:04 +0200 Subject: [PATCH] WIP: Decrease usage of lock.h further. --- contrib/pg_stat_statements/pg_stat_statements.c | 2 ++ src/backend/access/common/reloptions.c | 1 + src/backend/access/heap/syncscan.c | 2 ++ src/backend/access/nbtree/nbtutils.c| 2 ++ src/backend/access/transam/commit_ts.c | 1 + src/backend/catalog/toasting.c | 1 - src/backend/commands/discard.c | 1 + src/backend/commands/policy.c | 1 - src/backend/commands/tablecmds.c| 1 - src/backend/parser/parse_relation.c | 6 +- src/backend/tsearch/ts_typanalyze.c | 1 + src/backend/utils/adt/array_typanalyze.c| 1 + src/backend/utils/cache/ts_cache.c | 1 + src/include/access/heapam.h | 2 +- src/include/access/reloptions.h | 2 +- src/include/catalog/namespace.h | 2 +- src/include/catalog/pg_inherits_fn.h| 2 +- src/include/catalog/toasting.h | 2 +- src/include/commands/cluster.h | 2 +- src/include/commands/tablecmds.h| 2 +- src/include/commands/vacuum.h | 2 +- src/include/nodes/execnodes.h | 1 + src/include/parser/parse_oper.h | 1 + src/include/parser/parse_relation.h | 3 ++- 24 files changed, 25 insertions(+), 17 deletions(-) diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c index 59b8a2e..a3b40e8 100644 --- a/contrib/pg_stat_statements/pg_stat_statements.c +++ b/contrib/pg_stat_statements/pg_stat_statements.c @@ -72,6 +72,8 @@ #include pgstat.h #include storage/fd.h #include storage/ipc.h +#include storage/lwlock.h +#include storage/shmem.h #include storage/spin.h #include tcop/utility.h
Re: [HACKERS] How to compile, link and use a C++ extension
Andres Freund and...@anarazel.de writes: On 2015-08-14 18:38:36 +0200, jacques klein wrote: However I coudn't find any doc. about how to solve the problem of linking C++ code and libs into the .so of my extension, and nothing to solve the runtime-loading problem of the c++ specific .so files ( for ex. to make work a simple usage std::string ) libstdc++ (or whatever your platform uses) should be automatically loaded, I don't think you'll need to worry about that. Yeah. The painful issues you're going to face are not that. They are memory management (C++ new does not talk to palloc or vice versa) and error handling (throw does not interoperate with PG_TRY()). If you can build a bulletproof interface layer between your C++ code and the surrounding C-coded backend, you can make it work, but that part won't be any fun. There's way too much temptation to just call assorted C-coded functions from your C++, and *that won't work* with any reliability. There are discussions of these issues in the pgsql-hackers archives. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Management of simple_eval_estate for plpgsql DO blocks
In commit 0fc94a5ba I wrote: +* ... It's okay to update the [ session-wide ] +* hash table with the new tree because all plpgsql functions within a +* given transaction share the same simple_eval_estate. Um. Well, that's true for actual functions, but plpgsql DO blocks use their own private simple_eval_estate. That means that after a DO block runs, the cast_hash contains dangling pointers to expression eval state trees, which a subsequent plpgsql execution in the same transaction will think are still valid. Ooops. (See bug #13571.) The simplest fix for this would be to give up on the idea that DO blocks use private simple_eval_estates, and make them use the shared one. However, that would result in intra-transaction memory bloat for transactions executing large numbers of DO blocks; see commit c7b849a89, which installed that arrangement to begin with. Since that change was based on a user complaint, this answer doesn't seem appetizing. Or we could try to use the shared simple_eval_estate for CAST expressions even within DO blocks, but I'm afraid that would break things in subtle ways. We need to do actual execution in the block's own eval_estate, or we will have problems with leakage of pass-by-reference cast results because exec_eval_cleanup() won't know to clean them up. It's possible that we could get away with putting the expression state tree into the shared simple_eval_estate's per-query memory and then executing it with the block's private simple_eval_estate, but I'm afraid there are probably places in execQual and/or C functions that suppose that the expression state tree is in the estate's per-query memory. (That is, I doubt that we're totally consistent about whether we use fcinfo-flinfo-fn_mcxt or econtext-ecxt_per_query_memory for long-lived data, in which case an arrangement like this could lead to dangling pointers.) Or we could change things so that DO blocks use private cast_hash hashtables along with their private simple_eval_estates. This would give up some efficiency (since a DO block would then always need to do its own cast lookups) but it would be a simple and reliable fix. I'm kind of inclined to go with the last choice, but I wonder if anyone wants to argue differently, or sees another feasible solution. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to compile, link and use a C++ extension
On 2015-08-14 11:10:14 -0700, Peter Geoghegan wrote: On Fri, Aug 14, 2015 at 10:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah. The painful issues you're going to face are not that. They are memory management (C++ new does not talk to palloc or vice versa) and error handling (throw does not interoperate with PG_TRY()). It's worse than that. Any use of longjmp() will cause undefined behavior in C++. That's because each C++ object's destructor will not be called (possibly other reasons, too). Only if that longjmp goes through C++ code with non-POD objects. Which you should pretty much never allow. - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configure checks and optimizations/crc32 tests
Hi, On 2015-08-14 15:35:45 +0300, Heikki Linnakangas wrote: But we've played that whack-a-mole game in a bunch of configure tests now, and it seems likely that more are coming with compilers getting better. I'd not be surprised if some tests actually always succeed, we just don't notice it because it's only on new compilers that have the tested feature anyway... Yeah, wouldn't be surprised if the other similar tests for all __sync_* family of functions had the same problem. I don't think those are vulnerable because atomics do have a side effect - they're memory barriers. Maybe if we'd specified relaxed (we don't) consistency mode for the __atomic_* ones. One way trying to fix this would be to explicitly disable optimizations during tests. I'm not sure that's a good idea, but it's one. I think adding the return is better. Ok, will do. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to compile, link and use a C++ extension
Hi, On 2015-08-14 18:38:36 +0200, jacques klein wrote: However I coudn't find any doc. about how to solve the problem of linking C++ code and libs into the .so of my extension, and nothing to solve the runtime-loading problem of the c++ specific .so files ( for ex. to make work a simple usage std::string ) libstdc++ (or whatever your platform uses) should be automatically loaded, I don't think you'll need to worry about that. I started my tests by cloning the contrib/worker_spi code, and when transforming the code into C++, I could only note that C++ is not supported in the provided Makefiles. Yes, that doesn't surprise me. Postgres itself doesn't need to care about looking for a usable C++ compiler et al. How exactly do you need to use the C++ code? The easiest probably would be to have a separate object file, built using your own makefile rule, that contains a the C++ and provides a C interface, and then use that from a background worker purely written in C. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Management of simple_eval_estate for plpgsql DO blocks
On 08/14/2015 12:42 PM, Tom Lane wrote: In commit 0fc94a5ba I wrote: +* ... It's okay to update the [ session-wide ] +* hash table with the new tree because all plpgsql functions within a +* given transaction share the same simple_eval_estate. Um. Well, that's true for actual functions, but plpgsql DO blocks use their own private simple_eval_estate. That means that after a DO block runs, the cast_hash contains dangling pointers to expression eval state trees, which a subsequent plpgsql execution in the same transaction will think are still valid. Ooops. (See bug #13571.) The simplest fix for this would be to give up on the idea that DO blocks use private simple_eval_estates, and make them use the shared one. However, that would result in intra-transaction memory bloat for transactions executing large numbers of DO blocks; see commit c7b849a89, which installed that arrangement to begin with. Since that change was based on a user complaint, this answer doesn't seem appetizing. Or we could try to use the shared simple_eval_estate for CAST expressions even within DO blocks, but I'm afraid that would break things in subtle ways. We need to do actual execution in the block's own eval_estate, or we will have problems with leakage of pass-by-reference cast results because exec_eval_cleanup() won't know to clean them up. It's possible that we could get away with putting the expression state tree into the shared simple_eval_estate's per-query memory and then executing it with the block's private simple_eval_estate, but I'm afraid there are probably places in execQual and/or C functions that suppose that the expression state tree is in the estate's per-query memory. (That is, I doubt that we're totally consistent about whether we use fcinfo-flinfo-fn_mcxt or econtext-ecxt_per_query_memory for long-lived data, in which case an arrangement like this could lead to dangling pointers.) Or we could change things so that DO blocks use private cast_hash hashtables along with their private simple_eval_estates. This would give up some efficiency (since a DO block would then always need to do its own cast lookups) but it would be a simple and reliable fix. I'm kind of inclined to go with the last choice, but I wonder if anyone wants to argue differently, or sees another feasible solution. +1 for the last unless someone comes up with something better. Is it going to involve a huge hit anyway? It seems unlikely to matter that much. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Configure checks and optimizations/crc32 tests
Hi, On 2015-08-14 11:20:39 +0200, Andres Freund wrote: when building with optimizations clang-3.7, probably some older compilers as well, fail with a funny error: fatal error: error in backend: Cannot select: intrinsic %llvm.x86.sse42.crc32.64.64 turns out that is because configure chose to use the crc32 instruction, without the according compiler -msse4.2 flag. Why you ask? Because the whole configure test is optimized away. Nothing depends on it's output... We can easily fix the issue by adding /* return values dependent on crc, to prevent optimizations */ return crc == 0; or so to the end of PGAC_SSE42_CRC32_INTRINSICS. Going over my vpaths I noticed another problem with the test. With gcc I get slice-by-8 instead of the runtime variant: checking for builtin __atomic int64 atomic operations... yes checking for __get_cpuid... yes checking for __cpuid... no checking for _mm_crc32_u8 and _mm_crc32_u32 with CFLAGS=... no checking for _mm_crc32_u8 and _mm_crc32_u32 with CFLAGS=-msse4.2... no checking which CRC-32C implementation to use... slicing-by-8 That's because I get a warning conftest.c:179:1: warning: old-style function definition [-Wold-style-definition] main () ^ and PGAC_SSE42_CRC32_INTRINSICS turns on ac_c_werror_flag. Now I can work around this by , but I don't really see why that test needs to turn on -Werror? Isn't the point of using a linker test that we'd get a proper linker failure if the functions don't exist? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to compile, link and use a C++ extension
On Fri, Aug 14, 2015 at 10:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah. The painful issues you're going to face are not that. They are memory management (C++ new does not talk to palloc or vice versa) and error handling (throw does not interoperate with PG_TRY()). It's worse than that. Any use of longjmp() will cause undefined behavior in C++. That's because each C++ object's destructor will not be called (possibly other reasons, too). I suggest looking at the PL/V8 code for an example of how to make C++ code work as a Postgres extension. IIRC they've made specific trade-offs that might be useful for Jacques' use case too. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] How to compile, link and use a C++ extension
On 08/14/2015 02:10 PM, Peter Geoghegan wrote: On Fri, Aug 14, 2015 at 10:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah. The painful issues you're going to face are not that. They are memory management (C++ new does not talk to palloc or vice versa) and error handling (throw does not interoperate with PG_TRY()). It's worse than that. Any use of longjmp() will cause undefined behavior in C++. That's because each C++ object's destructor will not be called (possibly other reasons, too). I suggest looking at the PL/V8 code for an example of how to make C++ code work as a Postgres extension. IIRC they've made specific trade-offs that might be useful for Jacques' use case too. Yeah, although be aware that PLv8 still has odd buggy behaviours that I am not sure are not related to some C/C++ impedance mismatch. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why can the isolation tester handle only one waiting process?
On Fri, Aug 14, 2015 at 10:14 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: I had the idea that it would be useful to have some regression tests that verify that the deadlock checker works as advertised, because we currently have just about zero test coverage for it. And it's easy enough to write a regression test that causes a simple 2-way deadlock. But you can't test anything more interesting than that, because of this limitation described in the README: Currently, at most one step can be waiting at a time. As long as one step is waiting, subsequent steps are run to completion synchronously. Is there any reason not to try to lift that restriction? (And do we think it's hard?) It's just what we needed at the time, so we didn't press any further. Feel free to lift the restriction if you're so inclined. Thanks for the reply. It appears that this is to some degree a semantically relevant restriction. We assume that a step never unblocks except when we run a future step, which is false in the case of the deadlock detector. However, once one step is waiting, we run further steps to completion, which means that there's time for the deadlock detector to kick in after all. To make this useful for deadlock testing, something further is needed. The simplest thing to do seems to be to allow for a way to configure the maximum number of processes that are expected to wait during a particular test. That could default to 1, the current behavior. That's pretty coarse-grained, but I think it would be sufficient for what I want to do. Alternatively, we could try to provide a way to attach information to the step, or within the permutation, indicating the points at which we expect waiters to accumulate and to be released. I'm not sure exactly what that would look like, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] FDW question - how to identify columns to populate in response?
Hi, I'm working on a FDW for the unix/linux user database - think /etc/passwd and /etc/group although I'm actually using system calls that could be quietly redirected to LDAP or other backends. It's easy to create the FDW and a table associated with it, something like CREATE TABLE passwd ( name text, passwd text, uid int, ... The problem is the user could decide to reorder or remove columns so I can't make the assumption that values[0] is always going to be the username. I have a solution that requires looking at the rel, extracting the atts, and then doing a loop where I check the attname against all possible values for each column. Anything that doesn't match is set to null. This isn't too bad here but it would be a pain if there are many columns. Is there a cleaner way? I've looked at a number of other FDW implementations but they are generally mapping columns to columns (so it's a short bit of lookup code inside the loop), not copying data provided by a system call. Thanks, Bear
[HACKERS] Re: [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
2015-08-14 20:58 GMT-03:00 Dickson S. Guedes lis...@guedesoft.net: Hello hackers, * Description This patch is a proposal Now patch attached, sorry. -- Dickson S. Guedes mail/xmpp: gue...@guedesoft.net - skype: guediz http://github.com/guedes - http://guedesoft.net http://www.postgresql.org.br allow_semester_in_extract_data_part_to_char.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] FDW question - how to identify columns to populate in response?
Bear Giles bgi...@coyotesong.com writes: Hi, I'm working on a FDW for the unix/linux user database - think /etc/passwd and /etc/group although I'm actually using system calls that could be quietly redirected to LDAP or other backends. It's easy to create the FDW and a table associated with it, something like CREATE TABLE passwd ( name text, passwd text, uid int, ... The problem is the user could decide to reorder or remove columns so I can't make the assumption that values[0] is always going to be the username. I have a solution that requires looking at the rel, extracting the atts, and then doing a loop where I check the attname against all possible values for each column. Anything that doesn't match is set to null. This isn't too bad here but it would be a pain if there are many columns. Is there a cleaner way? The only thing that comes to mind is that you could probably amortize the figure-out-the-mapping overhead over multiple tuples. There's surely no reason to do it more often than once per query; and if that's not good-enough performance you could think about caching it longer, with some sort of invalidation logic. Take a look at src/include/access/tupconvert.h and src/backend/access/common/tupconvert.c for inspiration, or maybe even code you can use directly. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] alter-table-1 isolation test spec contains a duplicate step name
Commit f14a6bbedb79adce2298d0d4f5e2abe8563e0eca added an isolation
test, alter-table-1.spec, which reads in part:
session s2
setup { BEGIN; }
step rx1 { SELECT * FROM b WHERE a_id = 1 LIMIT 1; }
step wx { INSERT INTO b VALUES (0); }
step rx1 { SELECT * FROM b WHERE a_id = 3 LIMIT 3; }
step c2 { COMMIT; }
The step name rx1 is used twice, a condition the isolation tester
should probably reject as an error but currently doesn't. This
happens to work OK at the moment because we don't specify a list of
permutations, and except for that, the names aren't used for anything
that cares about whether they are duplicated. Nonetheless, this seems
like it should probably be changed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alter-table-1 isolation test spec contains a duplicate step name
Robert Haas robertmh...@gmail.com writes:
Commit f14a6bbedb79adce2298d0d4f5e2abe8563e0eca added an isolation
test, alter-table-1.spec, which reads in part:
session s2
setup { BEGIN; }
step rx1 { SELECT * FROM b WHERE a_id = 1 LIMIT 1; }
step wx { INSERT INTO b VALUES (0); }
step rx1 { SELECT * FROM b WHERE a_id = 3 LIMIT 3; }
step c2 { COMMIT; }
The step name rx1 is used twice, a condition the isolation tester
should probably reject as an error but currently doesn't.
+1 for throwing an error ... this is clearly a mistake.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
Hello hackers, * Description This patch is a proposal to allow the use of word 'semester' to extract it from date in functions like EXTRACT, DATE_PART, etc and adds the letter 'S' to format the date output in to_char. ** Example SELECT EXTRACT(semester FROM DATE '2015-07-07'); date_part --- 2 * Motivation The term is used in a school or college to represent a half-year. Actually it could be evaluated from a date using some math with 'quarter' but could not be extracted from date since the API to define a reserved word for EXTRACT is in the core, rather than in SQL. * Syntax I'm using the word 'semester' since it is common in college and school but I wondering if it should be other word (like 'halfyear', for example) Is this a feature that worth? BTW, I'll put this in the commit fest (2015-09). Thanks! -- Dickson S. Guedes mail/xmpp: gue...@guedesoft.net - skype: guediz http://github.com/guedes - http://guedesoft.net http://www.postgresql.org.br -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Raising our compiler requirements for 9.6
On Fri, Aug 14, 2015 at 2:40 PM, Andres Freund and...@anarazel.de wrote: To pick up on the general discussion and on the points you made here: I actually think the split came out to work far better than I'd anticipated. Having a slimmed-down version of lock.h for code that just needs to declare/pass lockmode parameters seems to improve our layering considerably. I got round to Alvaro's and Roberts position that removing lock.h from namespace.h and heapam.h would be a really nice improvemen on that front. Attached is a WIP patch to that end. After the further changes only few headers still have to include lock.h and they're pretty firmly in the backend-only territory. It also allows to remove the uglyness of passing around LOCKMODE as an int in parserOpenTable(). Yes, I like this. Imo lockdefs.h should be updated to describe that it contains the part of the lock infrastructure needed by indirect users of lock.h infrastructure, and that that currently unfortunately may include some frontend programs. Sure. I *also* think that removing lwlock.h from lock.h would be a good idea. In my opinion it's a bad idea to pointlessly expose so much low-level things to the wider world, even if it's only needed by relatively low level things. Given that dependent macros are in lwlock.h, it seems pretty sane to move LockHash* there too. Hmm. I dunno, lwlock.h is a pretty hideous layering violation as it is. I'd like to find a way to make that better, not worse. We could additionally move all but LOCKMETHODID, LockTagType, LockAcquire*(), LockRelease() and probably one or two more to lock_internals.h (although I'd maybe rather name it lock_details?). I think it'd be an improvement, although possibly not a tremendous one given how many places it's likely going to be included from. What benefit would we get out of this? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
On Fri, Aug 14, 2015 at 8:40 PM, David G. Johnston david.g.johns...@gmail.com wrote: Tbh, quarter in (1,2) does not seem that problematic... Yeah. It's so easy to compute the half from the quarter (or even from the month) that I can't see adding this. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why can the isolation tester handle only one waiting process?
Robert Haas wrote: On Fri, Aug 14, 2015 at 2:57 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Hmm, clearly you couldn't attach the info to the step itself, because a step that blocks in one permutation doesn't necessarily block in every permutation. You could attach it to each step that needed it in the permutation, but then it wouldn't work to leave permutation specification out for such a test. Maybe that's an acceptable restriction if you cause the test to fail with a specific error instead of stalling forever (which is what happens currently IIRC). After some study, I think the best thing to do here is change the way we handle the case where we reach a step that the use of a connection that is currently blocked on a lock. Right now, we handle that by declaring the permutation invalid; I'd like to change that so that consider that a cue to wait for that connnection to unblock itself. This will require a number of tests that currently blindly run through all permutations to specify a list of permutations, or they will hang. Well, hanging forever may not be all that great. Buildfarm animals with test processes stuck probably won't be happy. Maybe put a cap on the time we're willing to wait; something like a minute should suffice for all reasonable tests. At the same time I wonder if iterating as quickly as possible is really such a hot idea; why don't we sleep even 100ms if nothing is to be done immediately? That would reduce log traffic if you have log_statements=all, for one thing ... I guess (from a patch author perspective) we can just use isolationtester -n to produce appropriate permutation lines when developing a spec file, and then prune the ones causing trouble. FWIW I tried this with the spec I posted at http://www.postgresql.org/message-id/20141212205254.gc1...@alvh.no-ip.org and it seems to work fine (modulo a bug in the spec itself). I didn't try reverting the patch that fixed the bug. But I'm not sure that's such a bad thing, because running through all permutations in those cases provides no additional test coverage. Each invalid permutation runs the sequence of steps only up until the point where it chooses an invalid next step. Therefore, each invalid permutation is testing an initial prefix of the steps tested by some valid permutation. If the invalid permutation ceased to be invalid, because the command at which we give up returned immediately rather than waiting, that would also change the test output of the other, valid test of which it is the initial prefix. And therefore, at least as it seems to me, testing the invalid permutations is just a waste of CPU time, and we'd be better off not doing it. Well, the number of tests that actually exercise this is not large. More time is spent in the timeout test, ISTM (even though the CPU is sleeping during that, but it's still wasted clock time). Actually, I'm really rather wondering if the list of valid permutations should also be pruned for some of these tests. Some of these output files are thousands of lines long, and I'm not sure that somebody has really gone through that whole file and made sure that the output of each permutation is expected. And I'm sure some of them are functionally identical. No objections there, but alter-table-1 and alter-table-2 seem to be the only tests that have thousands of lines long of expected output and also have invalid permutations in the expected output. The only others with 1k+ lines are two-ids, receipt-reports and prepared-transactions, which don't have invalid permutations. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] why can the isolation tester handle only one waiting process?
Robert Haas robertmh...@gmail.com writes: After some study, I think the best thing to do here is change the way we handle the case where we reach a step that the use of a connection that is currently blocked on a lock. Right now, we handle that by declaring the permutation invalid; I'd like to change that so that consider that a cue to wait for that connnection to unblock itself. Maybe a timeout there would be a good idea? I see that the code currently will wait forever in some cases, which seems bad enough, but it seems like you want to increase the risk of that considerably. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
2015-08-14 21:40 GMT-03:00 David G. Johnston david.g.johns...@gmail.com: On Friday, August 14, 2015, Dickson S. Guedes lis...@guedesoft.net wrote: The term is used in a school or college to represent a half-year. Actually it could be evaluated from a date using some math with 'quarter' but could not be extracted from date since the API to define a reserved word for EXTRACT is in the core, rather than in SQL. If this is wanted a more consistent word would be half since we already accept quarter. In your opinion: half or halfyear ? I quite dislike semester because of its connotations in education and the fact that actual semesters do not align with calendar halves. I agree, my native language is pt_BR, and it is very common to use semestre ( semester ), but reading some text around the Internet I see half-year more often. Thank you. -- Dickson S. Guedes mail/xmpp: gue...@guedesoft.net - skype: guediz http://github.com/guedes - http://guedesoft.net http://www.postgresql.org.br -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
David G. Johnston david.g.johns...@gmail.com writes: On Friday, August 14, 2015, Dickson S. Guedes lis...@guedesoft.net wrote: The term is used in a school or college to represent a half-year. I quite dislike semester because of its connotations in education and the fact that actual semesters do not align with calendar halves. It's worse than that: some institutions use semester for third-of-a-year intervals (also called trimesters). I've also heard of places that use a fairly standard Sept-to-May calendar but call summer classes a third semester, even though it's shorter than their regular class sessions. I don't think we should accept a patch along this line, because it assigns a very specific meaning to semester that does not square all that well with real-world usage. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
2015-08-15 0:55 GMT-03:00 Tom Lane t...@sss.pgh.pa.us:
I don't think we should accept a patch along this line, because it assigns
a very specific meaning to semester that does not square all that well
with real-world usage.
I agree that semester was not nice, the real meaning is half or
halfyear as mentioned in other messages. That's it, I was proposing
the extraction of the first or second half year using a specific word
like in:
select extract('halfyear' from date '2015-08-15')); or
select date_trunc('halfyear', date '2015-08-15');
rather than trying to overloading `date_part` with a UDF or
calculating half like:
select ceil(date_part('quarter', date '2015-08-15') / 2);
BTW, I feel that I have enough feedback to set patch as rejected.
Thank you for your time.
--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net - skype: guediz
http://github.com/guedes - http://guedesoft.net
http://www.postgresql.org.br
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
On 15/08/15 11:58, Dickson S. Guedes wrote: Hello hackers, * Description This patch is a proposal to allow the use of word 'semester' to extract it from date in functions like EXTRACT, DATE_PART, etc and adds the letter 'S' to format the date output in to_char. ** Example SELECT EXTRACT(semester FROM DATE '2015-07-07'); date_part --- 2 * Motivation The term is used in a school or college to represent a half-year. Actually it could be evaluated from a date using some math with 'quarter' but could not be extracted from date since the API to define a reserved word for EXTRACT is in the core, rather than in SQL. * Syntax I'm using the word 'semester' since it is common in college and school but I wondering if it should be other word (like 'halfyear', for example) Is this a feature that worth? BTW, I'll put this in the commit fest (2015-09). Thanks! Note that the start and end points for a semester might well be different between universities in the Northern Southern Hemispheres of this planet- also between institutions in the same hemisphere. The first semester in 2015 for the University of Auckland begins Monday 2 March 2015: https://www.auckland.ac.nz/en/for/current-students/cs-academic-information/cs-academic-dates.html The first semester in 2015 for ANU (Australian National University) begins Monday 16 February 2015: http://www.anu.edu.au/directories/university-calendar So semesters don't appear to align with normal half year boundaries. Cheers, Gavin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
On 15/08/15 13:06, Dickson S. Guedes wrote:
2015-08-14 21:32 GMT-03:00 Gavin Flower gavinflo...@archidevsys.co.nz:
...
So semesters don't appear to align with normal half year boundaries.
Interesting links, thanks!
Which sounds better for a native English: 'half', 'halfyear'?
For example:
SELECT date_trunc('halfyear', current_date);
date_trunc
2015-07-01 00:00:00-03
(1 row)
Thanks!
I would go with 'halfyear', but I don't have a use case for it, so don't
put too much weight on my opinion of this!
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Test code is worth the space
On 8/14/15 12:11 AM, Jim Nasby wrote: I favor splitting the regression tests to add all the time and before commit targets as you describe. I think that once the facility is there, we can determine over time how expansive that second category gets to be. I don't know how many folks work in a github fork of Postgres, but anyone that does could run slow checks on every single push via Travis-CI. I setup a simple example of this with 64 variations of TAP tests, BLKSZ and WAL blocksize. Unfortunately to make this work you have to commit a .travis.yml file to your fork. build: https://travis-ci.org/decibel/postgres/builds/75692344 .travis.yml: https://github.com/decibel/postgres/blob/master/.travis.yml Looks like we might have some problems with BLKSZ != 8... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] alter-table-1 isolation test spec contains a duplicate step name
On Fri, Aug 14, 2015 at 5:18 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Commit f14a6bbedb79adce2298d0d4f5e2abe8563e0eca added an isolation
test, alter-table-1.spec, which reads in part:
session s2
setup { BEGIN; }
step rx1 { SELECT * FROM b WHERE a_id = 1 LIMIT 1; }
step wx { INSERT INTO b VALUES (0); }
step rx1 { SELECT * FROM b WHERE a_id = 3 LIMIT 3; }
step c2 { COMMIT; }
The step name rx1 is used twice, a condition the isolation tester
should probably reject as an error but currently doesn't.
+1 for throwing an error ... this is clearly a mistake.
OK, done. I made it throw an error, and changed the step name to rx3.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
On Friday, August 14, 2015, Dickson S. Guedes lis...@guedesoft.net wrote: The term is used in a school or college to represent a half-year. Actually it could be evaluated from a date using some math with 'quarter' but could not be extracted from date since the API to define a reserved word for EXTRACT is in the core, rather than in SQL. If this is wanted a more consistent word would be half since we already accept quarter. I quite dislike semester because of its connotations in education and the fact that actual semesters do not align with calendar halves. Tbh, quarter in (1,2) does not seem that problematic... David J.
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
2015-08-14 21:32 GMT-03:00 Gavin Flower gavinflo...@archidevsys.co.nz:
...
So semesters don't appear to align with normal half year boundaries.
Interesting links, thanks!
Which sounds better for a native English: 'half', 'halfyear'?
For example:
SELECT date_trunc('halfyear', current_date);
date_trunc
2015-07-01 00:00:00-03
(1 row)
Thanks!
--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net - skype: guediz
http://github.com/guedes - http://guedesoft.net
http://www.postgresql.org.br
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [RFC] allow 'semester' in functions EXTRACT, DATE_PART, TO_CHAR and so on.
On Fri, Aug 14, 2015 at 6:11 PM, Dickson S. Guedes lis...@guedesoft.net wrote: 2015-08-14 21:40 GMT-03:00 David G. Johnston david.g.johns...@gmail.com: On Friday, August 14, 2015, Dickson S. Guedes lis...@guedesoft.net wrote: The term is used in a school or college to represent a half-year. Actually it could be evaluated from a date using some math with 'quarter' but could not be extracted from date since the API to define a reserved word for EXTRACT is in the core, rather than in SQL. If this is wanted a more consistent word would be half since we already accept quarter. In your opinion: half or halfyear ? We don't say quarteryear so there would need to be a compelling argument for halfyear over simply half David J.
Re: [HACKERS] commitfest app bug/feature
On Thu, May 7, 2015 at 9:17 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: Hello, Some comments/bug report about the commitfest web app which just bite me:-) When creating a patch with the commitfest app, if the thread does not exist the application seems to reject the creation, so you may resubmit because you know the thread is going to exist soon... Well, this is a bad idea:-) Indeed the first submission, although it looked like it had failed, has really been created anyway, and the resubmission creates another entry. If you click again, you get a third. And so on. Clearly I'm very slow at some things, but at least I've just pushed a fix for this bug now. It validates the thread exists *before* it creates the entry now... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] Configure checks and optimizations/crc32 tests
On 08/14/2015 12:20 PM, Andres Freund wrote: Hi, when building with optimizations clang-3.7, probably some older compilers as well, fail with a funny error: fatal error: error in backend: Cannot select: intrinsic %llvm.x86.sse42.crc32.64.64 turns out that is because configure chose to use the crc32 instruction, without the according compiler -msse4.2 flag. Why you ask? Because the whole configure test is optimized away. Nothing depends on it's output... Oh, interesting. We can easily fix the issue by adding /* return values dependent on crc, to prevent optimizations */ return crc == 0; or so to the end of PGAC_SSE42_CRC32_INTRINSICS. Yeah, seems like a simple fix. But we've played that whack-a-mole game in a bunch of configure tests now, and it seems likely that more are coming with compilers getting better. I'd not be surprised if some tests actually always succeed, we just don't notice it because it's only on new compilers that have the tested feature anyway... Yeah, wouldn't be surprised if the other similar tests for all __sync_* family of functions had the same problem. One way trying to fix this would be to explicitly disable optimizations during tests. I'm not sure that's a good idea, but it's one. I think adding the return is better. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] query in PostgreSQL
On Fri, Aug 14, 2015 at 9:32 PM, Rahul Majumdar/TVM/TCS rahul.majum...@tcs.com wrote: Hello Everyone, I have a problem related to query in PostgreSQL.Some queries are not running.Please help me to find out a solution. Hi! You've reached the Postgres hackers mailing list where Postgres internals are discussed. The best place to start might be pgsql-general mailing list or the #postgresql IRC channel, where you could explain what you're trying to do and post the errors you're seeing etc and hopefully get some help. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] query in PostgreSQL
Hello Everyone, I have a problem related to query in PostgreSQL.Some queries are not running.Please help me to find out a solution. Thanks Rahul =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you
