date:20150317

On Tue, Mar 17, 2015 at 10:11 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
 A way to work around this is to leave the ForeignPaths (there can possibly be
 only one foreign path per join relation) in the joinrel without removing 
 them.
 FDW should work on joining two relations if they have foreign paths in the 
 list
 of paths, irrespective of whether the cheapest path is foreign join path or 
 not.
 For the topmost joinrel, if the foreign path happens to be the cheapest one, 
 the
 whole join tree will be pushed down.

 On the other thread implementing foreign join for postgres_fdw,
 postgresGetForeignJoinPaths(), is just looking at the cheapest path, which 
 would
 cause the problem you have described above.

 It might be an idea if foreign-scan path is not wiped out regardless of the
 estimated cost, we will be able to construct an entirely remote-join path
 even if intermediation path is expensive than local join.
 A problem is, how to distinct these special paths from usual paths that are
 eliminated on the previous stage once its path is more expensive.

Any solution that is based on not eliminating paths that would
otherwise be discarded based on cost seems to me to be unlikely to be
feasible.  We can't complicate the core path-cost-comparison stuff for
the convenience of FDW or custom-scan pushdown.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Future directions for inheritance-hierarchy statistics

For some reason, I didn't get Tom's email, only this reply.

On Tue, Mar 17, 2015 at 3:44 AM, Etsuro Fujita
fujita.ets...@lab.ntt.co.jp wrote:
 On 2015/03/17 5:18, Tom Lane wrote:
 A few days ago I posted a very-much-WIP patch for making the planner
 dynamically combine statistics for each member of an appendrel:
 http://www.postgresql.org/message-id/22598.1425686...@sss.pgh.pa.us

 That patch was only intended to handle the case of an appendrel generated
 by a UNION ALL construct.  But it occurs to me that we could easily
 change it to also apply to appendrels generated from inheritance trees.
 Then we'd no longer need the whole-inheritance-tree statistics that
 ANALYZE currently produces, because we'd only ever look at per-table
 statistics in pg_statistic.

 This would have one significant drawback, which is that planning for
 large inheritance trees (many children) would probably get noticeably
 slower.  (But in the common case that constraint exclusion limits a
 query to scanning just one or a few children, the hit would be small.)

That's a pretty big drawback.  I'm not sure whether it's big enough to
sink the whole idea, but we really need to make planning time on large
inheritance trees cheaper, not more expensive.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Future directions for inheritance-hierarchy statistics

Robert Haas robertmh...@gmail.com writes:
 For some reason, I didn't get Tom's email, only this reply.
 On 2015/03/17 5:18, Tom Lane wrote:
 This would have one significant drawback, which is that planning for
 large inheritance trees (many children) would probably get noticeably
 slower.  (But in the common case that constraint exclusion limits a
 query to scanning just one or a few children, the hit would be small.)

 That's a pretty big drawback.  I'm not sure whether it's big enough to
 sink the whole idea, but we really need to make planning time on large
 inheritance trees cheaper, not more expensive.

Ah, but note the point about how there's no added cost for partitions that
are removed by constraint exclusion.  That should mean that in practical
use it's not a huge problem.  (If you're going to scan K partitions, you
should not be surprised that planning time is O(K).  It will be anyway
thanks to other things such as index selection.)

Also, you're ignoring the prospect of getting better estimates and hence
better plans through having stats that dynamically adapt to the set of
partitions being scanned.  Given the lousy state of maintenance of
whole-tree stats, I really think that this consideration might outweigh
even a significant planning-time hit.  Shaving planning time by producing
crappy estimates isn't usually a good tradeoff.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Ashutosh Bapat

On Tue, Mar 17, 2015 at 8:34 PM, Robert Haas robertmh...@gmail.com wrote:

 On Tue, Mar 17, 2015 at 10:11 AM, Kouhei Kaigai kai...@ak.jp.nec.com
 wrote:
  A way to work around this is to leave the ForeignPaths (there can
 possibly be
  only one foreign path per join relation) in the joinrel without
 removing them.
  FDW should work on joining two relations if they have foreign paths in
 the list
  of paths, irrespective of whether the cheapest path is foreign join
 path or not.
  For the topmost joinrel, if the foreign path happens to be the cheapest
 one, the
  whole join tree will be pushed down.
 
  On the other thread implementing foreign join for postgres_fdw,
  postgresGetForeignJoinPaths(), is just looking at the cheapest path,
 which would
  cause the problem you have described above.
 
  It might be an idea if foreign-scan path is not wiped out regardless of
 the
  estimated cost, we will be able to construct an entirely remote-join path
  even if intermediation path is expensive than local join.
  A problem is, how to distinct these special paths from usual paths that
 are
  eliminated on the previous stage once its path is more expensive.

 Any solution that is based on not eliminating paths that would
 otherwise be discarded based on cost seems to me to be unlikely to be
 feasible.  We can't complicate the core path-cost-comparison stuff for
 the convenience of FDW or custom-scan pushdown.


We already have a precedence here. We cache different cheapest paths e.g
 439 struct Path *cheapest_startup_path;
 440 struct Path *cheapest_total_path;
 441 struct Path *cheapest_unique_path;
 442 List   *cheapest_parameterized_paths;

All we have to do is add yet another there cheapest_foreign_path which
can be NULL like cheapest_unique_path.


 --
 Robert Haas
 EnterpriseDB: http://www.enterprisedb.com
 The Enterprise PostgreSQL Company




-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

On Mon, Mar 16, 2015 at 6:22 AM, Petr Jelinek p...@2ndquadrant.com wrote:
 Do you think it is ready for committer?


 In my opinion, yes.

If it wasn't for the autoconf parts of this, I'd probably agree with
you. I need to go over that more carefully.


-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

On Wed, Mar 11, 2015 at 3:04 PM, Michael Paquier wrote:
 On Wed, Mar 11, 2015 at 2:47 PM, Noah Misch n...@leadboat.com wrote:
 On Sun, Mar 08, 2015 at 08:19:39PM +0900, Michael Paquier wrote:
 So I am planning to seriously focus soon on this stuff, basically
 using the TAP tests as base infrastructure for this regression test
 suite. First, does using the TAP tests sound fine?

 Yes.

 Check.

 On the top of my mind I got the following items that should be tested:
 - WAL replay: from archive, from stream
 - hot standby and read-only queries
 - node promotion
 - recovery targets and their interferences when multiple targets are
 specified (XID, name, timestamp, immediate)
 - timelines
 - recovery_target_action
 - recovery_min_apply_delay (check that WAL is fetch from a source at
 some correct interval, can use a special restore_command for that)
 - archive_cleanup_command (check that command is kicked at each restart 
 point)
 - recovery_end_command (check that command is kicked at the end of recovery)
 - timeline jump of a standby after reconnecting to a promoted node

So, as long as I had a clear picture of what I wanted to do regarding
this stuff (even if this is a busy commit fest, sorry), I have been
toying around with perl and I have finished with the patch attached,
adding some base structure for a new test suite covering recovery.

This patch includes basic tests for the following items:
- node promotion, test of archiving, streaming, replication cascading
- recovery targets XID, name, timestamp, immediate and PITR
- Timeline jump of a standby when reconnecting to a newly-promoted standby
- Replay delay
Tests are located in src/test/recovery, and are not part of the main
test suite, similarly to the ssl stuff.
I have dropped recovery_target_action for the time being as long as
the matter on the other thread is not set
(http://www.postgresql.org/message-id/20150315132707.gb19...@alap3.anarazel.de),
and I don't think that it would be complicated to create tests for
that btw.

The most important part of this patch is not the tests themselves, but
the base set of routines allowing to simply create nodes, take
backups, create standbys from backups, and set up nodes to do stuff
like streaming, archiving, or restoring from archives. There are many
configurations possible of course in recovery.conf, but the set of
routines that this patch present are made to be *simple* to not
overcomplicate the way tests can be written.

Feedback is of course welcome, but note that I am not seriously
expecting any until we get into 9.6 development cycle and I am adding
this patch to the next CF.

Regards,
-- 
Michael
diff --git a/src/test/Makefile b/src/test/Makefile
index b7cddc8..7174c2d 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -16,7 +16,7 @@ SUBDIRS = regress isolation modules
 
 # The SSL suite is not secure to run on a multi-user system, so don't run
 # it as part of global check target.
-ALWAYS_SUBDIRS = ssl
+ALWAYS_SUBDIRS = recovery ssl
 
 # We want to recurse to all subdirs for all standard targets, except that
 # installcheck and install should not recurse into the subdirectory modules.
diff --git a/src/test/perl/TestLib.pm b/src/test/perl/TestLib.pm
index 003cd9a..a035472 100644
--- a/src/test/perl/TestLib.pm
+++ b/src/test/perl/TestLib.pm
@@ -20,6 +20,7 @@ our @EXPORT = qw(
   program_version_ok
   program_options_handling_ok
   command_like
+  command_is
   issues_sql_like
 );
 
@@ -200,6 +201,16 @@ sub command_like
 	like($stdout, $expected_stdout, $test_name: matches);
 }
 
+sub command_is
+{
+	my ($cmd, $expected_stdout, $test_name) = @_;
+	my ($stdout, $stderr);
+	my $result = run $cmd, '', \$stdout, '2', \$stderr;
+	ok($result, @$cmd exit code 0);
+	is($stderr, '', @$cmd no stderr);
+	is($stdout, $expected_stdout, $test_name: matches);
+}
+
 sub issues_sql_like
 {
 	my ($cmd, $expected_sql, $test_name) = @_;
diff --git a/src/test/recovery/.gitignore b/src/test/recovery/.gitignore
new file mode 100644
index 000..499fa7d
--- /dev/null
+++ b/src/test/recovery/.gitignore
@@ -0,0 +1,3 @@
+# Generated by test suite
+/regress_log/
+/tmp_check/
diff --git a/src/test/recovery/Makefile b/src/test/recovery/Makefile
new file mode 100644
index 000..16c063a
--- /dev/null
+++ b/src/test/recovery/Makefile
@@ -0,0 +1,17 @@
+#-
+#
+# Makefile for src/test/recovery
+#
+# Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/recovery/Makefile
+#
+#-
+
+subdir = src/test/recovery
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+check:
+	$(prove_check)
diff --git a/src/test/recovery/README b/src/test/recovery/README
new file mode 100644
index 000..c194297
--- /dev/null
+++ b/src/test/recovery/README
@@ -0,0 +1,19 @@

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

2015-03-17 Thread Craig Ringer

On 13 March 2015 at 06:24, Ewan Higgs ewan_hi...@yahoo.co.uk wrote:

 Hi all,
 There has been some press regarding Pivotal's intent to release Greenplum
 source as part of an Open Development Platform (along with some of their
 Hadoop projects). Can anyone speak on whether any of Greenplum might find
 its way upstream? For example, if(!) the work is being released under an
 appropriate license, are people at Pivotal planning to push patches for the
 parallel architecture and associated query planner upstream?


Greenplum appears from what's visible on the outside to make heavy
modifications across a large part of the codebase, doing so with little
concern about removing support for existing features, breaking other use
cases, etc. So it does what it's meant to do well, but you can't
necessarily expect to drop it in place of PostgreSQL and have everything
just work.

My understanding is that they've written pretty much a new planner/executor
for plannable statements, retaining PostgreSQL's parser, protocol code,
utility statement handling, etc. But I'm finding it hard to find much hard
detail on the system's innards.

It's a valid approach, but it's one that means it's unlikely to be
practical to just cherry-pick a few features. There's sure to be a lot of
divergence between the codebases, and no doubt Greenplum will have
implemented infrastructure that overlaps with or duplicates things since
added in newer PostgreSQL releases - dynamic shmem, bgworkers, etc. Even if
it were feasible to pull in their features with the underlying
infrastructure it'd create a significant maintenance burden. So I expect
there'd need to be work done to move things over to use PostgreSQL features
where they exist.

Then there's the fact that Greenplum is based on a heavily modified
PostgreSQL 8.2. So even if desirable features were simple standalone
patches against 8.2 (which they won't be) there'd be a lot of work getting
them forward ported to 9.6.

I think it's more realistic that Greenplum's code would serve as an
interesting example of how something can be done, and maybe if the license
permits parts can be extracted and adapted where it takes less time than
rewriting. I wouldn't pin my hopes on seeing a big influx of Greenplum code.

I'd love to hear from someone at Pivotal, though, as the above is somewhere
between educated guesswork and complete hand-waving.


-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training  Services

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Etsuro Fujita


On 2015/03/14 7:18, Robert Haas wrote:

I think the foreign data wrapper join pushdown case, which also aims
to substitute a scan for a join, is interesting to think about, even
though it's likely to be handled by a new FDW method instead of via
the hook.  Where should the FDW method get called from?


I haven't had enough time to review the patch in details yet, so I don't 
know where we should call the method, but I'd vote for the idea of 
substituting a scan for a join, because I think that idea would probably 
allow update pushdown, which I'm proposing in the current CF, to scale 
up to handling a pushed-down update on a join.


Best regards,
Etsuro Fujita


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Redesigning checkpoint_segments

2015-03-17 Thread Jeff Janes

On Mon, Feb 23, 2015 at 8:56 AM, Heikki Linnakangas hlinnakan...@vmware.com
 wrote:


 Everyone seems to be happy with the names and behaviour of the GUCs, so
 committed.



The docs suggest that max_wal_size will be respected during archive
recovery (causing restartpoints and recycling), but I'm not seeing that
happening.  Is this a doc bug or an implementation bug?

Cheers,

Jeff

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Kouhei Kaigai

 -Original Message-
 From: Shigeru Hanada [mailto:shigeru.han...@gmail.com]
 Sent: Monday, March 16, 2015 9:59 PM
 To: Robert Haas
 Cc: Tom Lane; Thom Brown; Kaigai Kouhei(海外 浩平); pgsql-hackers@postgreSQL.org
 Subject: ##freemail## Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] 
 Custom
 Plan API)

 2015-03-14 7:18 GMT+09:00 Robert Haas robertmh...@gmail.com:
  I think the foreign data wrapper join pushdown case, which also aims
  to substitute a scan for a join, is interesting to think about, even
  though it's likely to be handled by a new FDW method instead of via
  the hook.  Where should the FDW method get called from?  Currently,
  the FDW method in KaiGai's patch is GetForeignJoinPaths, and that gets
  called from add_paths_to_joinrel().  The patch at

 http://www.postgresql.org/message-id/CAEZqfEfy7p=uRpwN-Q-NNgzb8kwHbfqF82YSb9
 ztfzg7zn6...@mail.gmail.com
  uses that to implement join pushdown in postgres_fdw; if you have A
  JOIN B JOIN C all on server X, we'll notice that the join with A and B
  can be turned into a foreign scan on A JOIN B, and similarly for A-C
  and B-C.  Then, if it turns out that the cheapest path for A-B is the
  foreign join, and the cheapest path for C is a foreign scan, we'll
  arrive at the idea of a foreign scan on A-B-C, and we'll realize the
  same thing in each of the other combinations as well.  So, eventually
  the foreign join gets pushed down.

 From the viewpoint of postgres_fdw, incremental approach seemed
 natural way, although postgres_fdw should consider paths in pathlist
 in additon to cheapest one as you mentioned in another thread.  This
 approarch allows FDW to use SQL statement generated for underlying
 scans as parts of FROM clause, as postgres_fdw does in the join
 push-down patch.

  But there's another possible approach: suppose that
  join_search_one_level, after considering left-sided and right-sided
  joins and after considering bushy joins, checks whether every relation
  it's got is from the same foreign server, and if so, asks that foreign
  server whether it would like to contribute any paths. Would that be
  better or worse?  A disadvantage is that if you've got something like
  A LEFT JOIN B LEFT JOIN C LEFT JOIN D LEFT JOIN E LEFT JOIN F LEFT
  JOIN G LEFT JOIN H LEFT JOIN I but none of the joins can be pushed
  down (say, each join clause calls a non-pushdown-safe function) you'll
  end up examining a pile of joinrels - at every level of the join tree
  - and individually rejecting each one.  With the
  build-it-up-incrementally approach, you'll figure that all out at
  level 2, and then after that there's nothing to do but give up
  quickly.  On the other hand, I'm afraid the incremental approach might
  miss a trick: consider small LEFT JOIN (big INNER JOIN huge ON big.x =
  huge.x) ON small.y = big.y AND small.z = huge.z, where all three are
  foreign tables on the same server.  If the output of the big/huge join
  is big, none of those paths are going to survive at level 2, but the
  overall join size might be very small, so we surely want a chance to
  recover at level 3.  (We discussed test cases of this form quite a bit
  in the context of e2fa76d80ba571d4de8992de6386536867250474.)

 Interesting, I overlooked that pattern.  As you pointed out, join
 between big foregin tables might be dominated, perhaps by a MergeJoin
 path.  Leaving dominated ForeignPath in pathlist for more optimization
 in the future (in higher join level) is an idea, but it would make
 planning time longer (and use more cycle and memory).

 Tom's idea sounds good for saving the path b), but I worry that
 whether FDW can get enough information at that timing, just before
 set_cheapest.  It would not be good I/F if each FDW needs to copy many
 code form joinrel.c...

I had a call to discuss this topic with Hanada-san. Even though he
expected FDW driver needs to check and extract relations involved
in a particular join, it also means we have less problem as long as
core backend can handle these common portion for all FDW/CSP drivers.
Thus, we need care about two hook locations. The first one is 
add_paths_to_joinrel() as current patch doing, for custom-scan that
adds an alternative join logic and takes underlying child nodes as
input. The other one is standard_join_search() as Tom pointed out,
for foreign-scan of remote join, or for custom-scan that replaces
an entire join subtree.

One positive aspect of this approach is, postgres_fdw can handle
whole-row-reference much simpler than bottom-up approach, according
to Hanada-san.

Remaining issue is, how to implement the core portion that extracts
relations in a particular join, and to identify join type to be
applied on a particular relations.
One rough idea is, we pull relids bitmap from the target joinrel,
then references the SpecialJoinInfo with identical union bitmap
of left/righthand. It allows to inform FDW driver which relations
and which another relations shall be joined in this level.

Re: [HACKERS] assessing parallel-safety

2015-03-17 Thread Noah Misch

On Mon, Mar 16, 2015 at 02:38:34PM -0400, Robert Haas wrote:
 On Sun, Mar 15, 2015 at 2:39 AM, Noah Misch n...@leadboat.com wrote:
  On Thu, Mar 12, 2015 at 11:21:37AM -0400, Robert Haas wrote:
  On Thu, Feb 19, 2015 at 1:19 AM, Noah Misch n...@leadboat.com wrote:
   Rereading my previous message, I failed to make the bottom line clear: I
   recommend marking eqsel etc. PROPARALLEL_UNSAFE but *not* checking an
   estimator's proparallel before calling it in the planner.
 
  But what do these functions do that is actually unsafe?
 
  They call the oprcode function of any operator naming them as an estimator.
  Users can add operators that use eqsel() as an estimator, and we have no 
  bound
  on what those operators' oprcode can do.  (In practice, parallel-unsafe
  operators using eqsel() as an estimator will be rare.)
 
 Is there a reason not to make a rule that opclass members must be
 parallel-safe?  I ask because I think it's important that the process
 of planning a query be categorically parallel-safe.  If we can't count
 on that, life gets a lot more difficult - what happens when we're in a
 parallel worker and call a SQL or PL/pgsql function?

Neither that rule, nor its variant downthread, would hurt operator authors too
much.  To make the planner categorically parallel-safe, though, means limiting
evaluate_function() to parallel-safe functions.  That would dramatically slow
selected queries.  It's enough for the PL scenario if planning a parallel-safe
query is itself parallel-safe.  If the planner is parallel-unsafe when
planning a parallel-unsafe query, what would suffer?

  RecordTransactionAbort() skips this for subtransaction aborts.  I would omit
  it here, because a parallel worker abort is, in this respect, more like a
  subtransaction abort than like a top-level transaction abort.
 
 No, I don't think so.  A subtransaction abort will be followed by
 either a toplevel commit or a toplevel abort, so any xlog written by
 the subtransaction will be flushed either synchronously or
 asynchronously at that time.  But for an aborting worker, that's not
 true: there's nothing to force the worker's xlog out to disk if it's
 ahead of the master's XactLastRecEnd.  If our XactLastRecEnd is behind
 the master's, then it doesn't matter what we do: an extra flush
 attempt is a no-op anyway.  If it's ahead, then we need it to be sure
 of getting the same behavior that we would have gotten in the
 non-parallel case.

Fair enough.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Future directions for inheritance-hierarchy statistics

2015-03-17 Thread Etsuro Fujita

On 2015/03/17 5:18, Tom Lane wrote:
 A few days ago I posted a very-much-WIP patch for making the planner
 dynamically combine statistics for each member of an appendrel:
 http://www.postgresql.org/message-id/22598.1425686...@sss.pgh.pa.us
 
 That patch was only intended to handle the case of an appendrel generated
 by a UNION ALL construct.  But it occurs to me that we could easily
 change it to also apply to appendrels generated from inheritance trees.
 Then we'd no longer need the whole-inheritance-tree statistics that
 ANALYZE currently produces, because we'd only ever look at per-table
 statistics in pg_statistic.
 
 This would have one significant drawback, which is that planning for
 large inheritance trees (many children) would probably get noticeably
 slower.  (But in the common case that constraint exclusion limits a
 query to scanning just one or a few children, the hit would be small.)
 
 On the other hand, there would be two very significant benefits.
 First, that we would automatically get statistics that account for
 partitions being eliminated by constraint exclusion, because only the
 non-eliminated partitions are present in the appendrel.  And second,
 that we'd be able to forget the whole problem of getting autovacuum
 to create whole-inheritance-tree stats.  Right now I'm doubtful that
 typical users are getting good up-to-date stats at all for queries of
 this sort, because autovacuum will only update those stats if it decides
 it needs to analyze the parent table.  Which is commonly empty, so that
 there's never a reason to fire an analyze on it.  (We'd left this as
 a problem to be solved later when we put in the whole-tree stats
 feature in 9.0, but no progress has been made on solving it.)
 
 So I think that going in this direction is clearly a win and we ought
 to pursue it.  It's not happening for 9.5 of course, because there's
 still a great deal of work to do before anything like this would be
 committable.  But I would like to establish a consensus that this
 would be a sensible thing to do in 9.6.
 
 The reason I bring it up now is that the inheritance-for-foreign-tables
 patch has some code that I don't much like for controlling what happens
 with those whole-tree stats when some of the children are foreign tables
 that lack ANALYZE support.  If the long-term plan is that whole-tree
 stats are going away altogether, then it won't be terribly important
 exactly what happens in that case, so we can just do some simple/easy
 kluge in the short term and not have to have an argument about what's
 the best thing to do.

That seems like a good idea.

Best regards,
Etsuro Fujita


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2015-03-17 Thread Julien Tachoires

On 15/03/2015 20:27, Julien Tachoires wrote:
 On 15/03/2015 04:34, Andreas Karlsson wrote:
 On 03/15/2015 04:25 AM, Andreas Karlsson wrote:
 Nice. You will also want to apply the attached patch which fixes support
 for the --no-tablespaces flag.

 Just realized that --no-tablespaces need to be fixed for pg_restore too.
 
 Indeed, after taking a look at pg_restore case, I would say it won't be
 so easy.
 Will try to fix it.

Here is a new version fixing this issue. I've added a new kind of TOC
entry for being able to handle pg_restore --no-tablespace case.

--
Julien


set_toast_tablespace_v0.15.patch.gz
Description: application/gzip

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

Robert Haas robertmh...@gmail.com writes:
 On Tue, Mar 17, 2015 at 1:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 We do have a process in which even committers have to think twice about
 whether it's appropriate to push something, but that's feature freeze
 during alpha/beta/RC testing, and we are still a long way away from that
 stage for 9.5.

 My understanding has been that for the last 5 years, the feature
 freeze deadline, for both committers and non-committers, is the
 beginning of the last CF, and earlier for major patches, most of
 which tend to come from committers.

Um, I consider that feature freeze marks a point at which we stop
*committing* new features, not a point after which new features can
no longer be submitted or considered.  Otherwise we'd be spending
four or more months a year in feature freeze, and that just isn't
appropriate IMV.

In any case, I reject the notion that the CF process has anything to
do with that decision.  The point of the CF submission deadline is
that we promise to consider every submission made before the deadline.
It is not to forbid committers from taking up items that arrived later,
if they feel motivated to.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in point releases 9.3.6 and 9.2.10?

On Thu, Mar 12, 2015 at 5:56 PM, Andres Freund and...@2ndquadrant.com wrote:
  Any chance that the new nodes also use a different kernel version or
  such?

 They may differ, but that doesn't seem likely to be relevant, at least
 to me.

 There've been some issues with seek(END) sometimes returning the wrong
 length in the past. And I've seen a report that might indicate a similar
 bug has been reintroduced. That could certainly cause such anerror.

It seems that I failed to take account of the fact that these cases
all involved Ubuntu 14.04 LTS (that wasn't clear to me before, but
I've verified that that's the case now), which is a small minority of
the kernels our instances use. I've verified that the issue arises
independently of point release, including on earlier point releases
from about a year ago, so I was hasty in suggesting that there was a
regression in the latest point releases.

I'm still going to get a back trace here, because it seems reasonable
to suppose that there is a Postgres bug even still - it may be that
whatever differences are in the 14.04 kernel are enough to make a
previously latent bug trip this code up. Plenty of people are using
Ubuntu 14.04 LTS with Postgres without seeing this issue, or any other
issue, and there is no other issue with the kernel that I know of. The
issue continues to occur as new instances with this configuration are
deployed.

I should also add that my temporary remediation (VACUUM FULL
pg_auth_members;) works consistently, as new cases start to crop up.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 1:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 What I was complaining about is new feature patches for 9.5 arriving
 after the start of the last CF.  There has to be some date after which
 a patch is too late to be considered for a given release, or we will
 never ship a release.  We can argue about what that date is, and it
 can be different for different people if we so choose, but at the end
 of the day, you have to cut it off someplace, or you never get the
 release out.

 Well, I'm going to push back on that concept a bit.  I do not think the
 CF process is, or ever has been, meant to tell committers they can't work
 on things at times they find convenient.  That would be holding back
 progress to little purpose.  What the CF process is about is making sure
 that things don't slip through the cracks, and in particular that
 submissions from non-committers get due consideration on a reasonably
 timely basis.

I agree with all of that.

 We do have a process in which even committers have to think twice about
 whether it's appropriate to push something, but that's feature freeze
 during alpha/beta/RC testing, and we are still a long way away from that
 stage for 9.5.

My understanding has been that for the last 5 years, the feature
freeze deadline, for both committers and non-committers, is the
beginning of the last CF, and earlier for major patches, most of
which tend to come from committers.  Now, I understand that we are
more free with making exceptions for committers, and there's some
justification for that, but, let's be honest, an awful lot of the
people submitting major patches at this point *are* committers, and
essentially all of those people take care to get their major patches
submitted before the last CF starts, and have for years, whether you
want to admit that or not.  I think we'd making a huge mistake if we
choose to back away from that, and I think you'd be arguing the other
side of this question in a heartbeat if somebody who happened to have
a commit bit submitted something you thought was half-baked.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Ashutosh Bapat

On Tue, Mar 17, 2015 at 10:28 AM, Shigeru Hanada shigeru.han...@gmail.com
wrote:

2015-03-14 7:18 GMT+09:00 Robert Haas robertmh...@gmail.com:
I think the foreign data wrapper join pushdown case, which also aims
to substitute a scan for a join, is interesting to think about, even
though it's likely to be handled by a new FDW method instead of via
the hook. Where should the FDW method get called from? Currently,
the FDW method in KaiGai's patch is GetForeignJoinPaths, and that gets
called from add_paths_to_joinrel(). The patch at

http://www.postgresql.org/message-id/CAEZqfEfy7p=urpwn-q-nngzb8kwhbfqf82ysb9ztfzg7zn6...@mail.gmail.com
uses that to implement join pushdown in postgres_fdw; if you have A
JOIN B JOIN C all on server X, we'll notice that the join with A and B
can be turned into a foreign scan on A JOIN B, and similarly for A-C
and B-C. Then, if it turns out that the cheapest path for A-B is the
foreign join, and the cheapest path for C is a foreign scan, we'll
arrive at the idea of a foreign scan on A-B-C, and we'll realize the
same thing in each of the other combinations as well. So, eventually
the foreign join gets pushed down.

From the viewpoint of postgres_fdw, incremental approach seemed
natural way, although postgres_fdw should consider paths in pathlist
in additon to cheapest one as you mentioned in another thread. This
approarch allows FDW to use SQL statement generated for underlying
scans as parts of FROM clause, as postgres_fdw does in the join
push-down patch.

But there's another possible approach: suppose that
join_search_one_level, after considering left-sided and right-sided
joins and after considering bushy joins, checks whether every relation
it's got is from the same foreign server, and if so, asks that foreign
server whether it would like to contribute any paths. Would that be
better or worse? A disadvantage is that if you've got something like
A LEFT JOIN B LEFT JOIN C LEFT JOIN D LEFT JOIN E LEFT JOIN F LEFT
JOIN G LEFT JOIN H LEFT JOIN I but none of the joins can be pushed
down (say, each join clause calls a non-pushdown-safe function) you'll
end up examining a pile of joinrels - at every level of the join tree
- and individually rejecting each one. With the
build-it-up-incrementally approach, you'll figure that all out at
level 2, and then after that there's nothing to do but give up
quickly. On the other hand, I'm afraid the incremental approach might
miss a trick: consider small LEFT JOIN (big INNER JOIN huge ON big.x =
huge.x) ON small.y = big.y AND small.z = huge.z, where all three are
foreign tables on the same server. If the output of the big/huge join
is big, none of those paths are going to survive at level 2, but the
overall join size might be very small, so we surely want a chance to
recover at level 3. (We discussed test cases of this form quite a bit
in the context of e2fa76d80ba571d4de8992de6386536867250474.)

Interesting, I overlooked that pattern. As you pointed out, join
between big foregin tables might be dominated, perhaps by a MergeJoin
path. Leaving dominated ForeignPath in pathlist for more optimization
in the future (in higher join level) is an idea, but it would make
planning time longer (and use more cycle and memory).

Tom's idea sounds good for saving the path b), but I worry that
whether FDW can get enough information at that timing, just before
set_cheapest. It would not be good I/F if each FDW needs to copy many
code form joinrel.c...

Even I have the same concern. A simple joinrel doesn't contain much
information about the individual two way joins involved in it, so FDW may
not be able to construct a query (or execution plan) and hence judge
whether a join is pushable or not, just by looking at the joinrel. There
will be a lot of code duplication to reconstruct that information, within
the FDW code.

--
Shigeru HANADA

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Ashutosh Bapat

On Sat, Mar 14, 2015 at 3:48 AM, Robert Haas robertmh...@gmail.com wrote:

On Fri, Mar 13, 2015 at 2:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Another bit of this that I think we could commit without fretting
about it too much is the code adding set_join_pathlist_hook. This is
- I think - analogous to set_rel_pathlist_hook, and like that hook,
could be used for other purposes than custom plan generation - e.g. to
delete paths we do not want to use. I've extracted this portion of
the patch and adjusted the comments; if there are no objections, I
will commit this bit also.

I don't object to the concept, but I think that is a pretty bad place
to put the hook call: add_paths_to_joinrel is typically called multiple
(perhaps *many*) times per joinrel and thus this placement would force
any user of the hook to do a lot of repetitive work.

Interesting point. I guess the question is whether a some or all
callers are going to actually *want* a separate call for each
invocation of add_paths_to_joinrel(), or whether they'll be happy to
operate on the otherwise-complete path list. It's true that if your
goal is to delete paths, it's probably best to be called just once
after the path list is complete, and there might be a use case for
that, but I guess it's less useful than for baserels. For a baserel,
as long as you don't nuke the sequential-scan path, there is always
going to be a way to complete the plan; so this would be a fine way to
implement a disable-an-index extension. But for joinrels, it's not so
easy to rule out, say, a hash-join here. Neither hook placement is
much good for that; the path you want to get rid of may have already
dominated paths you want to keep.

Suppose you want to add paths - e.g. you have an extension that goes
and looks for a materialized view that matches this subtree of the
query, and if it finds one, it substitutes a scan of the materialized
view for a scan of the baserel. Or, as in KaiGai's case, you have an
extension that can perform the whole join in GPU-land and produce the
same results we would have gotten via normal execution. Either way,
you want - and this is the central point of the whole patch here - to
inject a scan path into a joinrel. It is not altogether obvious to me
what the best placement for this is. In the materialized view case,
you probably need a perfect match between the baserels in the view and
the baserels in the joinrel to do anything. There's no point in
re-checking that for every innerrels/outerrels combination. I don't
know enough about the GPU case to reason about it intelligently; maybe
KaiGai can comment.

I think the foreign data wrapper join pushdown case, which also aims
to substitute a scan for a join, is interesting to think about, even
though it's likely to be handled by a new FDW method instead of via
the hook. Where should the FDW method get called from? Currently,
the FDW method in KaiGai's patch is GetForeignJoinPaths, and that gets
called from add_paths_to_joinrel(). The patch at

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

Robert Haas robertmh...@gmail.com writes:
 On Tue, Mar 17, 2015 at 10:11 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote:
 It might be an idea if foreign-scan path is not wiped out regardless of the
 estimated cost, we will be able to construct an entirely remote-join path
 even if intermediation path is expensive than local join.
 A problem is, how to distinct these special paths from usual paths that are
 eliminated on the previous stage once its path is more expensive.

 Any solution that is based on not eliminating paths that would
 otherwise be discarded based on cost seems to me to be unlikely to be
 feasible.  We can't complicate the core path-cost-comparison stuff for
 the convenience of FDW or custom-scan pushdown.

I concur.  I'm not even so worried about the cost of add_path as such;
the real problem with not discarding paths as aggressively as possible
is that it will result in a combinatorial explosion in the number of
path combinations that have to be examined at higher join levels.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Question about TEMP tables

2015-03-17 Thread Воронин Дмитрий

Hello, all.

We can create temp namespaces and temp objects that contains it. So, for 
example, temp table will be create at pg_temp_N (N - backendID). But afrer 
cluster init we have pg_temp_1 and pg_toast_temp_1 namespaces with OIDs 11333 
and 11334. Those namespaces are visible from any cluster database, but we 
cannot create any temp objects (please, correct me).

So, how can we use those namespaces and what are needed for?
 
Thank you.
 
-- 
Best regards, Dmitry Voronin


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Can pg_dump make use of CURRENT/SESSION_USER


Are there any use-cases for pg_dump to use CURRENT/SESSION_USER in its
output, so that restores will not be hard-coded to the dump user?  I
didn't see any cases of that, but wanted to ask.

pg_dump doesn't have to restore into old clusters so there isn't a
problem with backward compatibility.

---

- Forwarded message from Alvaro Herrera alvhe...@alvh.no-ip.org -

Date: Mon, 09 Mar 2015 18:46:02 +
From: Alvaro Herrera alvhe...@alvh.no-ip.org
To: pgsql-committ...@postgresql.org
Subject: [COMMITTERS] pgsql: Allow CURRENT/SESSION_USER to be used in certain 
commands

Allow CURRENT/SESSION_USER to be used in certain commands

Commands such as ALTER USER, ALTER GROUP, ALTER ROLE, GRANT, and the
various ALTER OBJECT / OWNER TO, as well as ad-hoc clauses related to
roles such as the AUTHORIZATION clause of CREATE SCHEMA, the FOR clause
of CREATE USER MAPPING, and the FOR ROLE clause of ALTER DEFAULT
PRIVILEGES can now take the keywords CURRENT_USER and SESSION_USER as
user specifiers in place of an explicit user name.

This commit also fixes some quite ugly handling of special standards-
mandated syntax in CREATE USER MAPPING, which in particular would fail
to work in presence of a role named current_user.

The special role specifiers PUBLIC and NONE also have more consistent
handling now.

Also take the opportunity to add location tracking to user specifiers.

Authors: Kyotaro Horiguchi.  Heavily reworked by Álvaro Herrera.
Reviewed by: Rushabh Lathia, Adam Brightwell, Marti Raudsepp.

Branch
--
master

Details
---
http://git.postgresql.org/pg/commitdiff/31eae6028eca4365e7165f5f33fee1ed0486aee0

Modified Files
--
doc/src/sgml/ref/alter_aggregate.sgml|3 +-
doc/src/sgml/ref/alter_collation.sgml|2 +-
doc/src/sgml/ref/alter_conversion.sgml   |2 +-
doc/src/sgml/ref/alter_database.sgml |2 +-
doc/src/sgml/ref/alter_domain.sgml   |2 +-
doc/src/sgml/ref/alter_event_trigger.sgml|2 +-
doc/src/sgml/ref/alter_foreign_data_wrapper.sgml |2 +-
doc/src/sgml/ref/alter_foreign_table.sgml|2 +-
doc/src/sgml/ref/alter_function.sgml |2 +-
doc/src/sgml/ref/alter_group.sgml|   10 +-
doc/src/sgml/ref/alter_language.sgml |2 +-
doc/src/sgml/ref/alter_large_object.sgml |2 +-
doc/src/sgml/ref/alter_materialized_view.sgml|2 +-
doc/src/sgml/ref/alter_opclass.sgml  |   11 +-
doc/src/sgml/ref/alter_operator.sgml |7 +-
doc/src/sgml/ref/alter_opfamily.sgml |   19 +-
doc/src/sgml/ref/alter_role.sgml |   35 +-
doc/src/sgml/ref/alter_schema.sgml   |2 +-
doc/src/sgml/ref/alter_sequence.sgml |2 +-
doc/src/sgml/ref/alter_server.sgml   |2 +-
doc/src/sgml/ref/alter_table.sgml|2 +-
doc/src/sgml/ref/alter_tablespace.sgml   |2 +-
doc/src/sgml/ref/alter_tsconfig.sgml |2 +-
doc/src/sgml/ref/alter_tsdictionary.sgml |2 +-
doc/src/sgml/ref/alter_type.sgml |2 +-
doc/src/sgml/ref/alter_user.sgml |   16 +-
doc/src/sgml/ref/alter_user_mapping.sgml |2 +-
doc/src/sgml/ref/alter_view.sgml |2 +-
doc/src/sgml/ref/create_schema.sgml  |   14 +-
doc/src/sgml/ref/grant.sgml  |   33 +-
src/backend/catalog/aclchk.c |   54 +-
src/backend/commands/alter.c |2 +-
src/backend/commands/extension.c |2 +-
src/backend/commands/foreigncmds.c   |   58 +-
src/backend/commands/policy.c|   18 +-
src/backend/commands/schemacmds.c|   21 +-
src/backend/commands/tablecmds.c |4 +-
src/backend/commands/tablespace.c|2 +-
src/backend/commands/user.c  |  116 +--
src/backend/nodes/copyfuncs.c|   49 +-
src/backend/nodes/equalfuncs.c   |   45 +-
src/backend/parser/gram.y|  228 --
src/backend/parser/parse_utilcmd.c   |4 +-
src/backend/utils/adt/acl.c  |  116 ++-
src/include/commands/user.h  |2 +-
src/include/nodes/nodes.h|1 +
src/include/nodes/parsenodes.h   |   50 +-
src/include/utils/acl.h  |8 +-
src/test/regress/expected/rolenames.out  |  940 ++
src/test/regress/parallel_schedule   |2 +-
src/test/regress/serial_schedule |1 +
src/test/regress/sql/rolenames.sql   |  434 ++
52 files changed, 2000 insertions(+), 347 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committ...@postgresql.org)
To make changes to your

Re: [HACKERS] Install shared libs in lib/ and bin/ with MSVC (Was: install libpq.dll in bin directory on Windows / Cygwin)

Michael Paquier wrote:

 So I have recoded the patch to use an hash of arrays (makes the code
 more readable IMO) to be able to track more easily what to install
 where, and process now does the following for shared libraries:
 - In lib/, install all .dll and .lib
 - In bin/, install all .dll

I wonder why do we need this part:

 @@ -247,22 +250,47 @@ sub CopySolutionOutput
  
   my $proj = read_file($pf.$vcproj)
 || croak Could not open $pf.$vcproj\n;
 +
 + # Check if this project uses a shared library by looking if
 + # SO_MAJOR_VERSION is defined in its Makefile, whose path
 + # can be found using the resource file of this project.
 + if (($vcproj eq 'vcxproj' 
 +  $proj =~ qr{ResourceCompile\s*Include=([^]+)}) ||
 + ($vcproj eq 'vcproj' 
 +  $proj =~ qr{File\s*RelativePath=([^\]+)\.rc}))
 + {
 + my $projpath = dirname($1);
 + my $mfname = -e $projpath/GNUmakefile ?
 + $projpath/GNUmakefile : $projpath/Makefile;
 + my $mf = read_file($mfname) ||
 + croak Could not open $mfname\n;
 +
 + if ($mf =~ /^SO_MAJOR_VERSION\s*=\s*(.*)$/mg)
 + {
 + $is_sharedlib = 1;
 + }
 + }

I mean, can't we just do the push unconditionally here?

   elsif ($1 == 2)
   {
 - $dir = lib;
 - $ext = dll;
 + push( @{ $install_list { 'lib' } }, dll);
 + if ($is_sharedlib)
 + {
 + push( @{ $install_list { 'bin' } }, 
 dll);
 + push( @{ $install_list { 'lib' } }, 
 lib);
 + }
   }

Surely if there are no lib/dll files in the subdirectory, nothing will
happen, right?  (I haven't actually tried.)

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

2015-03-17 Thread Stephen Frost

* Bruce Momjian (br...@momjian.us) wrote:
 I think the larger issue is that we have to adjust to a new-normal where
 Tom isn't going to be as helpful in this area.  Do we need more
 committers?  Do we need to adjust the process or dates?  These are
 probably the questions we should be addressing.

I'm afraid we are seeing issues beyond this- there don't seem (at least
to me) to be as many reviewers working on reviewing patches to give
feedback on them prior to having committers look either.  I suspect
that's largely because there hasn't been the 'push' for that to happen
of late.

Further, as it relates to individuals and the organizations that they
work for, I think we should be very clear that we are happy to accept
the work they are doing even if it isn't furthering the commitfest,
provided the code is at the level of quality we expect, the capability
is one we desire, and the design is sound.  Insisting that we punt
patches from Tom, or any other committer, which meet those criteria
because Tom (or whomever) didn't do enough work on the latest commitfest
is doing ourselves a disservice.  That these companies are funding Tom
and other committers to write open source code for all of us to benefit
from is great and should be encouraged.

That said, we certainly don't want to abandon the commitfests or, more
generally, patch submission from non-committers.  I haven't got any
great solutions to that problem, as I've been having trouble finding
time to support the commitfest too, but cutting off all progress would
surely be worse.

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

Robert Haas robertmh...@gmail.com writes:
 What I was complaining about is new feature patches for 9.5 arriving
 after the start of the last CF.  There has to be some date after which
 a patch is too late to be considered for a given release, or we will
 never ship a release.  We can argue about what that date is, and it
 can be different for different people if we so choose, but at the end
 of the day, you have to cut it off someplace, or you never get the
 release out.

Well, I'm going to push back on that concept a bit.  I do not think the
CF process is, or ever has been, meant to tell committers they can't work
on things at times they find convenient.  That would be holding back
progress to little purpose.  What the CF process is about is making sure
that things don't slip through the cracks, and in particular that
submissions from non-committers get due consideration on a reasonably
timely basis.

We do have a process in which even committers have to think twice about
whether it's appropriate to push something, but that's feature freeze
during alpha/beta/RC testing, and we are still a long way away from that
stage for 9.5.

Or in short: yes, the rules are different for committers and non
committers.  That's one of the reasons we are slow to hand out commit
bits.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] One question about security label command

2015-03-17 Thread Adam Brightwell


 The attached patch fixes the policy module of regression test.
 However, I also think we may stop to rely permission set of pre-defined
 selinux domains. Instead of pre-defined one, sepgsql-regtest.te may be
 ought to define own domain with appropriate permission set independent
 from the base selinux-policy version.


I have applied this patch and ran the tests.  All seems to work except that
I have a minor error in the 'label' regression tests.  It is simply a
result order issue, modifying the expected order in my environment resolves
the issue.  I have attached the 'regression.diffs' for reference as well,
FWIW, I have also attached a patch that corrects this issue for me,
hopefully it is useful.

-Adam

-- 
Adam Brightwell - adam.brightw...@crunchydatasolutions.com
Database Engineer - www.crunchydatasolutions.com
diff --git a/contrib/sepgsql/expected/label.out b/contrib/sepgsql/expected/label.out
new file mode 100644
index 9d1f904..d41eb48
*** a/contrib/sepgsql/expected/label.out
--- b/contrib/sepgsql/expected/label.out
*** SELECT objtype, objname, label FROM pg_s
*** 82,106 
  (3 rows)
  
  SELECT objtype, objname, label FROM pg_seclabels
! WHERE provider = 'selinux' AND objtype = 'column' AND (objname like 't3.%' OR objname like 't4.%');
   objtype |   objname   | label 
  -+-+---
-  column  | t3.t| unconfined_u:object_r:user_sepgsql_table_t:s0
-  column  | t3.s| unconfined_u:object_r:user_sepgsql_table_t:s0
-  column  | t3.ctid | unconfined_u:object_r:user_sepgsql_table_t:s0
-  column  | t3.xmin | unconfined_u:object_r:user_sepgsql_table_t:s0
-  column  | t3.cmin | unconfined_u:object_r:user_sepgsql_table_t:s0
-  column  | t3.xmax | unconfined_u:object_r:user_sepgsql_table_t:s0
   column  | t3.cmax | unconfined_u:object_r:user_sepgsql_table_t:s0
   column  | t3.tableoid | unconfined_u:object_r:user_sepgsql_table_t:s0
!  column  | t4.n| unconfined_u:object_r:sepgsql_table_t:s0
!  column  | t4.m| unconfined_u:object_r:sepgsql_table_t:s0
!  column  | t4.ctid | unconfined_u:object_r:sepgsql_sysobj_t:s0
!  column  | t4.xmin | unconfined_u:object_r:sepgsql_sysobj_t:s0
!  column  | t4.cmin | unconfined_u:object_r:sepgsql_sysobj_t:s0
!  column  | t4.xmax | unconfined_u:object_r:sepgsql_sysobj_t:s0
   column  | t4.cmax | unconfined_u:object_r:sepgsql_sysobj_t:s0
   column  | t4.tableoid | unconfined_u:object_r:sepgsql_sysobj_t:s0
  (16 rows)
  
  --
--- 82,107 
  (3 rows)
  
  SELECT objtype, objname, label FROM pg_seclabels
! WHERE provider = 'selinux' AND objtype = 'column' AND (objname like 't3.%' OR objname like 't4.%')
! ORDER BY objname ASC;
   objtype |   objname   | label 
  -+-+---
   column  | t3.cmax | unconfined_u:object_r:user_sepgsql_table_t:s0
+  column  | t3.cmin | unconfined_u:object_r:user_sepgsql_table_t:s0
+  column  | t3.ctid | unconfined_u:object_r:user_sepgsql_table_t:s0
+  column  | t3.s| unconfined_u:object_r:user_sepgsql_table_t:s0
+  column  | t3.t| unconfined_u:object_r:user_sepgsql_table_t:s0
   column  | t3.tableoid | unconfined_u:object_r:user_sepgsql_table_t:s0
!  column  | t3.xmax | unconfined_u:object_r:user_sepgsql_table_t:s0
!  column  | t3.xmin | unconfined_u:object_r:user_sepgsql_table_t:s0
   column  | t4.cmax | unconfined_u:object_r:sepgsql_sysobj_t:s0
+  column  | t4.cmin | unconfined_u:object_r:sepgsql_sysobj_t:s0
+  column  | t4.ctid | unconfined_u:object_r:sepgsql_sysobj_t:s0
+  column  | t4.m| unconfined_u:object_r:sepgsql_table_t:s0
+  column  | t4.n| unconfined_u:object_r:sepgsql_table_t:s0
   column  | t4.tableoid | unconfined_u:object_r:sepgsql_sysobj_t:s0
+  column  | t4.xmax | unconfined_u:object_r:sepgsql_sysobj_t:s0
+  column  | t4.xmin | unconfined_u:object_r:sepgsql_sysobj_t:s0
  (16 rows)
  
  --
diff --git a/contrib/sepgsql/sql/label.sql b/contrib/sepgsql/sql/label.sql
new file mode 100644
index 7a05c24..f4d50c3
*** a/contrib/sepgsql/sql/label.sql
--- b/contrib/sepgsql/sql/label.sql
*** INSERT INTO t4 VALUES (1,'mmm'), (2,'nnn
*** 78,84 
  SELECT objtype, objname, label FROM pg_seclabels
  WHERE provider = 'selinux' AND objtype = 'table' AND objname in ('t1', 't2', 't3');
  SELECT objtype, objname, label FROM pg_seclabels
! WHERE provider = 'selinux' AND objtype = 'column' AND (objname like 't3.%' OR objname like 't4.%');
  
  --
  -- Tests for SECURITY LABEL
--- 78,85 
  SELECT objtype, objname, label FROM pg_seclabels
  WHERE provider = 'selinux' AND objtype = 'table' AND objname in ('t1', 't2', 't3');
  SELECT objtype, objname, label FROM pg_seclabels
! WHERE provider = 'selinux' AND objtype = 'column' AND (objname like 't3.%' OR

Re: [HACKERS] Can pg_dump make use of CURRENT/SESSION_USER

Bruce Momjian wrote:
 
 Are there any use-cases for pg_dump to use CURRENT/SESSION_USER in its
 output, so that restores will not be hard-coded to the dump user?  I
 didn't see any cases of that, but wanted to ask.

Good question.  I don't know, probably not.

If we ever implement something like

COMMENT ON CURRENT_DATABASE IS ...

it will be useful, because you will be able to restore a dump into
another database and have the comment apply to the target database.
(Also, I wonder about
ALTER USER foo IN DATABASE current_database ...
because that will let us dump per-database user options too.)


-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in point releases 9.3.6 and 9.2.10?

2015-03-17 Thread Josh Berkus

On 03/17/2015 10:35 AM, Peter Geoghegan wrote:
 I'm still going to get a back trace here, because it seems reasonable
 to suppose that there is a Postgres bug even still - it may be that
 whatever differences are in the 14.04 kernel are enough to make a
 previously latent bug trip this code up. Plenty of people are using
 Ubuntu 14.04 LTS with Postgres without seeing this issue, or any other
 issue, and there is no other issue with the kernel that I know of. The
 issue continues to occur as new instances with this configuration are
 deployed.

Yeah ... I think I have logins on something like 50 servers running
9.3.6 on 14.04, in replication, and haven't seen this issue.

Which specific kernel version?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

2015-03-17 Thread Magnus Hagander

On Tue, Mar 17, 2015 at 6:31 PM, Stephen Frost sfr...@snowman.net wrote:

 * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
  I went to change this patch status in the commitfest app, and the app
  told me I cannot change the status in the current commitfest.  Please
  somebody with commitfest mace superpowers set it as ready for committer.

 I'm afraid the issue is a business decision which is incorrect as it's
 complaining that it's in a Closed state and you're trying to change it
 to an Open state.  While it's neat to think a patch could never be
 reopened, it's clearly not accurate.

 Adding Magnus to this, as I'm pretty sure he has some code to go hack
 (or perhaps just remove.. :).


I could've sworn I'd fixed that, but pretty obviously I hadn't. Sorry about
that.

Fixed now - a patch can now go from closed back to open in the last
commitfest where it was closed.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Re: [HACKERS] Add LINE: hint when schemaname.typename is a non-existent schema

In the interest of moving forward, I have updated this patch because
Ryan has been inactive for over a month now.

Tom Lane wrote:

 Yeah, setup_parser_errposition_callback would work too.  I'm not sure
 offhand which I like better.  One thing to keep in mind is that the
 callback approach results in adding an error cursor position to *any*
 error thrown while the callback is active, not only schema not found.
 There are pluses and minuses to that.  I've seen error cursors attached
 to very bizarre internal problems that (more or less by chance) showed up
 while the parser was looking up a table name, but weren't really connected
 to the table name at all.  OTOH, most of the time you'd just as soon not
 be too picky about what conditions you provide a cursor for.

I think we can live with cursor positions in some weird corner cases.
If we later find out that we don't like it for some reason, we can
reduce the scope that this applies to.

 The main place I'd question what you did is the callback placement around
 make_oper_cache_key --- that might work, but it seems very bizarre, and
 perhaps more than usually prone to the cursor given for unrelated
 problem issue.  Perhaps better to push it down inside that function
 so it surrounds just the namespace lookup call.

Agreed; the attached patch does it that way.  (I notice that we have the
pstate as first arg in many places; I put at the end for
make_oper_cache_key, together with location.  Is there some convention
to have it as first arg?)

 Also the diffs in parse_utilcmd.c are very confusing and seem to change
 more code than is necessary --- why did that happen?

The reason appears to be that Ryan wanted to have the pstate set, but
that was only created after looking other things up, so he moved a
largish block down; this was pretty bogus AFAICT.  The attached patch
fixes this by first creating the pstate, then doing the namespace
lookup, then doing the rest of the setup.

It's a bit disappointing to see so little changes in regression expected
output ...

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
index 53bbaec..1385776 100644
--- a/src/backend/parser/parse_func.c
+++ b/src/backend/parser/parse_func.c
@@ -93,6 +93,7 @@ ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs,
 	Oid			vatype;
 	FuncDetailCode fdresult;
 	char		aggkind = 0;
+	ParseCallbackState pcbstate;
 
 	/*
 	 * If there's an aggregate filter, transform it using transformWhereClause
@@ -235,12 +236,18 @@ ParseFuncOrColumn(ParseState *pstate, List *funcname, List *fargs,
 	 * type.  We'll fix up the variadic case below.  We may also have to deal
 	 * with default arguments.
 	 */
+
+	setup_parser_errposition_callback(pcbstate, pstate, location);
+
 	fdresult = func_get_detail(funcname, fargs, argnames, nargs,
 			   actual_arg_types,
 			   !func_variadic, true,
 			   funcid, rettype, retset,
 			   nvargs, vatype,
 			   declared_arg_types, argdefaults);
+
+	cancel_parser_errposition_callback(pcbstate);
+
 	if (fdresult == FUNCDETAIL_COERCION)
 	{
 		/*
diff --git a/src/backend/parser/parse_oper.c b/src/backend/parser/parse_oper.c
index 10de97b..f0786bd 100644
--- a/src/backend/parser/parse_oper.c
+++ b/src/backend/parser/parse_oper.c
@@ -76,7 +76,8 @@ static void op_error(ParseState *pstate, List *op, char oprkind,
 		 Oid arg1, Oid arg2,
 		 FuncDetailCode fdresult, int location);
 static bool make_oper_cache_key(OprCacheKey *key, List *opname,
-	Oid ltypeId, Oid rtypeId);
+	Oid ltypeId, Oid rtypeId,
+	ParseState *pstate, int location);
 static Oid	find_oper_cache_entry(OprCacheKey *key);
 static void make_oper_cache_entry(OprCacheKey *key, Oid opr_oid);
 static void InvalidateOprCacheCallBack(Datum arg, int cacheid, uint32 hashvalue);
@@ -383,7 +384,8 @@ oper(ParseState *pstate, List *opname, Oid ltypeId, Oid rtypeId,
 	/*
 	 * Try to find the mapping in the lookaside cache.
 	 */
-	key_ok = make_oper_cache_key(key, opname, ltypeId, rtypeId);
+	key_ok = make_oper_cache_key(key, opname, ltypeId, rtypeId, pstate, location);
+
 	if (key_ok)
 	{
 		operOid = find_oper_cache_entry(key);
@@ -529,7 +531,8 @@ right_oper(ParseState *pstate, List *op, Oid arg, bool noError, int location)
 	/*
 	 * Try to find the mapping in the lookaside cache.
 	 */
-	key_ok = make_oper_cache_key(key, op, arg, InvalidOid);
+	key_ok = make_oper_cache_key(key, op, arg, InvalidOid, pstate, location);
+
 	if (key_ok)
 	{
 		operOid = find_oper_cache_entry(key);
@@ -607,7 +610,8 @@ left_oper(ParseState *pstate, List *op, Oid arg, bool noError, int location)
 	/*
 	 * Try to find the mapping in the lookaside cache.
 	 */
-	key_ok = make_oper_cache_key(key, op, InvalidOid, arg);
+	key_ok = make_oper_cache_key(key, op, InvalidOid, arg, pstate, location);
+
 	if (key_ok)
 	{
 		operOid =

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 01:28:21PM -0400, Tom Lane wrote:
 Or in short: yes, the rules are different for committers and non
 committers.  That's one of the reasons we are slow to hand out commit
 bits.

I think one reason the rules are different for committers and
non-committers is that committers are responsible for quickly fixing
whatever breakage their patch application caused.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

2015-03-17 Thread Stephen Frost

* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote:
 I went to change this patch status in the commitfest app, and the app
 told me I cannot change the status in the current commitfest.  Please
 somebody with commitfest mace superpowers set it as ready for committer.

I'm afraid the issue is a business decision which is incorrect as it's
complaining that it's in a Closed state and you're trying to change it
to an Open state.  While it's neat to think a patch could never be
reopened, it's clearly not accurate.

Adding Magnus to this, as I'm pretty sure he has some code to go hack
(or perhaps just remove.. :).

Thanks!

Stephen


signature.asc
Description: Digital signature

Re: [HACKERS] proposal: searching in array function - array_position

My main question regarding this patch is whether the behavior with MD
arrays is useful at all.  Suppose I give it this:

alvherre=# select array_offset('{{{1,2},{3,4},{5,6}},{{2,3},{4,5},{6,7}}}', 3);
 array_offset 
--
3
(1 fila)

What can I do with the 3 value it returned?  Certainly not use it as
an offset to get a slice of the original array.  The only thing that
seems sensible to me here is to reject the whole thing with an error,
that is, only accept 1-D arrays here.  We can later extend the function
by allowing higher dimensionality as long as the second argument is an
array one dimension less than the first argument.  But if we allow the
case on its appearance, it's going to be difficult to change the
behavior later.

Has a case been made for the current behavior?

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services
diff --git a/doc/src/sgml/array.sgml b/doc/src/sgml/array.sgml
index 9ea1068..092013b 100644
--- a/doc/src/sgml/array.sgml
+++ b/doc/src/sgml/array.sgml
@@ -600,6 +600,25 @@ SELECT * FROM sal_emp WHERE pay_by_quarter  ARRAY[1];
   index, as described in xref linkend=indexes-types.
  /para
 
+ para
+  You can also search for specific values in an array using the functionarray_offset/
+  and functionarray_offsets/ functions. The former returns the position of
+  the first occurrence of a value in an array; the latter returns an array with the
+  positions of all occurrences of the value in the array.  For example:
+
+programlisting
+SELECT array_offset(ARRAY['sun','mon','tue','wed','thu','fri','sat'], 'mon');
+ array_offset
+--
+ 2
+
+SELECT array_offsets(ARRAY[1, 4, 3, 1, 3, 4, 2, 1], 1);
+ array_offsets 
+---
+ {1,4,8}
+/programlisting
+ /para
+
  tip
   para
Arrays are not sets; searching for specific array elements
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c198bea..d8c542e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -11480,6 +11480,12 @@ SELECT NULLIF(value, '(none)') ...
 primaryarray_lower/primary
   /indexterm
   indexterm
+primaryarray_offset/primary
+  /indexterm
+  indexterm
+primaryarray_offsets/primary
+  /indexterm
+  indexterm
 primaryarray_prepend/primary
   /indexterm
   indexterm
@@ -11599,6 +11605,39 @@ SELECT NULLIF(value, '(none)') ...
row
 entry
  literal
+  functionarray_offset/function(typeanyarray/type, typeanyelement/type optional, typeint/type/optional)
+ /literal
+/entry
+entrytypeint/type/entry
+entryreturns the offset of the first occurrence of the second
+argument in the array.  Each array element is compared using the 
+literalIS NOT DISTINCT FROM/ operator.  The optional third
+argument specifies an initial offset to begin the search at.
+Returns literalNULL/ if the value is not found. 
+Multi-dimensional arrays are squashed to one dimension before
+searching./entry
+entryliteralarray_offset(ARRAY['sun','mon','tue','wed','thu','fri','sat'], 'mon')/literal/entry
+entryliteral2/literal/entry
+   /row
+   row
+entry
+ literal
+  functionarray_offsets/function(typeanyarray/type, typeanyelement/type)
+ /literal
+/entry
+entrytypeint[]/type/entry
+entryreturns an array of offsets of all occurrences of the second
+argument in the array given as first argument.  Each array element is
+compared using the the literalIS NOT DISTINCT FROM/ operator.
+Returns an empty array when there are no occurences of the value in
+the array. Multi-dimensional arrays are squashed to one dimension before
+searching./entry
+entryliteralarray_offsets(ARRAY['A','A','B','A'], 'A')/literal/entry
+entryliteral{1,2,4}/literal/entry
+   /row
+   row
+entry
+ literal
   functionarray_prepend/function(typeanyelement/type, typeanyarray/type)
  /literal
 /entry
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 6679333..dcc7c62 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -12,9 +12,14 @@
  */
 #include postgres.h
 
+#include catalog/pg_type.h
 #include utils/array.h
 #include utils/builtins.h
 #include utils/lsyscache.h
+#include utils/typcache.h
+
+
+static Datum array_offset_common(FunctionCallInfo fcinfo);
 
 
 /*
@@ -652,3 +657,276 @@ array_agg_array_finalfn(PG_FUNCTION_ARGS)
 
 	PG_RETURN_DATUM(result);
 }
+
+/*-
+ * array_offset, array_offset_start :
+ *			return the offset of a value in an array.
+ *
+ * IS NOT DISTINCT FROM semantics are used for comparisons.  Return NULL when
+ * the value is not found.
+

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

Michael Paquier wrote:
 On Wed, Mar 18, 2015 at 2:22 AM, Alvaro Herrera wrote:

  1. ordered the argument list to vacuum(), hopefully it's more sensible
  now.
 
 Fine for me.

Actually, why don't we move va_cols to VacuumParams too?

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

On Wed, Mar 18, 2015 at 10:51 AM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
 Michael Paquier wrote:
 On Wed, Mar 18, 2015 at 2:22 AM, Alvaro Herrera wrote:

  1. ordered the argument list to vacuum(), hopefully it's more sensible
  now.

 Fine for me.

 Actually, why don't we move va_cols to VacuumParams too?

Because AnalyzeStmt assigns it in gram.y. Parameters directly from
VacuumStmt should not be added in Params, at least that's the spirit
of the patch as originally written.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reduce pinning in btree indexes

2015-03-17 Thread Jim Nasby


On 3/16/15 11:47 AM, Robert Haas wrote:

I am sure there are more sophisticated things to be done here, but I
guess my feeling is that time is a good way to go here for a first cut
- lots of people have suggested it, and there's clearly a use case for
it.  If the setting turns out to be popular, we can fine-tune it more
once we've got some experience with it.  But I'm nervous about trying
to to design something more complicated than that right now,
especially so late in the release cycle.  We know that this setting,
with time-based units, will meet the needs of the customer for whom it
was developed, and that is a good-enough reason to think that time is
a reasonable metric for this, even if we eventually have others.


+1. As you said, time is easy for people to understand, and I think it 
will handle a large chunk of the use cases.


I used to have this quote (or something close to it) on my whiteboard... 
I think it's appropriate here ;)


The perfect is the enemy of the good. -Simon Riggs
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] xloginsert.c hole_length warning on gcc 4.8.3

2015-03-17 Thread Fujii Masao

On Sat, Mar 14, 2015 at 8:39 AM, Tomas Vondra
tomas.von...@2ndquadrant.com wrote:
 Hi there,

 with gcc 4.8.3, I'm getting this warning in xloginsert.c:

Thanks for the report! I fixed this problem at the commit cd6c45c.

Regards,

-- 
Fujii Masao


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal: searching in array function - array_position

2015-03-17 Thread Jim Nasby


On 3/17/15 8:06 PM, Alvaro Herrera wrote:

My main question regarding this patch is whether the behavior with MD
arrays is useful at all.  Suppose I give it this:

alvherre=# select array_offset('{{{1,2},{3,4},{5,6}},{{2,3},{4,5},{6,7}}}', 3);
  array_offset
--
 3
(1 fila)

What can I do with the 3 value it returned?  Certainly not use it as
an offset to get a slice of the original array.  The only thing that
seems sensible to me here is to reject the whole thing with an error,
that is, only accept 1-D arrays here.  We can later extend the function
by allowing higher dimensionality as long as the second argument is an
array one dimension less than the first argument.  But if we allow the
case on its appearance, it's going to be difficult to change the
behavior later.


+1


Has a case been made for the current behavior?


Not that I remember. There was discussion about how this should properly 
support MD arrays.

--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 8:01 PM, Bruce Momjian br...@momjian.us wrote:
 On Tue, Mar 17, 2015 at 04:21:16PM -0700, Peter Geoghegan wrote:
 I, as a non-committer, have proposed that the rules be bent once or
 twice in the past, and those suggestions were rejected without
 exception, even though I imagined that there was a compelling
 cost/benefit ratio. I thought that was fine. I always assumed that I
 had the same right to suggest something as a committer. The only
 fundamental difference was that I had to convince a committer that my
 assessment was correct, rather than simply avoiding having the
 suggestion be vetoed. I'd need to do both. Clearly my previous
 understanding of this was questionable, to say the least.

 Basically, the same rules apply to all commitfests, i.e. a committer can
 apply anything during that period.  I think the only restriction for the
 last commitfest is that the committer can not apply a new patch that
 would have been too big to be submitted to the last commitfest. If
 enough people feel that this committer behavior during the last
 commitfest is a problem, we can discuss changing that policy.

One thing that's crystal clear here is that we don't all agree on what
the policy actually is.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 5:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Tue, Mar 17, 2015 at 1:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 We do have a process in which even committers have to think twice about
 whether it's appropriate to push something, but that's feature freeze
 during alpha/beta/RC testing, and we are still a long way away from that
 stage for 9.5.

 My understanding has been that for the last 5 years, the feature
 freeze deadline, for both committers and non-committers, is the
 beginning of the last CF, and earlier for major patches, most of
 which tend to come from committers.

 Um, I consider that feature freeze marks a point at which we stop
 *committing* new features, not a point after which new features can
 no longer be submitted or considered.  Otherwise we'd be spending
 four or more months a year in feature freeze, and that just isn't
 appropriate IMV.

Fine, call it a feature submission deadline.

 In any case, I reject the notion that the CF process has anything to
 do with that decision.  The point of the CF submission deadline is
 that we promise to consider every submission made before the deadline.
 It is not to forbid committers from taking up items that arrived later,
 if they feel motivated to.

So this is really what it boils down to: you evidently think there is
no problem with new feature patches showing up a month or two after
the last CommitFest has started.  I do.  It distracts everyone from
focusing on the patches that were submitted earlier, so that they
don't get dealt with.  If the patch comes from a committer, it's
actually worse, because patches from non-committers can be safely
ignored until a committer expresses interest in them, but if the patch
is from a committer who obviously intend to push it along, you'd
better drop what you're doing and object pretty fast, or it'll already
be in the tree before you get around to it.  I think it's perfectly
appropriate for the project to have a feature submission deadline, I
think having such a deadline is important to both the timeliness and
the quality of our releases, and I think most people here understand
that deadline to be the start of the last CF.  I'm just repeating
myself here, so I'm going to shut up now.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On Mon, Mar 16, 2015 at 8:10 AM, Dean Rasheed dean.a.rash...@gmail.com wrote:
 (Note there is some bitrot in gram.y that prevents the first patch
 from applying cleanly to HEAD)

That's trivially fixable. I'll have those fixes in the next revision,
once I firm some things up with Heikki.

 I tested using the attached script, and one test didn't behave as I
 expected. I believe the following should have been a valid upsert
 (following the update path) but actually it failed:

 INSERT INTO t1 VALUES (4, 0) ON CONFLICT (a) UPDATE SET b = 1;

 AFAICT, it is applying a WITH CHECK OPTION with qual b  0 AND a % 2
 = 0 to the about-to-be-updated tuple (a=4, b=0), which is wrong
 because the b  0 check (policy p3) should only be applied to the
 post-update tuple.

 Possibly I'm missing something though.

I think that you may have. Did you read the commit message/docs of the
RLS commit 0004-*? You must consider the second point here, I believe:


The 3 places that RLS policies are enforced are:

* Against row actually inserted, after insertion proceeds successfully
  (INSERT-applicable policies only).

* Against row in target table that caused conflict.  The implementation
  is careful not to leak the contents of that row in diagnostic
  messages (INSERT-applicable *and* UPDATE-applicable policies).

* Against the version of the row added by to the relation after
  ExecUpdate() is called (INSERT-applicable *and* UPDATE-applicable
  policies).



You're seeing a failure that applies to the target tuple of the UPDATE
(the tuple that we can't leak the contents of). I felt it was best to
check all policies against the target/existing tuple, including both
WITH CHECK OPTIONS and USING quals (which are both enforced).

I can see why you might not like that behavior, but it is the intended
behavior. I thought that this whole intersection of RLS + UPSERT is
complex enough that it would be best to be almost as conservative as
possible in what fails and what succeeds. The one exception is when
the insert path is actually taken, since the statement is an INSERT
statement.
-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On Tue, Mar 17, 2015 at 12:11 PM, Heikki Linnakangas hlinn...@iki.fi wrote:
 I'm still not sure the way the speculative locking works is the best
 approach. Instead of clearing xmin on super-deletion, a new flag on the heap
 tuple seems more straightforward. And we could put the speculative insertion
 token in t_ctid, instead of stashing it in the PGPROC array. That would
 again seem more straightforward.

I see the appeal of that approach. What concerns me about that
approach is that it makes it the problem of the inserter to confirm
its insertion, even though overwhelmingly, speculative insertions
succeeds (the race window is small due to the pre-check). The current
speculative token is legitimately a very short lived thing, a thing
that I hesitate to write to disk at all. So our optimistic approach to
inserting speculatively loses its optimism, which I don't like, if you
know what I mean.

OTOH, apart from avoiding stashing things in PGPROC, I do see another
advantage to what you outline. Doing things this way (and making the
insertion and confirmation of a speculative insertion a two-phased
thing) unambiguously fixes all problems with logical decoding, without
adding unexpected cross-change-coordination tricks during transaction
reassembly, and really without much further thought. All we do is get
a new case to decode, that ultimately produces a
REORDER_BUFFER_CHANGE_INSERT change, which Andres more or less wanted
to do anyway.

Under this scheme with using t_ctid, heap_insert() would do minimal
WAL logging, necessary for the same reasons that some WAL logging is
required within heap_lock_tuple() (so the new record type is very
similar to the existing, simple xl_heap_lock record type). We'd use
one of the two remaining bits within t_infomask2 for this, so that
waiters will know to interpret t_ctid as a speculative insertion token
(and then wait on it). Then, after speculative insertion succeeded,
we'd WAL log something closer to an UPDATE to confirm that insertion
was in fact successful, which is where the contents of the new tuple
is actually finally WAL-logged (like UPDATEs used to be, but without a
new tuple being WAL-logged at all, since there of course is none).

Is that more or less what you have in mind?

 A couple of quick random comments:

 /*
  * plan_speculative_use_index
  *  Use the planner to decide speculative insertion arbiter
 index
  *
  * Among indexes on target of INSERT ... ON CONFLICT, decide which index
 to
  * use to arbitrate taking alternative path.  This should be called
  * infrequently in practice, because it's unusual for more than one index
 to
  * be available that can satisfy a user-specified unique index inference
  * specification.
  *
  * Note: caller had better already hold some type of lock on the table.
  */
 Oid
 plan_speculative_use_index(PlannerInfo *root, List *indexList)
 {
 ...
 /* Locate cheapest IndexOptInfo for the target index */


 If I'm reading this correctly, if there are multiple indexes that match the
 unique index inference specification, we pick the cheapest one. Isn't that
 unstable? Two backends running the same INSERT ON CONFLICT statement might
 pick different indexes, and the decision might change over time as the table
 is analyzed. I think we should have a more robust rule. Could we easily just
 use all matching indexes?

Robert feel pretty strongly that there should be a costing aspect to
this. Initially I was skeptical of this, but just did what he said all
the same. But I've since come around to his view entirely because we
now support partial unique indexes.

You can add a predicate to a unique index inference specification, and
you're good, even if there is no partial index on those
attributes/expressions exactly matching the DML/inference predicate -
we use predicate_implied_by() for this, which works with an equivalent
non-partial unique index. This is because a non-partial unique index
covers those attributes sufficiently. There may be value in preferring
the cheap partial index, and then verifying that it actually did cover
the final inserted tuple version (which is how things work now). If we
cannot discriminate against one particular unique index, making sure
it covers the inserted heap tuple (by throwing a user-visible
ERRCODE_TRIGGERED_ACTION_EXCEPTION error if it doesn't, as I currently
do within ExecInsertIndexTuples()) is on shaky ground as a
user-visible behavior. I like that behavior, though - seems less
surprising than letting a failure to actually cover a selective
partial unique index predicate (that of the one and only arbiter
index) slide. You can only get this ERRCODE_TRIGGERED_ACTION_EXCEPTION
error when you actually had a predicate in your unique index inference
specification in the first place, so seems likely that you want the
error. But, I also like supporting unique indexes that are non-partial
even in the presence of a predicate in the unique index inference
specification clause,

Re: [HACKERS] Install shared libs in lib/ and bin/ with MSVC (Was: install libpq.dll in bin directory on Windows / Cygwin)

On Wed, Mar 18, 2015 at 3:13 AM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
 Michael Paquier wrote:

 So I have recoded the patch to use an hash of arrays (makes the code
 more readable IMO) to be able to track more easily what to install
 where, and process now does the following for shared libraries:
 - In lib/, install all .dll and .lib
 - In bin/, install all .dll

 I wonder why do we need this part:

 @@ -247,22 +250,47 @@ sub CopySolutionOutput

   my $proj = read_file($pf.$vcproj)
 || croak Could not open $pf.$vcproj\n;
 +
 + # Check if this project uses a shared library by looking if
 + # SO_MAJOR_VERSION is defined in its Makefile, whose path
 + # can be found using the resource file of this project.
 + if (($vcproj eq 'vcxproj' 
 +  $proj =~ qr{ResourceCompile\s*Include=([^]+)}) ||
 + ($vcproj eq 'vcproj' 
 +  $proj =~ qr{File\s*RelativePath=([^\]+)\.rc}))
 + {
 + my $projpath = dirname($1);
 + my $mfname = -e $projpath/GNUmakefile ?
 + $projpath/GNUmakefile : $projpath/Makefile;
 + my $mf = read_file($mfname) ||
 + croak Could not open $mfname\n;
 +
 + if ($mf =~ /^SO_MAJOR_VERSION\s*=\s*(.*)$/mg)
 + {
 + $is_sharedlib = 1;
 + }
 + }

 I mean, can't we just do the push unconditionally here?

Why should we install unnecessary stuff? This complicates the
installation contents, the point being to have only shared libraries's
dll installed in bin/, and make things consistent with what MinGW
does.

   elsif ($1 == 2)
   {
 - $dir = lib;
 - $ext = dll;
 + push( @{ $install_list { 'lib' } }, dll);
 + if ($is_sharedlib)
 + {
 + push( @{ $install_list { 'bin' } }, 
 dll);
 + push( @{ $install_list { 'lib' } }, 
 lib);
 + }
   }

 Surely if there are no lib/dll files in the subdirectory, nothing will
 happen, right?  (I haven't actually tried.)

No, it fails. And we should actually have a bin/lib that has been
correctly generated. Do you think this is a problem? Normally we
*should* fail IMO, meaning that the build process has broken what it
should have done.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

On Wed, Mar 18, 2015 at 2:22 AM, Alvaro Herrera wrote:
 Here's an updated patch.  I took your latest version and made some extra
 changes:

Thanks for taking the time to look at it!

 1. ordered the argument list to vacuum(), hopefully it's more sensible
 now.

Fine for me.

 2. changed struct autovac_table so that it uses options (the same
 VacuumOption bitmask to be passed to vacuum) and VacuumParams, instead
 of having each struct member separately.  That way, the parameters to
 vacuum() are constructed at once in autovac_recheck_table, and
 autovacuum_do_vac_analyze becomes much simpler.

 3. Added VACOPT_SKIPTOAST to VacuumOptions, currently only used by
 autovacuum.  We remove the do_toast argument.

Those are good ideas, and it simplifies a bit more code.

I had a look at your modified version, and it looks good to me.

 I think this is pretty sensible and my inclination is to commit as is,
 so that we can finally move on to more interesting things (such as the
 new reloption being proposed in a nearby thread).

Thanks. I'll do a rebase if this goes in first.
Regards,
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 04:21:16PM -0700, Peter Geoghegan wrote:
 I, as a non-committer, have proposed that the rules be bent once or
 twice in the past, and those suggestions were rejected without
 exception, even though I imagined that there was a compelling
 cost/benefit ratio. I thought that was fine. I always assumed that I
 had the same right to suggest something as a committer. The only
 fundamental difference was that I had to convince a committer that my
 assessment was correct, rather than simply avoiding having the
 suggestion be vetoed. I'd need to do both. Clearly my previous
 understanding of this was questionable, to say the least.

Basically, the same rules apply to all commitfests, i.e. a committer can
apply anything during that period.  I think the only restriction for the
last commitfest is that the committer can not apply a new patch that
would have been too big to be submitted to the last commitfest. If
enough people feel that this committer behavior during the last
commitfest is a problem, we can discuss changing that policy.

Now, with that right comes the significant responsibility to fix any
breakage they cause.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 3:50 PM, Robert Haas robertmh...@gmail.com wrote:
 In any case, I reject the notion that the CF process has anything to
 do with that decision.  The point of the CF submission deadline is
 that we promise to consider every submission made before the deadline.
 It is not to forbid committers from taking up items that arrived later,
 if they feel motivated to.

 So this is really what it boils down to: you evidently think there is
 no problem with new feature patches showing up a month or two after
 the last CommitFest has started.  I do.  It distracts everyone from
 focusing on the patches that were submitted earlier, so that they
 don't get dealt with.  If the patch comes from a committer, it's
 actually worse, because patches from non-committers can be safely
 ignored until a committer expresses interest in them, but if the patch
 is from a committer who obviously intend to push it along, you'd
 better drop what you're doing and object pretty fast, or it'll already
 be in the tree before you get around to it.  I think it's perfectly
 appropriate for the project to have a feature submission deadline, I
 think having such a deadline is important to both the timeliness and
 the quality of our releases, and I think most people here understand
 that deadline to be the start of the last CF.

I agree. New feature patches being committed that were originally
posted significantly after that deadline are not against the letter of
the law. However, that does not make them consistent with its spirit.
It's a value judgement as to whether or not any individual case
violates this spirit, but in my judgement this work does. I don't
think it's outrageous that it was suggested, but that's how I feel
about it. I don't see a very compelling cost/benefit ratio for the
community as a whole.

I, as a non-committer, have proposed that the rules be bent once or
twice in the past, and those suggestions were rejected without
exception, even though I imagined that there was a compelling
cost/benefit ratio. I thought that was fine. I always assumed that I
had the same right to suggest something as a committer. The only
fundamental difference was that I had to convince a committer that my
assessment was correct, rather than simply avoiding having the
suggestion be vetoed. I'd need to do both. Clearly my previous
understanding of this was questionable, to say the least.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Add LINE: hint when schemaname.typename is a non-existent schema

Alvaro Herrera alvhe...@2ndquadrant.com writes:
 Agreed; the attached patch does it that way.  (I notice that we have the
 pstate as first arg in many places; I put at the end for
 make_oper_cache_key, together with location.  Is there some convention
 to have it as first arg?)

Yes, parser-related functions always have pstate as first arg if they
need it at all.  Please follow that convention.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ranlib bleating about dirmod.o being empty

2015-03-17 Thread Peter Eisentraut

On 3/17/15 12:51 PM, Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
 It looks like ar isn't even the preferred method to build static
 libraries on OS X anymore.  Instead, one should use libtool (not GNU
 libtool), which has a -no_warning_for_no_symbols option.
 
 I looked into this a little bit, but that option seems to be a somewhat
 recent innovation; at least the libtool on my 10.6 (Snow Leopard)
 buildfarm critter doesn't recognize it.

Yeah, that's what I was unsure about.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

On Tue, Mar 17, 2015 at 02:09:34PM +0800, Craig Ringer wrote:
 It's a valid approach, but it's one that means it's unlikely to be practical 
 to
 just cherry-pick a few features. There's sure to be a lot of divergence 
 between
 the codebases, and no doubt Greenplum will have implemented infrastructure 
 that
 overlaps with or duplicates things since added in newer PostgreSQL releases -
 dynamic shmem, bgworkers, etc. Even if it were feasible to pull in their
 features with the underlying infrastructure it'd create a significant
 maintenance burden. So I expect there'd need to be work done to move things
 over to use PostgreSQL features where they exist.

I think we would need to create a team to learn the Greenplum code and
move over what is reasonable.  My guess is there is no desire in our
community to totally merge or maintain the Greenplum code --- of course,
that is just a guess.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2015: Introduction and Doubt regarding a Project.

2015-03-17 Thread Peter Eisentraut

On 3/17/15 10:26 AM, hitesh ramani wrote:
 1. As I did some research on this project, I found *date_trunc()
 supporting intervals‏ *was suggested last year but not selected as a
 GSoC project. Is it being floated this year too(as mentioned on the GSoC
 2015 wiki page of Postgres)? If yes, what are the exact expected outputs?

It seems to me that that would be too small for a GSoC project.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Tue, Mar 17, 2015 at 12:47 PM, Bruce Momjian br...@momjian.us wrote:
 Sorry to be coming late to this thread.  I don't think the problem is
 that Tom is working on these patches.  Rather, I think since Tom's
 employer now cares more about his current work, Tom just isn't as
 available to help with commitfest stuff and patch application.

 It is hard to see how the community can complain about that --- it is
 like having an uncle who used to give you $20 every time you saw him,
 then suddenly stops.  You would certainly be disappointed, but it is
 hard to see how you have a right to complain.  Now, if Tom's work was
 interrupting others' work, then there is a reasonable complaint.

 As a contrast, as someone who almost never applies things from the
 commitfest, I would be even more open to criticism.  I spend my spare
 time closing out unaddressed emails, but again, I have decided that is
 the best use of my time, and I didn't ask anyone if they agreed.  My
 assumption has always been that if activity is positive, it is up to the
 individual to decide which efforts to pursue.  My employer could adjust
 my activity too.

 I think the larger issue is that we have to adjust to a new-normal where
 Tom isn't going to be as helpful in this area.  Do we need more
 committers?  Do we need to adjust the process or dates?  These are
 probably the questions we should be addressing.

I was NOT complaining about Tom spending less time on PostgreSQL
CommitFests.  I would like him to spend more, but that's his decision,
in conjunction with his employer.  I would like to spend more time on
PostgreSQL CommitFests myself, but I have trouble finding the time,
too.

What I was complaining about is new feature patches for 9.5 arriving
after the start of the last CF.  There has to be some date after which
a patch is too late to be considered for a given release, or we will
never ship a release.  We can argue about what that date is, and it
can be different for different people if we so choose, but at the end
of the day, you have to cut it off someplace, or you never get the
release out.

I think these are really two different problems.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Strange assertion using VACOPT_FREEZE in vacuum.c

I went to change this patch status in the commitfest app, and the app
told me I cannot change the status in the current commitfest.  Please
somebody with commitfest mace superpowers set it as ready for committer.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-03-17 Thread Heikki Linnakangas


On 03/05/2015 03:18 AM, Peter Geoghegan wrote:

Attached patch series forms what I'm calling V3.0 of the INSERT ... ON
CONFLICT IGNORE/UPDATE feature. (Sorry about all the threads. I feel
this development justifies a new thread, though.)


I'm still not sure the way the speculative locking works is the best 
approach. Instead of clearing xmin on super-deletion, a new flag on the 
heap tuple seems more straightforward. And we could put the speculative 
insertion token in t_ctid, instead of stashing it in the PGPROC array. 
That would again seem more straightforward.


A couple of quick random comments:


/*
 * plan_speculative_use_index
 *  Use the planner to decide speculative insertion arbiter index
 *
 * Among indexes on target of INSERT ... ON CONFLICT, decide which index to
 * use to arbitrate taking alternative path.  This should be called
 * infrequently in practice, because it's unusual for more than one index to
 * be available that can satisfy a user-specified unique index inference
 * specification.
 *
 * Note: caller had better already hold some type of lock on the table.
 */
Oid
plan_speculative_use_index(PlannerInfo *root, List *indexList)
{
...
/* Locate cheapest IndexOptInfo for the target index */


If I'm reading this correctly, if there are multiple indexes that match 
the unique index inference specification, we pick the cheapest one. 
Isn't that unstable? Two backends running the same INSERT ON CONFLICT 
statement might pick different indexes, and the decision might change 
over time as the table is analyzed. I think we should have a more robust 
rule. Could we easily just use all matching indexes?




... Deferred unique constraints are not supported as
+   arbiters of whether an alternative literalON CONFLICT/ path
+   should be taken.


We really need to find a shorter term for arbiter of whether an 
alternative path should be taken. Different variations of that term are 
used a lot, and it's tedious to read.



* There is still an unresolved semantics issue with unique index
inference and non-default opclasses. I think it's sufficient that the
available/defined unique indexes dictate our idea of a unique
violation (that necessitates taking the alternative path). Even in a
world where there exists a non-default opclass with an equals
operator that does not match that of the default opclass (that is not
really the world we live in, because the only counter-example known is
made that way specifically to *discourage* its use by users), this
seems okay to me. It seems okay to me because surely the relevant
definition of equality is the one actually chosen for the available
unique index. If there exists an ambiguity for some strange reason
(i.e. there are two unique indexes, on the same attribute(s), but with
different equals operators), then its a costing issue, so the
behavior given is essentially non-predictable (it could end up being
either...but hey, those are the semantics). I have a very hard time
imagining how that could ever be the case, even when we have (say)
case insensitive opclasses for the text type. A case insensitive
opclass is stricter than a case sensitive opclass.  Why would a user
ever want both on the same attribute(s) of the same table? Is the user
really more or less expecting to never get a unique violation on the
non-arbitrating unique index, despite all this?

If reviewers are absolutely insistent that this theoretical ambiguity
is a problem, we can add an optional CREATE INDEX style opclass
specification (I'm already using the IndexElems representation from
CREATE INDEX for the inference specification, actually, so that would
be easy enough). I really have a hard time believing that the ugliness
is a problem for those hypothetical users that eventually consider
equals operator ambiguity among opclasses among available unique
indexes to be a problem. I haven't just gone and implemented this
already because I didn't want to document that an opclass
specification will be accepted. Since there is literally no reason why
anyone would care today, I see no reason to add what IMV would really
just be cruft.


I've been thinking that it would be nice to be able to specify a 
constraint name. Naming an index directly feels wrong, as in relational 
and SQL philosophy, indexes are just an implementation detail, but 
naming a constraint is a fair game. It would also be nice to be able to 
specify use the primary key.


Attached patch contains a few more things I saw at a quick read.

- Heikki

diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 46b8db8..d6fa98c 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -1014,6 +1014,10 @@ GetForeignServerByName(const char *name, bool missing_ok);
  source provides.
 /para
 
+!-- FIXME: If this is a hard limitation with the backend, the backend
+ should check and reject these cases. Otherwise, if it's possible
+ that a

Re: [HACKERS] Bug in point releases 9.3.6 and 9.2.10?

2015-03-17 Thread Greg Stark

Well, when a database is first initdb'd that relation is in fact 0 bytes:

::***# select pg_relation_filenode(oid) from pg_class where relname =
'pg_auth_members';
┌──┐
│ pg_relation_filenode │
├──┤
│12610 │
└──┘
(1 row)

$ find . -name 12610 -ls
1317720 -rw---   1 starkstark   0 Mar  3 18:52
./global/12610

So it's not surprising to find a 0-byte file there sometime. The question
is how the ALTER ROLE commands failed to extend it or why they're expecting
it to be non-zero when it is.

Is it possible the permissions on the file are wrong for some reason? The
symptoms would make sense if the directory had permissions that allowed
unlinking (for vacuum full) but the file didn't have r access for the
postgres user or SE rules blocked it from being read or something like that.



-- 
greg

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On Thu, Mar 5, 2015 at 3:44 PM, Peter Geoghegan p...@heroku.com wrote:
 Bruce Momjian kindly made available a server for stress-testing [1].
 I'm using jjanes_upsert for this task (I stopped doing this for a
 little while, and was in need of a new server).

BTW, this was run for about another week on Bruce's server, and no
further issues arose affecting either the B-Tree or exclusion
constraint implementations. I've stopped with it for now, because it
feels unlikely that I'm going to find any more bugs this way.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

On Tue, Mar 17, 2015 at 1:06 PM, Bruce Momjian br...@momjian.us wrote:
 I think we would need to create a team to learn the Greenplum code and
 move over what is reasonable.  My guess is there is no desire in our
 community to totally merge or maintain the Greenplum code --- of course,
 that is just a guess.

Let's wait until they actually release the code. I see no reason to
assume anything about it one way or the other just yet.


-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2015: Introduction and Doubt regarding a Project.

2015-03-17 Thread hitesh ramani

Hello Peter,

  1. As I did some research on this project, I found *date_trunc()
  supporting intervals‏ *was suggested last year but not selected as a
  GSoC project. Is it being floated this year too(as mentioned on the GSoC
  2015 wiki page of Postgres)? If yes, what are the exact expected outputs?
 
 It seems to me that that would be too small for a GSoC project.

I agree, but I saw the project mentioned on the GSoC wiki page of Postgres, and 
that's why I was curious to know the expected outputs, because if it was 
considered to be floated as a GSoC project, there must be something more than 
just what the title depicts.
--Hitesh

[HACKERS] GSoC 2015 Idea Discussion

2015-03-17 Thread Jackson Isaac

Hi,

I am Jackson Isaac pursuing 3rd Year B.Tech Computer Science and
Engineering at Amrita Vishwa Vidyapeetham, India.

I was looking at the proposal ideas at
https://wiki.postgresql.org/wiki/GSoC_2015 and was very much
interested in the idea 'PL/PgSQL x++; x+= 2; operators'.

I have been using postgres since 1 year and I had thought about the
same idea while working on functions and triggers, and hoped to do a
project on that (luckily it is proposed too as a GSoC project :) ). I
would like to know more about the idea and start working on the
project proposal too.

Can anyone please tell me who is mentoring this project who can guide
me on the right path. I am willing to read tutorials and other
materials needed to go on with this project.

Thank You,
Jackson Isaac
S6 B.Tech CSE
Amrita Vishwa Vidyapeetham
jacksonisaac.wordpress.com
Github/JacksonIsaac


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Resetting crash time of background worker

On Tue, Mar 17, 2015 at 1:33 AM, Amit Khandekar amitdkhan...@gmail.com wrote:
 When the postmaster recovers from a backend or worker crash, it resets bg
 worker's crash time (rw-rw_crashed_at) so that the bgworker will
 immediately restart (ResetBackgroundWorkerCrashTimes).

 But resetting rw-rw_crashed_at to 0 means that we have lost the information
 that the bgworker had actuallly crashed. So later when postmaster tries to
 find any workers that should start (maybe_start_bgworker), it treats this
 worker as a new worker, as against treating it as one that had crashed and
 is to be restarted. So for this bgworker, it does not consider
 BGW_NEVER_RESTART :

 if (rw-rw_crashed_at != 0) { if (rw-rw_worker.bgw_restart_time ==
 BGW_NEVER_RESTART) { ForgetBackgroundWorker(iter); continue; }  
 That means, it will not remove the worker, and it will be restarted. Now if
 the worker again crashes, postmaster would keep on repeating the crash and
 restart cycle for the whole system.

 From what I understand, BGW_NEVER_RESTART applies even to a crashed server.
 But let me know if I am missing anything.

 I think we either have to retain the knowledge that the worker has crashed
 using some new field, or else, we should reset the crash time only if it is
 not flagged BGW_NEVER_RESTART.

I think you're right, and I think we should do the second of those.
Thanks for tracking this down.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] assessing parallel-safety

On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch n...@leadboat.com wrote:
 Neither that rule, nor its variant downthread, would hurt operator authors too
 much.  To make the planner categorically parallel-safe, though, means limiting
 evaluate_function() to parallel-safe functions.  That would dramatically slow
 selected queries.  It's enough for the PL scenario if planning a parallel-safe
 query is itself parallel-safe.  If the planner is parallel-unsafe when
 planning a parallel-unsafe query, what would suffer?

Good point.  So I guess the rule can be that planning a parallel-safe
query should be parallel-safe.  From there, it follows that estimators
for a parallel-safe operator must also be parallel-safe.  Which seems
fine.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: pgbench - merging transaction logs

On Sun, Mar 15, 2015 at 3:35 PM, Fabien COELHO coe...@cri.ensmp.fr wrote:
 Firstly, the fact that pgbench produces one file per thread is awkward.

 I agree, but I think it is due to the multi process thread emulation: if you
 have real threads, you can do a simple fprintf, possibly with some mutex,
 and you're done. There is really nothing to do to implement this
 feature.

I think that's probably not a good idea, because fprintf() might then
become a bottleneck.  I fixed a similar problem with random() in
commit 4af43ee3f165c8e4b332a7e680a44f4b7ba2d3c1.  Even though the
locking was happening inside libc, it was still locking, and it still
caused a contention problem.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSoC 2015 Idea Discussion

Jackson Isaac jacksonisaac2...@gmail.com writes:
 I was looking at the proposal ideas at
 https://wiki.postgresql.org/wiki/GSoC_2015 and was very much
 interested in the idea 'PL/PgSQL x++; x+= 2; operators'.

Hm.  I don't think that idea has been discussed on-list at all (or maybe
it has but I missed it).  I'm not too excited about the ++ aspect of it
because that would require plpgsql to invent semantics out of whole
cloth, and it's hard to see how to do it in any respectably
datatype-independent way.  But you could imagine transforming
x += y into x := x + (y), for any arbitrary operator +, without
making any datatype-dependent assumptions.

I'm not entirely sure how to make that work syntactically though ---
there are lots of operator names that end with =, so you would have
difficulty deciding when it was OK to split that operator token apart.

Another problem is that commit bb1b8f694ad2efc35ebae2acfa2c18a2197b82a1
added an assumption that the second token of an assignment would be
exactly := (or its alias =).  I doubt it'd be all right to extend that
to say any operator ending in = is assumed not to be possible as the
second token of a non-assignment statement.  We don't currently have
any prefix operators whose names end in =, so the rule would work today,
but that seems like kind of a nasty restriction.

It would be easier to solve these problems with a variant syntax, like
x + := y
but that gives up a lot of the attractiveness of the idea :-(

I'm kind of interested in this if anyone can think of a way around the
syntactic difficulties, because it would tie nicely into the rough
ideas I had in
http://www.postgresql.org/message-id/20178.1423598...@sss.pgh.pa.us
about allowing extension datatypes to participate in optimized
update-in-place assignments.  We could legislate that you only get the
performance benefit of an in-place update if you invoke it with this
new syntax; that would make it a lot easier to identify when to pass
a R/W pointer to a function.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-03-17 Thread Kouhei Kaigai

 On Sat, Mar 14, 2015 at 3:48 AM, Robert Haas robertmh...@gmail.com wrote:
 
 
   On Fri, Mar 13, 2015 at 2:31 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Another bit of this that I think we could commit without fretting
about it too much is the code adding set_join_pathlist_hook.  This
 is
- I think - analogous to set_rel_pathlist_hook, and like that hook,
could be used for other purposes than custom plan generation - e.g.
 to
delete paths we do not want to use.  I've extracted this portion of
the patch and adjusted the comments; if there are no objections, I
will commit this bit also.
   
I don't object to the concept, but I think that is a pretty bad place
to put the hook call: add_paths_to_joinrel is typically called 
 multiple
(perhaps *many*) times per joinrel and thus this placement would force
any user of the hook to do a lot of repetitive work.
 
   Interesting point.  I guess the question is whether a some or all
   callers are going to actually *want* a separate call for each
   invocation of add_paths_to_joinrel(), or whether they'll be happy to
   operate on the otherwise-complete path list.  It's true that if your
   goal is to delete paths, it's probably best to be called just once
   after the path list is complete, and there might be a use case for
   that, but I guess it's less useful than for baserels.  For a baserel,
   as long as you don't nuke the sequential-scan path, there is always
   going to be a way to complete the plan; so this would be a fine way to
   implement a disable-an-index extension.  But for joinrels, it's not so
   easy to rule out, say, a hash-join here.  Neither hook placement is
   much good for that; the path you want to get rid of may have already
   dominated paths you want to keep.
 
   Suppose you want to add paths - e.g. you have an extension that goes
   and looks for a materialized view that matches this subtree of the
   query, and if it finds one, it substitutes a scan of the materialized
   view for a scan of the baserel.  Or, as in KaiGai's case, you have an
   extension that can perform the whole join in GPU-land and produce the
   same results we would have gotten via normal execution.  Either way,
   you want - and this is the central point of the whole patch here - to
   inject a scan path into a joinrel.  It is not altogether obvious to me
   what the best placement for this is.  In the materialized view case,
   you probably need a perfect match between the baserels in the view and
   the baserels in the joinrel to do anything.  There's no point in
   re-checking that for every innerrels/outerrels combination.  I don't
   know enough about the GPU case to reason about it intelligently; maybe
   KaiGai can comment.
 
   I think the foreign data wrapper join pushdown case, which also aims
   to substitute a scan for a join, is interesting to think about, even
   though it's likely to be handled by a new FDW method instead of via
   the hook.  Where should the FDW method get called from?  Currently,
   the FDW method in KaiGai's patch is GetForeignJoinPaths, and that gets
   called from add_paths_to_joinrel().  The patch at
   http://www.postgresql.org/message-id/CAEZqfEfy7p=uRpwN-Q-NNgzb8kwHbf
 qf82ysb9ztfzg7zn6...@mail.gmail.com
   uses that to implement join pushdown in postgres_fdw; if you have A
   JOIN B JOIN C all on server X, we'll notice that the join with A and B
   can be turned into a foreign scan on A JOIN B, and similarly for A-C
   and B-C.  Then, if it turns out that the cheapest path for A-B is the
   foreign join, and the cheapest path for C is a foreign scan, we'll
   arrive at the idea of a foreign scan on A-B-C, and we'll realize the
   same thing in each of the other combinations as well.  So, eventually
   the foreign join gets pushed down.
 
   But there's another possible approach: suppose that
   join_search_one_level, after considering left-sided and right-sided
   joins and after considering bushy joins, checks whether every relation
   it's got is from the same foreign server, and if so, asks that foreign
   server whether it would like to contribute any paths. Would that be
   better or worse?  A disadvantage is that if you've got something like
   A LEFT JOIN B LEFT JOIN C LEFT JOIN D LEFT JOIN E LEFT JOIN F LEFT
   JOIN G LEFT JOIN H LEFT JOIN I but none of the joins can be pushed
   down (say, each join clause calls a non-pushdown-safe function) you'll
   end up examining a pile of joinrels - at every level of the join tree
   - and individually rejecting each one.  With the
   build-it-up-incrementally approach, you'll figure that all out at
   level 2, and

Re: [HACKERS] Precedence of standard comparison operators

On Tue, Mar 10, 2015 at 04:10:01PM -0400, Peter Eisentraut wrote:
 On 3/10/15 1:12 PM, Tom Lane wrote:
  Robert Haas robertmh...@gmail.com writes:
  On Tue, Mar 10, 2015 at 12:45 PM, David G. Johnston
  david.g.johns...@gmail.com wrote:
  I would vote for Auto meaning On in the .0 release.
  
  I don't think users will appreciate an auto value whose meaning might
  change at some point, and I doubt we've have much luck identifying the
  correct point, either.  Users will upgrade over the course of years,
  not months, and will not necessarily complete their application
  retesting within any particular period of time thereafter.
  
  Yeah, I think that's too cute by far.  And people do not like things like
  this changing in point releases.  If we do this, I envision it as being
  on by default in 9.5 and then changing the default in 9.6 or 9.7 or so.
 
 Well, I point again to standards_conforming_strings: Leave the warning
 off for one release (or more), then default to on for one (or more),
 then change the behavior.
 
 We can change the timeline, but I don't think the approach was unsound.

Sorry to be replying late (but when has that ever stopped me :-) ), but
for standards_conforming_strings, there were security concerns about the
change, and we are forcing people to move to a new necessary syntax when
using backslash escapes, i.e. E''.  Once they moved to E'', the warnings
went away.

In this case, the way to avoid the warnings is to add _unnecessary_
parentheses, and sometimes in cases that don't need them, and never
will.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Seq Scan

On Tue, Mar 17, 2015 at 1:42 AM, Amit Kapila amit.kapil...@gmail.com wrote:
 The problem occurs in second loop inside DestroyParallelContext()
 where it calls WaitForBackgroundWorkerShutdown().  Basically
 WaitForBackgroundWorkerShutdown() just checks for BGWH_STOPPED
 status, refer below code in parallel-mode patch:

 + status = GetBackgroundWorkerPid(handle, pid);
 + if (status == BGWH_STOPPED)
 + return status;

 So if the status here returned is BGWH_NOT_YET_STARTED, then it
 will go for WaitLatch and will there forever.

 I think fix is to check if status is BGWH_STOPPED or  BGWH_NOT_YET_STARTED,
 then just return the status.

 What do you say?

No, that's not right.  If we return when the status is
BGWH_NOT_YET_STARTED, then the postmaster could subsequently start the
worker.

Can you try this:

diff --git a/src/backend/postmaster/bgworker.c
b/src/backend/postmaster/bgworker.c
index f80141a..39b919f 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -244,6 +244,8 @@ BackgroundWorkerStateChange(void)
rw-rw_terminate = true;
if (rw-rw_pid != 0)
kill(rw-rw_pid, SIGTERM);
+   else
+   ReportBackgroundWorkerPID(rw);
}
continue;
}

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] GSoC 2015: Introduction and Doubt regarding a Project.

2015-03-17 Thread hitesh ramani

Hello,
I introduced myself on the pgsql-students list but just to introduce here too, 
my name is Hitesh Ramani, I'm a student of Hyderabad Central University, 
Hyderabad, India. Currently I'm pursuing a project in PostgreSQL as my Post 
Graduation project hence I've hacked into the Postgres code a lot of times to 
add some functionalities to it, specifically to the sorting code. I'm also 
familiar with the query processing and backend internals.
I had a few doubts, which I need to ask and start on the proposal as soon as 
possible because the applications have opened.
1. As I did some research on this project, I found date_trunc() supporting 
intervals‏ was suggested last year but not selected as a GSoC project. Is it 
being floated this year too(as mentioned on the GSoC 2015 wiki page of 
Postgres)? If yes, what are the exact expected outputs?
2. Can I send a proposal for 2 projects to Postgres itself? Out of which one 
can be taken forward.
3. Does PG Strom take care of sorting as well on the GPUs?
Thanks and Regards,Hitesh Ramani

Re: [HACKERS] [PATCH] Add transforms feature

On Mon, Mar 16, 2015 at 9:51 PM, Peter Eisentraut pete...@gmx.net wrote:
 4. Why guc-use-transforms? Is there some possible negative side effect
 of transformations, so we have to disable it? If somebody don't would to
 use some transformations, then he should not to install some specific
 transformation.

 Well, there was extensive discussion last time around where people
 disagreed with that assertion.

I think we need to the ability to control it per-function, but having
a global disabling knob on top of that doesn't seem especially useful.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: pgbench - merging transaction logs

On Tue, Mar 17, 2015 at 11:27 AM, Fabien COELHO coe...@cri.ensmp.fr wrote:
 The fprintf we are talking about occurs at most once per pgbench
 transaction, possibly much less when aggregation is activated, and this
 transaction involves networks exchanges and possibly disk writes on the
 server.

random() was occurring four times per transaction rather than once,
but OTOH I think fprintf() is probably a much heavier-weight
operation.  The way to know if there's a real problem here is to test
it, but I'd be pretty surprised if there isn't.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: pgbench - merging transaction logs

2015-03-17 Thread Fabien COELHO



Hello,


I agree, but I think it is due to the multi process thread emulation: if you
have real threads, you can do a simple fprintf, possibly with some mutex,
and you're done. There is really nothing to do to implement this
feature.


I think that's probably not a good idea, because fprintf() might then
become a bottleneck.  I fixed a similar problem with random() in
commit 4af43ee3f165c8e4b332a7e680a44f4b7ba2d3c1.  Even though the
locking was happening inside libc, it was still locking, and it still
caused a contention problem.


The fprintf we are talking about occurs at most once per pgbench 
transaction, possibly much less when aggregation is activated, and this 
transaction involves networks exchanges and possibly disk writes on the 
server.


So I would have thought that the risk of contention because of such a lock 
would be very low in this case. If it really becomes a bottleneck, it 
means a lot of threads doing a lot of small transactions, and I would 
suggest that using the aggregated option would be the right answer to 
that.


--
Fabien.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

On Mon, Mar  9, 2015 at 03:06:24PM -0400, Robert Haas wrote:
 On Mon, Mar 9, 2015 at 2:33 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  As far as that goes, it has never been the case that we expected every
  patch to go through the commitfest review process.  (If we did, our
  response time to bugs would be probably a couple orders of magnitude
  longer than it is.)  The way I see the policy is that committers have
  authority to commit things that they feel are ready and that haven't
  been objected to by other developers.  Depending on the particular
  committer and the particular patch, feeling that a patch is ready
  might or might not require that it's been through a commitfest review.
  There is no process-based restriction on committing ready patches
  except when we are in alpha/beta/RC states, which is surely months
  away for 9.5.
 
  If I were looking for a formal review on this one, I would certainly
  not expect that it would get one during the current CF.  I wasn't
  though.
 
 Yes, I understand.  I don't really have anything more to say about
 this.  Nothing here changes my basic feeling that we need to stop
 putting new irons in the fire and start working hard on taking irons
 out of the fire; and I think most if not all other committers are
 already doing that.  Even to the extent that they are focusing on
 their own patches rather than other people's patches, I think it is
 mostly patches that they wrote some time ago, not new things that they
 have only just written.

Sorry to be coming late to this thread.  I don't think the problem is
that Tom is working on these patches.  Rather, I think since Tom's
employer now cares more about his current work, Tom just isn't as
available to help with commitfest stuff and patch application.  

It is hard to see how the community can complain about that --- it is
like having an uncle who used to give you $20 every time you saw him,
then suddenly stops.  You would certainly be disappointed, but it is
hard to see how you have a right to complain.  Now, if Tom's work was
interrupting others' work, then there is a reasonable complaint. 

As a contrast, as someone who almost never applies things from the
commitfest, I would be even more open to criticism.  I spend my spare
time closing out unaddressed emails, but again, I have decided that is
the best use of my time, and I didn't ask anyone if they agreed.  My
assumption has always been that if activity is positive, it is up to the
individual to decide which efforts to pursue.  My employer could adjust
my activity too.

I think the larger issue is that we have to adjust to a new-normal where
Tom isn't going to be as helpful in this area.  Do we need more
committers?  Do we need to adjust the process or dates?  These are
probably the questions we should be addressing.

I think one valid criticism is that Tom should transition his
commitments to this new-normal, especially for the the Grouping Set
patch, rather than allowing things to dangle in an unknown state. 
Again, communicating expectations goes a long way in avoiding conflict.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ranlib bleating about dirmod.o being empty

Peter Eisentraut pete...@gmx.net writes:
 It looks like ar isn't even the preferred method to build static
 libraries on OS X anymore.  Instead, one should use libtool (not GNU
 libtool), which has a -no_warning_for_no_symbols option.

I looked into this a little bit, but that option seems to be a somewhat
recent innovation; at least the libtool on my 10.6 (Snow Leopard)
buildfarm critter doesn't recognize it.  We might be able to adopt it
in a couple more years when nobody cares about such old OS X releases.
In the meantime, though, reclassifying dirmod.o solves the problem
so I don't feel a need to have an argument about dropping support for
old releases.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit

Bruce Momjian br...@momjian.us writes:
 I think one valid criticism is that Tom should transition his
 commitments to this new-normal, especially for the the Grouping Set
 patch, rather than allowing things to dangle in an unknown state. 

Well, as far as that goes, I had every intention of getting to the
GROUPING SETS patch before the end of the final commitfest, and I still
will if nobody else picks it up soon.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rethinking the parameter access hooks for plpgsql's benefit