[HACKERS] CREATE FUNCTION .. LEAKPROOF docs

2015-06-14 Thread Dean Rasheed
I think the docs for the LEAKPROOF option in create_function.sgml ought to mention RLS as well as security barrier views. Also the current text is no longer strictly correct in light of commit dcbf5948e12aa60b4d6ab65b6445897dfc971e01. Suggested rewording attached. Regards, Dean diff --git

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Dean Rasheed
On 11 June 2015 at 05:15, Bruce Momjian br...@momjian.us wrote: I have committed the first draft of the 9.5 release notes. You can view the output here: http://momjian.us/pgsql_docs/release-9-5.html I think it's worth mentioning dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably

Re: [HACKERS] s_lock() seems too aggressive for machines with many sockets

2015-06-14 Thread Jan Wieck
The whole thing turns out to be based on wrong baseline data, taken with a pgbench client running from a remote machine. It all started out from an investigation against 9.3. Curiously enough, the s_lock() problem that existed in 9.3 has a very similar effect on throughput as a network

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote: Note that I guess MauMau is a nickname. I think we are fine consistenly putting Japanese last names first _if_ we always capitalize the last name, e.g. FUJITA Etsuro --- is that a

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-14 Thread Michael Paquier
On Sat, Jun 13, 2015 at 6:25 PM, Michael Meskes mes...@postgresql.org wrote: On Sat, Jun 13, 2015 at 12:02:40AM -0400, Tom Lane wrote: But having said that, I would not be in a hurry to remove any existing if-guards for the case. For one thing, it makes the code look more similar to backend

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Bruce Momjian
On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote: Note that I guess MauMau is a nickname. I think we are fine consistenly putting Japanese last names first _if_ we always capitalize the last name, e.g. FUJITA Etsuro --- is that a good idea? Does everyone like that? Does any

Re: [HACKERS] git push hook to check for outdated timestamps

2015-06-14 Thread Magnus Hagander
On Fri, Jun 12, 2015 at 4:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 06/12/2015 09:31 AM, Robert Haas wrote: Could we update our git hook to refuse a push of a new commit whose timestamp is more than, say, 24 hours in the past? Our commit

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Petr Jelinek
On 2015-06-14 03:02, Bruce Momjian wrote: On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: Petr Jelinek wrote: Hi, + listitem + para +Add typeJSONB/ functions functionjsonb_set()/ and +functionjsonb_pretty/ (Dmitry Dolgov, Andrew Dunstan) +

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-14 Thread Shay Rojansky
Tom, I agree this is entirely a client-side issue. Regardless, as Simon says it would be useful to have some documentation for client implementors. Sehrope, thanks for the JDBC link! I was actually thinking of going about it another way in Npgsql: 1. Send messages normally until the first

[HACKERS] Tab completion for TABLESAMPLE

2015-06-14 Thread Petr Jelinek
Hi, looks like I omitted psql tab completion from the TABLESAMPLE patch. The attached patch adds it. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c

Re: [HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

2015-06-14 Thread Thom Brown
On 14 June 2015 at 04:25, Michael Paquier michael.paqu...@gmail.com wrote: On Sun, Jun 14, 2015 at 11:38 AM, Thom Brown t...@linux.com wrote: As you can see, 3 indexes are missing, which happen to be ones that would duplicate the column definition of another index. Is this intentional? If

Re: [HACKERS] On columnar storage

2015-06-14 Thread Andres Freund
On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: Rewriter Parsing occurs as currently. During query rewrite, specifically at the bottom of the per-relation loop in fireRIRrules(), we will modify the query tree: each relation RTE containing a colstore will be replaced with a JoinExpr

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: Parsing occurs as currently. During query rewrite, specifically at the bottom of the per-relation loop in fireRIRrules(), we will modify the query tree: each relation RTE containing a colstore will be

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Won't this cause issues to MergeAppend optimizations? Like what? Well, as I understand, MergeAppend needs to know the sort order of the child node, right? But that's available

Re: [HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

2015-06-14 Thread Tom Lane
Thom Brown t...@linux.com writes: The commit refers to duplicate index names, and only for UNIQUE indexes. This behaviour is beyond that. And how does it determine which index to copy? In my example, I placed an index in a different tablespace. That could be on a drive with very different

[HACKERS] pg_resetsysid

2015-06-14 Thread Petr Jelinek
Hi, This is an attempt to revive the resetsysid tool submission that didn't go anywhere last year [1]. In that thread there were mainly two complains, one that pg_resetxlog would preferably be tool for more advanced stuff and similar utilities that are not as dangerous should have separate

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tomas Vondra
Hi, On 06/13/15 00:07, Michael Nolan wrote: On Thu, Jun 11, 2015 at 7:03 PM, Alvaro Herrera alvhe...@2ndquadrant.com mailto:alvhe...@2ndquadrant.com wrote: We hope to have a chance to discuss this during the upcoming developer unconference in Ottawa. Here are some preliminary ideas

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Robert Haas wrote: On Thu, Jun 11, 2015 at 7:03 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I've been trying to figure out a plan to enable native column stores (CS or colstore) for Postgres. Motivations: * avoid the 32 TB limit for tables * avoid the 1600 column limit for

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: OK, new idea. What about, instead of having the last name be all-caps, we have the last name start with an uppercase letter, then use smallcaps for the rest of the last name: https://en.wikipedia.org/wiki/Small_caps That way, the last name will

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tomas Vondra
On 06/14/15 17:22, Alvaro Herrera wrote: Robert Haas wrote: ,,, Second, there's the idea of a way of storing tuples that is different from PostgreSQL's usual mechanism - i.e. a storage manager API. I understand your concerns about going through the FDW API so maybe that's not the right way to

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Andres Freund wrote: On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: Rewriter Parsing occurs as currently. During query rewrite, specifically at the bottom of the per-relation loop in fireRIRrules(), we will modify the query tree: each relation RTE containing a colstore will be

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Bruce Momjian
On Sun, Jun 14, 2015 at 01:57:08PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: Also, it looks like we could spell your last name with an accent, assuming the i-acute character in Latin1 is fine. Ah, you are

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: I wrote: Another model that could be followed is expansion of inheritance-tree references, which happens early in the planner. Actually ... if you intend to allow column storage to work with inherited tables (and if you don't, you'd better have a darn good reason why not),

Re: [HACKERS] Git humor

2015-06-14 Thread Andrew Dunstan
On 06/14/2015 02:07 PM, Bruce Momjian wrote: If you ever had trouble understanding a git manual page, this humorous random git man page generator is full of fun: http://git-man-page-generator.lokaltog.net/ Try a few generate runs to find one that seems logical. :-) This looked

[HACKERS] The real reason why TAP testing isn't ready for prime time

2015-06-14 Thread Tom Lane
Buildfarm member hamster has failed a pretty significant fraction of its recent runs in the BinInstallCheck step: http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamsterbr=HEAD Since other critters aren't equally distressed, it seems likely that this is just an out-of-disk-space type

Re: [HACKERS] Need Multixact Freezing Docs

2015-06-14 Thread Alvaro Herrera
Josh Berkus wrote: You can see the current multixact value in pg_controldata output. Keep timestamped values of that somewhere (a table?) so that you can measure consumption rate. I don't think we provide SQL-level access to those values. Bleh. Do we provide SQL-level access in

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-14 Thread Shay Rojansky
On Sun, Jun 14, 2015 at 6:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Shay Rojansky r...@roji.org writes: [ rely on non-blocking sockets to avoid deadlock ] Yeah, that's pretty much the approach libpq has taken: write (or read) when you can, but press on when you can't. Good to hear. The

[HACKERS] Git humor

2015-06-14 Thread Bruce Momjian
If you ever had trouble understanding a git manual page, this humorous random git man page generator is full of fun: http://git-man-page-generator.lokaltog.net/ Try a few generate runs to find one that seems logical. :-) -- Bruce Momjian br...@momjian.ushttp://momjian.us

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Actually ... if you intend to allow column storage to work with inherited tables (and if you don't, you'd better have a darn good reason why not), I think you probably want to do this join insertion *after*

[HACKERS] 9.5 feature count

2015-06-14 Thread Bruce Momjian
I have run a script to count the number of listitem items in the major release notes of each major version of Postgres back to 7.4: 7.4280 8.0238 8.1187 8.2230 8.3237 8.4330 9.0252 9.1213 9.2

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Alvaro Herrera
Bruce Momjian wrote: On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: Also, it looks like we could spell your last name with an accent, assuming the i-acute character in Latin1 is fine. Ah, you are correct. I found a few commits that did have that accent. All _seven_ of

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
I wrote: Another model that could be followed is expansion of inheritance-tree references, which happens early in the planner. Actually ... if you intend to allow column storage to work with inherited tables (and if you don't, you'd better have a darn good reason why not), I think you probably

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Another choice I considered was subquery_planner: in the spot where expand_inherited_tables() is called, add a new call to expand the RTEs that correspond to tables containing stores. But I had second thoughts because that function says it's

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). ... My general answer to the performance concerns is that they aren't a

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Peter Geoghegan
On Sat, Jun 13, 2015 at 6:24 PM, Bruce Momjian br...@momjian.us wrote: I think we should really address this. Attached patch adds a new release note item for it. It also adds to the documentation that explains why users should prefer varchar(n)/text to character(n); the lack of abbreviated key

Re: [HACKERS] Git humor

2015-06-14 Thread Jan de Visser
On June 14, 2015 02:07:16 PM Bruce Momjian wrote: If you ever had trouble understanding a git manual page, this humorous random git man page generator is full of fun: http://git-man-page-generator.lokaltog.net/ Try a few generate runs to find one that seems logical. :-) To my

[HACKERS] Memory Accounting v11

2015-06-14 Thread Jeff Davis
This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). It also adds a new function, MemoryContextMemAllocated() which can either retrieve the total allocated for the

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Amit Kapila wrote: Will the column store obey snapshot model similar to current heap tuples, if so will it derive the transaction information from heap tuple? Yes, visibility will be tied to the heap tuple -- a value is

Re: [HACKERS] On columnar storage

2015-06-14 Thread Petr Jelinek
On 2015-06-14 17:36, Tomas Vondra wrote: On 06/14/15 17:22, Alvaro Herrera wrote: Robert Haas wrote: ,,, Second, there's the idea of a way of storing tuples that is different from PostgreSQL's usual mechanism - i.e. a storage manager API. I understand your concerns about going through the

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Actually ... if you intend to allow column storage to work with inherited tables (and if you don't, you'd better have a darn good reason why not), I think you probably want to do this join insertion *after* inheritance expansion,

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Won't this cause issues to MergeAppend optimizations? Like what? Well, as I understand, MergeAppend needs to know the sort order of the child node,

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tomas Vondra
Hi, On 06/14/15 22:21, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). ... My general answer to the

[HACKERS] creating extension including dependencies

2015-06-14 Thread Petr Jelinek
Hi, I am getting tired installing manually required extensions manually. I was wondering if we might want to add option to CREATE SEQUENCE that would allow automatic creation of the extensions required by the extension that is being installed by the user. I also wrote some prototype patch

[HACKERS] Sharing aggregate states between different aggregate functions

2015-06-14 Thread David Rowley
Simon and I have been going over some ideas about how to make improvements to aggregate performance by cutting down on the duplicate work that's done when 2 aggregate functions are used where one knows how to satisfy all the requirements of the other. To cut a long story short, all our ideas will

Re: [HACKERS] Patch to improve a few appendStringInfo* calls

2015-06-14 Thread David Rowley
On 29 May 2015 at 12:51, Peter Eisentraut pete...@gmx.net wrote: On 5/12/15 4:33 AM, David Rowley wrote: Shortly after I sent the previous patch I did a few more searches and also found some more things that are not quite right. Most of these are to use the binary append method when the

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tomas Vondra
Hi, On 06/14/15 21:43, Jeff Davis wrote: This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). I see it's adding the new field as int64 - wouldn't a Size be more

Re: [HACKERS] The real reason why TAP testing isn't ready for prime time

2015-06-14 Thread Michael Paquier
On Mon, Jun 15, 2015 at 3:37 AM, Tom Lane t...@sss.pgh.pa.us wrote: Buildfarm member hamster has failed a pretty significant fraction of its recent runs in the BinInstallCheck step: http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamsterbr=HEAD Since other critters aren't equally

Re: [HACKERS] On columnar storage

2015-06-14 Thread Michael Nolan
On Sun, Jun 14, 2015 at 10:30 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Are you looking to avoid all hardware-based limits, or would using a 64 bit row pointer be possible? That would give you 2^64 or 1.8 E19 unique rows over whatever granularity/uniqueness you use (per table,

[HACKERS] query execution time faster with geqo on than off: bug?

2015-06-14 Thread David Kamholz
I've encountered a query with 11 joins whose execution time (i.e., the time not taken up by planning) is significantly faster with geqo on rather than off. This is surprising to me and seems like it might be a bug in the planner, so I am posting it here rather than to -performance. The query is

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

2015-06-14 Thread Craig Ringer
Has anything actually happened on the Greenplum release at this point? All I can find are the same old press releases. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Jeff Davis
On Sun, 2015-06-14 at 16:21 -0400, Tom Lane wrote: Meh. HashAgg could track its memory usage without loading the entire system with a penalty. When I tried doing it outside of the MemoryContexts, it seemed to get awkward quickly. I'm open to suggestion if you can point me in the right

Re: [HACKERS] creating extension including dependencies

2015-06-14 Thread Pavel Stehule
+1 Is it working in runtime too? Dne 15.6.2015 0:51 napsal uživatel Petr Jelinek p...@2ndquadrant.com: Hi, I am getting tired installing manually required extensions manually. I was wondering if we might want to add option to CREATE SEQUENCE that would allow automatic creation of the

[HACKERS] Time to get rid of PQnoPasswordSupplied?

2015-06-14 Thread Craig Ringer
Hi all I frequently see users confused by one of our more common and less clear error messages: fe_sendauth: no password supplied What this really means is that the server requested a password for md5 or cleartext authentication but no password was supplied to the client and it cannot

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

2015-06-14 Thread Josh Berkus
On 06/14/2015 11:44 PM, Craig Ringer wrote: Has anything actually happened on the Greenplum release at this point? All I can find are the same old press releases. Nothing has happened. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] The Future of Aggregation

2015-06-14 Thread David Rowley
On 12 June 2015 at 23:57, David Rowley david.row...@2ndquadrant.com wrote: On 11 June 2015 at 01:39, Kevin Grittner kgri...@ymail.com wrote: One question that arose in my mind running this was whether might be able to combine sum(x) with count(*) if x was NOT NULL, even though the arguments