Re: [HACKERS] "could not adopt C locale" failure at startup on Windows

2015-06-14 Thread Noah Misch
On Wed, Jun 10, 2015 at 10:09:38AM -0400, Tom Lane wrote: > Noah Misch writes: > > I can reproduce this with "initdb --locale=C", > > postgresql-9.4.3-1-windows-binaries.zip (32-bit), Windows 7 x64, and the > > Windows ANSI code page set to CP936. (Choose "Chinese (Simplified, PRC)" in > > Contro

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

2015-06-14 Thread Josh Berkus
On 06/14/2015 11:44 PM, Craig Ringer wrote: > Has anything actually happened on the Greenplum release at this point? > > All I can find are the same old press releases. > Nothing has happened. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] The Future of Aggregation

2015-06-14 Thread David Rowley
On 12 June 2015 at 23:57, David Rowley wrote: > On 11 June 2015 at 01:39, Kevin Grittner wrote: > >> >> One question that arose in my mind running this was whether might >> be able to combine sum(x) with count(*) if x was NOT NULL, even >> though the arguments don't match. It might not be worth

Re: [HACKERS] Moving Pivotal's Greenplum work upstream

2015-06-14 Thread Craig Ringer
Has anything actually happened on the Greenplum release at this point? All I can find are the same old press releases. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hacker

[HACKERS] Time to get rid of PQnoPasswordSupplied?

2015-06-14 Thread Craig Ringer
Hi all I frequently see users confused by one of our more common and less clear error messages: fe_sendauth: no password supplied What this really means is that the server requested a password for md5 or cleartext authentication but no password was supplied to the client and it cannot prompt

Re: [HACKERS] creating extension including dependencies

2015-06-14 Thread Pavel Stehule
+1 Is it working in runtime too? Dne 15.6.2015 0:51 napsal uživatel "Petr Jelinek" : > Hi, > > I am getting tired installing manually required extensions manually. I was > wondering if we might want to add option to CREATE SEQUENCE that would > allow automatic creation of the extensions required

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Jeff Davis
On Sun, 2015-06-14 at 16:21 -0400, Tom Lane wrote: > Meh. HashAgg could track its memory usage without loading the entire > system with a penalty. When I tried doing it outside of the MemoryContexts, it seemed to get awkward quickly. I'm open to suggestion if you can point me in the right directi

Re: [HACKERS] Patch to improve a few appendStringInfo* calls

2015-06-14 Thread David Rowley
On 29 May 2015 at 12:51, Peter Eisentraut wrote: > On 5/12/15 4:33 AM, David Rowley wrote: > > Shortly after I sent the previous patch I did a few more searches and > > also found some more things that are not quite right. > > Most of these are to use the binary append method when the length of t

[HACKERS] Sharing aggregate states between different aggregate functions

2015-06-14 Thread David Rowley
Simon and I have been going over some ideas about how to make improvements to aggregate performance by cutting down on the duplicate work that's done when 2 aggregate functions are used where one knows how to satisfy all the requirements of the other. To cut a long story short, all our ideas will

[HACKERS] query execution time faster with geqo on than off: bug?

2015-06-14 Thread David Kamholz
I've encountered a query with 11 joins whose execution time (i.e., the time not taken up by planning) is significantly faster with geqo on rather than off. This is surprising to me and seems like it might be a bug in the planner, so I am posting it here rather than to -performance. The query is be

Re: [HACKERS] The real reason why TAP testing isn't ready for prime time

2015-06-14 Thread Michael Paquier
On Mon, Jun 15, 2015 at 3:37 AM, Tom Lane wrote: > Buildfarm member hamster has failed a pretty significant fraction of > its recent runs in the BinInstallCheck step: > http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamster&br=HEAD > > Since other critters aren't equally distressed, it

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tomas Vondra
Hi, On 06/14/15 22:21, Tom Lane wrote: Jeff Davis writes: This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). ... My general answer to the performance concerns is th

[HACKERS] creating extension including dependencies

2015-06-14 Thread Petr Jelinek
Hi, I am getting tired installing manually required extensions manually. I was wondering if we might want to add option to CREATE SEQUENCE that would allow automatic creation of the extensions required by the extension that is being installed by the user. I also wrote some prototype patch th

Re: [HACKERS] On columnar storage

2015-06-14 Thread Michael Nolan
On Sun, Jun 14, 2015 at 10:30 AM, Tomas Vondra wrote: > >> Are you looking to avoid all hardware-based limits, or would using a 64 >> bit row pointer be possible? That would give you 2^64 or 1.8 E19 unique >> rows over whatever granularity/uniqueness you use (per table, per >> database, etc.)

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tomas Vondra
Hi, On 06/14/15 21:43, Jeff Davis wrote: This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). I see it's adding the new field as int64 - wouldn't a Size be more appr

Re: [HACKERS] Git humor

2015-06-14 Thread Jan de Visser
On June 14, 2015 02:07:16 PM Bruce Momjian wrote: > If you ever had trouble understanding a git manual page, this humorous > random git man page generator is full of fun: > > http://git-man-page-generator.lokaltog.net/ > > Try a few "generate" runs to find one that seems logical. :-) To m

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Peter Geoghegan
On Sat, Jun 13, 2015 at 6:24 PM, Bruce Momjian wrote: >> I think we should really address this. Attached patch adds a new >> release note item for it. It also adds to the documentation that >> explains why users should prefer varchar(n)/text to character(n); the >> lack of abbreviated key support

Re: [HACKERS] Memory Accounting v11

2015-06-14 Thread Tom Lane
Jeff Davis writes: > This patch tracks memory usage (at the block level) for all memory > contexts. Individual palloc()s aren't tracked; only the new blocks > allocated to the memory context with malloc(). > ... > My general answer to the performance concerns is that they aren't a > reason to bloc

[HACKERS] Memory Accounting v11

2015-06-14 Thread Jeff Davis
This patch tracks memory usage (at the block level) for all memory contexts. Individual palloc()s aren't tracked; only the new blocks allocated to the memory context with malloc(). It also adds a new function, MemoryContextMemAllocated() which can either retrieve the total allocated for the contex

[HACKERS] The real reason why TAP testing isn't ready for prime time

2015-06-14 Thread Tom Lane
Buildfarm member hamster has failed a pretty significant fraction of its recent runs in the BinInstallCheck step: http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamster&br=HEAD Since other critters aren't equally distressed, it seems likely that this is just an out-of-disk-space type o

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Alvaro Herrera writes: > >>> Won't this cause issues to MergeAppend optimizations? > > >> Like what? > > > Well, as I understand, MergeAppend needs to know the sort order of the > > child node, right? But that's available only o

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Alvaro Herrera writes: >>> Won't this cause issues to MergeAppend optimizations? >> Like what? > Well, as I understand, MergeAppend needs to know the sort order of the > child node, right? But that's available only on the relation RTE, not > on the c

Re: [HACKERS] Git humor

2015-06-14 Thread Andrew Dunstan
On 06/14/2015 02:07 PM, Bruce Momjian wrote: If you ever had trouble understanding a git manual page, this humorous random git man page generator is full of fun: http://git-man-page-generator.lokaltog.net/ Try a few "generate" runs to find one that seems logical. :-) This looked g

[HACKERS] 9.5 feature count

2015-06-14 Thread Bruce Momjian
I have run a script to count the number of "" items in the major release notes of each major version of Postgres back to 7.4: 7.4280 8.0238 8.1187 8.2230 8.3237 8.4330 9.0252 9.1213 9.2250

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Actually ... if you intend to allow column storage to work with inherited > >> tables (and if you don't, you'd better have a darn good reason why not), > >> I think you probably want to do this join insertion *after* inheritance > >

[HACKERS] Git humor

2015-06-14 Thread Bruce Momjian
If you ever had trouble understanding a git manual page, this humorous random git man page generator is full of fun: http://git-man-page-generator.lokaltog.net/ Try a few "generate" runs to find one that seems logical. :-) -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Actually ... if you intend to allow column storage to work with inherited >> tables (and if you don't, you'd better have a darn good reason why not), >> I think you probably want to do this join insertion *after* inheritance >> expansion, so you can join

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: > I wrote: > > Another model that could be followed is expansion of inheritance-tree > > references, which happens early in the planner. > > Actually ... if you intend to allow column storage to work with inherited > tables (and if you don't, you'd better have a darn good reason wh

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Alvaro Herrera writes: > Another choice I considered was subquery_planner: in the spot where > expand_inherited_tables() is called, add a new call to expand the RTEs > that correspond to tables containing stores. But I had second thoughts > because that function says "it's safe because it only ad

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Bruce Momjian
On Sun, Jun 14, 2015 at 01:57:08PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: > > > > Also, it looks like we could spell your last name with an accent, > > > assuming the i-acute character in Latin1 is fine. > > > > Ah,

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Andres Freund wrote: > On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: > > Rewriter > > > > Parsing occurs as currently. During query rewrite, specifically at the > > bottom of the per-relation loop in fireRIRrules(), we will modify the > > query tree: each relation RTE containing a colstore

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
I wrote: > Another model that could be followed is expansion of inheritance-tree > references, which happens early in the planner. Actually ... if you intend to allow column storage to work with inherited tables (and if you don't, you'd better have a darn good reason why not), I think you probably

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tom Lane
Andres Freund writes: > On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: >> Parsing occurs as currently. During query rewrite, specifically at the >> bottom of the per-relation loop in fireRIRrules(), we will modify the >> query tree: each relation RTE containing a colstore will be replaced >>

Re: [HACKERS] On columnar storage

2015-06-14 Thread Andres Freund
On 2015-06-11 20:03:16 -0300, Alvaro Herrera wrote: > Rewriter > > Parsing occurs as currently. During query rewrite, specifically at the > bottom of the per-relation loop in fireRIRrules(), we will modify the > query tree: each relation RTE containing a colstore will be replaced > with a JoinExp

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Alvaro Herrera
Bruce Momjian wrote: > On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: > > Also, it looks like we could spell your last name with an accent, > > assuming the i-acute character in Latin1 is fine. > > Ah, you are correct. I found a few commits that did have that accent. > All _sev

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-14 Thread Shay Rojansky
On Sun, Jun 14, 2015 at 6:31 PM, Tom Lane wrote: > Shay Rojansky writes: > > [ rely on non-blocking sockets to avoid deadlock ] > > Yeah, that's pretty much the approach libpq has taken: write (or read) > when you can, but press on when you can't. > Good to hear. > The main issue I'm concerned

Re: [HACKERS] On columnar storage

2015-06-14 Thread Petr Jelinek
On 2015-06-14 17:36, Tomas Vondra wrote: On 06/14/15 17:22, Alvaro Herrera wrote: Robert Haas wrote: ,,, Second, there's the idea of a way of storing tuples that is different from PostgreSQL's usual mechanism - i.e. a storage manager API. I understand your concerns about going through the FDW

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tomas Vondra
On 06/14/15 17:22, Alvaro Herrera wrote: Robert Haas wrote: ,,, Second, there's the idea of a way of storing tuples that is different from PostgreSQL's usual mechanism - i.e. a storage manager API. I understand your concerns about going through the FDW API so maybe that's not the right way to

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-14 Thread Tom Lane
Shay Rojansky writes: > [ rely on non-blocking sockets to avoid deadlock ] Yeah, that's pretty much the approach libpq has taken: write (or read) when you can, but press on when you can't. > The main issue I'm concerned about > is SSL/TLS, which is a layer on top of the sockets and which might n

[HACKERS] pg_resetsysid

2015-06-14 Thread Petr Jelinek
Hi, This is an attempt to revive the resetsysid tool submission that didn't go anywhere last year [1]. In that thread there were mainly two complains, one that pg_resetxlog would preferably be tool for more advanced stuff and similar utilities that are not as dangerous should have separate b

Re: [HACKERS] On columnar storage

2015-06-14 Thread Tomas Vondra
Hi, On 06/13/15 00:07, Michael Nolan wrote: On Thu, Jun 11, 2015 at 7:03 PM, Alvaro Herrera mailto:alvhe...@2ndquadrant.com>> wrote: We hope to have a chance to discuss this during the upcoming developer unconference in Ottawa. Here are some preliminary ideas to shed some light o

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Jun 11, 2015 at 7:03 PM, Alvaro Herrera > wrote: > > I've been trying to figure out a plan to enable native column stores > > (CS or "colstore") for Postgres. Motivations: > > > > * avoid the 32 TB limit for tables > > * avoid the 1600 column limit for tables > > * in

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Tom Lane
Bruce Momjian writes: > OK, new idea. What about, instead of having the last name be all-caps, > we have the last name start with an uppercase letter, then use smallcaps > for the rest of the last name: > https://en.wikipedia.org/wiki/Small_caps > That way, the last name will not appear too

Re: [HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

2015-06-14 Thread Tom Lane
Thom Brown writes: > The commit refers to duplicate index names, and only for UNIQUE > indexes. This behaviour is beyond that. And how does it determine > which index to copy? In my example, I placed an index in a different > tablespace. That could be on a drive with very different read/write

Re: [HACKERS] Need Multixact Freezing Docs

2015-06-14 Thread Alvaro Herrera
Josh Berkus wrote: > > > You can see the current multixact value in pg_controldata output. Keep > > timestamped values of that somewhere (a table?) so that you can measure > > consumption rate. I don't think we provide SQL-level access to those > > values. > > Bleh. Do we provide SQL-level acc

Re: [HACKERS] On columnar storage

2015-06-14 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Amit Kapila wrote: > >> Will the column store obey snapshot model similar to current heap tuples, > >> if so will it derive the transaction information from heap tuple? > > > Yes, visibility will be tied to the heap tuple -- a value is accessed > > onl

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote: > > > Note that I guess "MauMau" is a nickname. > > > > I think we are fine consistenly putting Japanese last names first _if_ > > we always capitalize the last name, e.g. "FUJITA Etsuro" ---

Re: [HACKERS] s_lock() seems too aggressive for machines with many sockets

2015-06-14 Thread Jan Wieck
The whole thing turns out to be based on wrong baseline data, taken with a pgbench client running from a remote machine. It all started out from an investigation against 9.3. Curiously enough, the s_lock() problem that existed in 9.3 has a very similar effect on throughput as a network bottlene

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Bruce Momjian
On Sat, Jun 13, 2015 at 11:30:30PM -0400, Bruce Momjian wrote: > > Note that I guess "MauMau" is a nickname. > > I think we are fine consistenly putting Japanese last names first _if_ > we always capitalize the last name, e.g. "FUJITA Etsuro" --- is that a > good idea? Does everyone like that? Do

Re: [HACKERS] Collection of memory leaks for ECPG driver

2015-06-14 Thread Michael Paquier
On Sat, Jun 13, 2015 at 6:25 PM, Michael Meskes wrote: > On Sat, Jun 13, 2015 at 12:02:40AM -0400, Tom Lane wrote: >> But having said that, I would not be in a hurry to remove any existing >> if-guards for the case. For one thing, it makes the code look more >> similar to backend code that uses p

Re: [HACKERS] Problem with CREATE TABLE ... (LIKE ... INCLUDING INDEXES)

2015-06-14 Thread Thom Brown
On 14 June 2015 at 04:25, Michael Paquier wrote: > On Sun, Jun 14, 2015 at 11:38 AM, Thom Brown wrote: >> As you can see, 3 indexes are missing, which happen to be ones that >> would duplicate the column definition of another index. Is this >> intentional? If so, shouldn't it be documented beha

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Dean Rasheed
On 11 June 2015 at 05:15, Bruce Momjian wrote: > I have committed the first draft of the 9.5 release notes. You can view > the output here: > > http://momjian.us/pgsql_docs/release-9-5.html > I think it's worth mentioning dcbf5948e12aa60b4d6ab65b6445897dfc971e01, probably under "General

[HACKERS] CREATE FUNCTION .. LEAKPROOF docs

2015-06-14 Thread Dean Rasheed
I think the docs for the LEAKPROOF option in create_function.sgml ought to mention RLS as well as security barrier views. Also the current text is no longer strictly correct in light of commit dcbf5948e12aa60b4d6ab65b6445897dfc971e01. Suggested rewording attached. Regards, Dean diff --git a/doc/sr

[HACKERS] Tab completion for TABLESAMPLE

2015-06-14 Thread Petr Jelinek
Hi, looks like I omitted psql tab completion from the TABLESAMPLE patch. The attached patch adds it. -- Petr Jelinek http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c i

Re: [HACKERS] 9.5 release notes

2015-06-14 Thread Petr Jelinek
On 2015-06-14 03:02, Bruce Momjian wrote: On Sat, Jun 13, 2015 at 05:47:59AM -0300, Alvaro Herrera wrote: Petr Jelinek wrote: Hi, + + +Add JSONB functions jsonb_set() and +jsonb_pretty (Dmitry Dolgov, Andrew Dunstan) + + I believe I should be 3rd auth

Re: [HACKERS] git push hook to check for outdated timestamps

2015-06-14 Thread Magnus Hagander
On Fri, Jun 12, 2015 at 4:33 PM, Tom Lane wrote: > Andrew Dunstan writes: > > On 06/12/2015 09:31 AM, Robert Haas wrote: > >> Could we update our git hook to refuse a push of a new commit whose > >> timestamp is more than, say, 24 hours in the past? Our commit history > >> has some timestamps i

Re: [HACKERS] Entities created in one query not available in another in extended protocol

2015-06-14 Thread Shay Rojansky
Tom, I agree this is entirely a client-side issue. Regardless, as Simon says it would be useful to have some documentation for client implementors. Sehrope, thanks for the JDBC link! I was actually thinking of going about it another way in Npgsql: 1. Send messages normally until the first Exec