Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2011-11-07 Thread Heikki Linnakangas
On 07.11.2011 20:36, Tom Lane wrote: Alexander Korotkov writes: So, !? and ? operators are mentioned in documentation, but don't present in catalog. Are them just missed in the catalog or there is some more serious problem? IIRC, Heikki removed them from the final commit. Sounds like he miss

Re: [HACKERS] heap vacuum & cleanup locks

2011-11-07 Thread Simon Riggs
On Tue, Nov 8, 2011 at 2:54 AM, Robert Haas wrote: > It would still be nice to fix the case where we need to freeze a tuple > that is on a page someone else has pinned, but I don't have any good > ideas for how to do that. I think we need to avoid long pin hold times generally. --  Simon Riggs

Re: [HACKERS] ProcArrayLock contention

2011-11-07 Thread Simon Riggs
On Tue, Nov 8, 2011 at 4:52 AM, Robert Haas wrote: > With 80 clients (but not 32 or fewer), I occasionally get the > following error: > > ERROR:  t_xmin is uncommitted in tuple to be updated > > So it seems that there's some way in which this locking is actually > incorrect, though I'm not seeing

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Simon Riggs
On Tue, Nov 8, 2011 at 2:35 AM, Robert Haas wrote: > What I was expecting you to do is eliminate > wal_writer_delay altogether and drive the wakeups entirely off of the > latch. Please continue to expect that, I just haven't finished it yet... --  Simon Riggs   http://www.2ndQu

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Nov 7, 2011, at 9:35 PM, Robert Haas wrote: > On Mon, Nov 7, 2011 at 6:33 PM, Simon Riggs wrote: >> On Mon, Nov 7, 2011 at 5:26 PM, Simon Riggs wrote: >> 5. Make the WAL writer more responsive, maybe using latches, so that it doesn't take as long for the commit record to make it ou

[HACKERS] ProcArrayLock contention

2011-11-07 Thread Robert Haas
I've been playing with the attached patch, which adds an additional light-weight lock mode, LW_SHARED2. LW_SHARED2 conflicts with LW_SHARED and LW_EXCLUSIVE, but not with itself. The patch changes ProcArrayEndTransaction() to use this new mode. IOW, multiple processes can commit at the same time

Re: [HACKERS] psql expanded auto

2011-11-07 Thread Peter Eisentraut
On lör, 2011-11-05 at 12:26 -0400, Noah Misch wrote: > On Sat, Nov 05, 2011 at 04:53:56PM +0200, Peter Eisentraut wrote: > > On fre, 2011-11-04 at 07:34 -0400, Noah Misch wrote: > > > For "\pset format wrapped", we only use $COLUMNS when the output is a > > > tty. I'm thinking it's best, although

Re: [HACKERS] proposal: psql concise mode

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 11:01 PM, Josh Kupershmidt wrote: > What really prompted the proposal was my somewhat antiquated use of > 80-column terminal windows (so that 2 or 3 fit side-by-side > comfortably on my screen). A lot of the backslash commands are > creeping well over that 80-column limit, e

Re: [HACKERS] proposal: psql concise mode

2011-11-07 Thread Josh Kupershmidt
On Mon, Nov 7, 2011 at 10:04 PM, Robert Haas wrote: > I don't strongly object to this, but I wonder how useful it will > really be in practice.  It strikes me as the sort of advanced psql > hackery that only a few people will use, and only some of those will > gain any benefit. I'm probably just

Re: [HACKERS] proposal: psql concise mode

2011-11-07 Thread Robert Haas
On Sun, Nov 6, 2011 at 3:29 PM, Josh Kupershmidt wrote: > On Sun, Nov 6, 2011 at 1:16 PM, Dickson S. Guedes > wrote: >>> test=# \d+ foo >>>                         Table "public.foo" >>>  Column |  Type   | Storage >>> +-+- >>>  a      | integer | plain >>>  b      | inte

Re: [HACKERS] removing =>(text, text) in 9.2

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 2:37 PM, Tom Lane wrote: > Robert Haas writes: >> Thanks for the review.  New version attached. > > This looks sane to me too (though I only read the patch and didn't > actually test upgrading). OK, committed. Thanks for the reviews, all. -- Robert Haas EnterpriseDB: ht

Re: [HACKERS] heap vacuum & cleanup locks

2011-11-07 Thread Robert Haas
On Fri, Nov 4, 2011 at 3:39 PM, Robert Haas wrote: > Doing that actually makes the patch simpler, so I went ahead and did > that in the attached version, along with the renaming mentioned above. Hearing no objections, I went ahead and committed this version. Thanks for coding this up; I think th

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 6:33 PM, Simon Riggs wrote: > On Mon, Nov 7, 2011 at 5:26 PM, Simon Riggs wrote: > >>> 5. Make the WAL writer more responsive, maybe using latches, so that >>> it doesn't take as long for the commit record to make it out to disk. >> >> I'm working on this already as part of

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Simon Riggs
On Mon, Nov 7, 2011 at 5:26 PM, Simon Riggs wrote: >> 5. Make the WAL writer more responsive, maybe using latches, so that >> it doesn't take as long for the commit record to make it out to disk. > > I'm working on this already as part of the update for power > reduction/group commit/replication

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-11-07 Thread Tom Lane
Josh Berkus writes: > As I said elsewhere, I think that a modified version of Simon's proposal > gets us all what we want except code cleanliness. I'd like to hear from > Tom on that issue. Well, code complexity is hard to gauge without coding a draft implementation, but I think this largely fai

Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2011-11-07 Thread Alexander Korotkov
First version of GiST for range types patch is here. Comments & refactoring & testing are coming soon. -- With best regards, Alexander Korotkov. rangetypegist-0.1.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make cha

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Greg Jaskiewicz
On 7 Nov 2011, at 15:44, Heikki Linnakangas wrote: > On 07.11.2011 14:01, Gregg Jaskiewicz wrote: >> On 7 November 2011 11:57, Heikki Linnakangas >> wrote: >> >>> Looks like the range types patch was broken for float timestamps. I'll go >>> fix that. Thanks for the report! >> >> yup, no probs

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-11-07 Thread Josh Berkus
> Agreed. This thread has already expended too much of our valuable time, > in my opinion. As I said elsewhere, I think that a modified version of Simon's proposal gets us all what we want except code cleanliness. I'd like to hear from Tom on that issue. The proposal is: 1. No changes are exp

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Merlin Moncure
On Mon, Nov 7, 2011 at 12:19 PM, Robert Haas wrote: > On Mon, Nov 7, 2011 at 1:08 PM, Merlin Moncure wrote: >> Thinking about this more, the backend local cache approach is probably >> going to be useless in terms of addressing this problem -- mostly due >> to the fact that the cache is, well, lo

Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2011-11-07 Thread Tom Lane
Alexander Korotkov writes: > So, !? and ? operators are mentioned in documentation, but don't present in > catalog. Are them just missed in the catalog or there is some more serious > problem? IIRC, Heikki removed them from the final commit. Sounds like he missed some documentation.

Re: GiST for range types (was Re: [HACKERS] Range Types - typo + NULL string constructor)

2011-11-07 Thread Alexander Korotkov
During work on gist for range types I've faced with following problem: test=# select 'empty'::int4range !?; ERROR: operator does not exist: int4range !? LINE 1: select 'empty'::int4range !?; ^ HINT: No operator matches the given name and argument type(s). You mi

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 1:08 PM, Merlin Moncure wrote: > Thinking about this more, the backend local cache approach is probably > going to be useless in terms of addressing this problem -- mostly due > to the fact that the cache is, well, local.  Even if backend A takes > the time to mark the bit i

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Merlin Moncure
On Mon, Nov 7, 2011 at 9:25 AM, Merlin Moncure wrote: > On Mon, Nov 7, 2011 at 8:31 AM, Robert Haas wrote: >> I've long considered synchronous_commit=off to be one of our best >> performance features.  Certainly, it's not applicable in every >> situation, but there are many applications where los

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 12:26 PM, Simon Riggs wrote: >> 5. Make the WAL writer more responsive, maybe using latches, so that >> it doesn't take as long for the commit record to make it out to disk. > > I'm working on this already as part of the update for power > reduction/group commit/replication

Re: [HACKERS] voting to the xslt_process() need

2011-11-07 Thread Tom Lane
Andrew Dunstan writes: > Please also see earlier discussions of these items in the mailing list. Yes. It's very unlikely that we'll accept a patch that just moves xslt_process into core without doing anything about its definitional and implementation shortcomings. See for instance http://archi

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:53 AM, Florian Pflug wrote: > > Various issues with OSX and UTF-8 locales seems to come up quite often, yet > we're not really in a position to do anything about them. > > Thus, I think we should warn about these issues and save people the trouble > of finding out about t

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Would it even really be worth it to look into any of the other locale > issues on OSX, given that PostgreSQL is now included in their default > installs starting with 10.7, or would this really be more of a case of > hoping Apple some day fixes the issue upstream? To my mind, th

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:59 AM, Tom Lane wrote: > > If you have time to check that the patch I just committed fixes your > problem, it'd be worth doing.  I did not test it on OS X ... Looks good to me, thanks. Would it even really be worth it to look into any of the other locale issues on OSX,

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Simon Riggs
On Mon, Nov 7, 2011 at 2:45 PM, Robert Haas wrote: >> 2. Improve CLOG concurrency or performance in some way so that >> consulting it repeatedly doesn't slow us down so much. We should also ask what makes the clog slow. I think it shows physical contention as well as logical contention on the lw

Re: [HACKERS] voting to the xslt_process() need

2011-11-07 Thread Andrew Dunstan
On 11/07/2011 12:10 PM, Peter Padua Krauss wrote: I vote to see the contrib/xml2's xslt_process() build-in function (only xslt_process) moved into PostgreSQL core. Database servers can offer some "load balancing" with another CPUs, like a PHP server... I have in mind some main exa

Re: [HACKERS] type privileges and default privileges

2011-11-07 Thread Robert Haas
On Sat, Nov 5, 2011 at 10:35 AM, Peter Eisentraut wrote: > During the closing days of the 9.1 release, we had discussed that we > should add privileges on types (and domains), so that owners can prevent > others from using their types because that would prevent the owners from > changing them in c

[HACKERS] voting to the xslt_process() need

2011-11-07 Thread Peter Padua Krauss
I vote to see the contrib/xml2's xslt_process() build-in function (only xslt_process) moved into PostgreSQL core. Database servers can offer some "load balancing" with another CPUs, like a PHP server... I have in mind some main examples: * If database server not process XSLT, the *balance*

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Anyways, lemme know if there's anything else I could help with or > could test and whatnot. Cheers. If you have time to check that the patch I just committed fixes your problem, it'd be worth doing. I did not test it on OS X ... regards, tom lane -- S

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread J Smith
On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane wrote: > I looked at this a bit and realized that sscanf is actually doing a > couple of critical things for us, which are lost in translation when > doing it like this: > > 1. It ignores whitespace other than the dividing tab.  If we don't > continue to d

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Florian Pflug
On Nov7, 2011, at 17:46 , J Smith wrote: > On Mon, Nov 7, 2011 at 11:12 AM, Tom Lane wrote: >> If OS X's UTF8 locales weren't so thoroughly broken (eg sorting does not >> work), I might be tempted to try to do it that way, but I still fail >> to see the point. After reviewing the code I feel that

Re: [HACKERS] btree gist known problems

2011-11-07 Thread Jeff Davis
On Mon, 2011-11-07 at 10:18 -0500, Tom Lane wrote: > Jeff Davis writes: > > I'm looking for known problem areas in btree_gist. I see: > > http://archives.postgresql.org/message-id/8973.1286841...@sss.pgh.pa.us > > > With Range Types, I'm anticipating that btree_gist will become more > > important

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 10:12 AM, Tom Lane wrote: > Robert Haas writes: >> SetHintBits() can't set HEAP_XMIN_COMMITTED or HEAP_XMAX_COMMITTED >> hints until the commit record has been durably flushed to disk.  It >> turns out that can cause a major performance regression on systems >> with many CP

Re: [HACKERS] unaccent extension missing some accents

2011-11-07 Thread Tom Lane
J Smith writes: > Alright, I wrote up another patch that uses strchr to parse out the > lines of the unaccent.rules file, foregoing sscanf completely. > Hopefully this looks a bit better than using swscanf. I looked at this a bit and realized that sscanf is actually doing a couple of critical thi

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 10:27 AM, Greg Smith wrote: > On 11/07/2011 09:03 AM, Robert Haas wrote: >> >> I think it's an established principle that the design for features >> like this should, for best results, be discussed on -hackers before >> writing a lot of code. > > You can see from the commit

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Greg Smith
On 11/06/2011 06:00 PM, Tom Lane wrote: Peter Geoghegan writes: > A major consideration was backwards compatibility; This is not a consideration that the community is likely to weigh heavily, or indeed at all. We aren't going to back-port this feature into prior release branches, and we aren'

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Heikki Linnakangas
On 07.11.2011 14:01, Gregg Jaskiewicz wrote: On 7 November 2011 11:57, Heikki Linnakangas wrote: Looks like the range types patch was broken for float timestamps. I'll go fix that. Thanks for the report! yup, no probs. Fixed. -- Heikki Linnakangas EnterpriseDB http://www.enterprise

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Merlin Moncure
On Mon, Nov 7, 2011 at 9:12 AM, Tom Lane wrote: > Robert Haas writes: >> SetHintBits() can't set HEAP_XMIN_COMMITTED or HEAP_XMAX_COMMITTED >> hints until the commit record has been durably flushed to disk.  It >> turns out that can cause a major performance regression on systems >> with many CPU

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Greg Smith
On 11/07/2011 09:03 AM, Robert Haas wrote: I think it's an established principle that the design for features like this should, for best results, be discussed on -hackers before writing a lot of code. You can see from the commit history this idea is less than a month old. Do we need to get co

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Merlin Moncure
On Mon, Nov 7, 2011 at 8:31 AM, Robert Haas wrote: > I've long considered synchronous_commit=off to be one of our best > performance features.  Certainly, it's not applicable in every > situation, but there are many applications where losing a second or so > worth of transactions is an acceptable

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Andrew Dunstan
On 11/07/2011 09:40 AM, Kevin Grittner wrote: Gregg Jaskiewicz wrote: On 7 November 2011 12:14, Thom Brown wrote: I notice that there are no machines in the build farm which build HEAD and use float timestamps, so this could silently happen again unless the Grzegorz Jaskiewiczs out there co

Re: [HACKERS] btree gist known problems

2011-11-07 Thread Tom Lane
Jeff Davis writes: > I'm looking for known problem areas in btree_gist. I see: > http://archives.postgresql.org/message-id/8973.1286841...@sss.pgh.pa.us > With Range Types, I'm anticipating that btree_gist will become more > important, so I'd like to know what bugs are holding it back. > So, any

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Tom Lane
Robert Haas writes: > SetHintBits() can't set HEAP_XMIN_COMMITTED or HEAP_XMAX_COMMITTED > hints until the commit record has been durably flushed to disk. It > turns out that can cause a major performance regression on systems > with many CPU cores. It seems to me that you've jumped to proposing

Re: [HACKERS] -Wcast-qual cleanup, part 1

2011-11-07 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 7, 2011 at 12:13 AM, Peter Eisentraut wrote: >> typedef struct ErrorContextCallback >> { >> struct ErrorContextCallback *previous; >> -void(*callback) (void *arg); >> -void *arg; >> +void(*callback) (const void *ar

Re: [HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 9:31 AM, Robert Haas wrote: > So, what could we do about this?  Ideas: > > 1. Set the hint bits right away, and avoid letting the page be flushed > to disk until the commit record is durably on disk (by bumping the > page LSN?). > 2. Improve CLOG concurrency or performance i

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Kevin Grittner
Gregg Jaskiewicz wrote: > On 7 November 2011 12:14, Thom Brown wrote: >> >> I notice that there are no machines in the build farm which build >> HEAD and use float timestamps, so this could silently happen >> again unless the Grzegorz Jaskiewiczs out there continue to >> report build issues. > >

[HACKERS] synchronous commit vs. hint bits

2011-11-07 Thread Robert Haas
I've long considered synchronous_commit=off to be one of our best performance features. Certainly, it's not applicable in every situation, but there are many applications where losing a second or so worth of transactions is an acceptable price to pay for not needing to wait for the disk to spin ar

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Robert Haas
On Sun, Nov 6, 2011 at 2:56 PM, Greg Smith wrote: > Peter's patch adds a planner hook and hashes at that level.  Since this cat > is rapidly escaping its bag and impacting other contributors, we might as > well share the work in progress early if anyone has a burning desire to look > at the code.

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Gregg Jaskiewicz
On 7 November 2011 12:14, Thom Brown wrote: > > I notice that there are no machines in the build farm which build HEAD > and use float timestamps, so this could silently happen again unless > the Grzegorz Jaskiewiczs out there continue to report build issues. I am rebuilding head once a week-ish

Re: [HACKERS] xslt_process() need

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 8:20 AM, Peter Padua Krauss wrote: > Can I vote to the "xslt_process need" in the SQL/XML core of the PostgreSQL > 9.X ? Not sure I understand this. Are you saying that you'd like to see contrib/xml2's xslt_process moved into core? ...Robert -- Sent via pgsql-hackers ma

[HACKERS] xslt_process() need

2011-11-07 Thread Peter Padua Krauss
Can I vote to the "xslt_process need" in the SQL/XML core of the PostgreSQL 9.X ?

Re: [HACKERS] -Wcast-qual cleanup, part 1

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 12:13 AM, Peter Eisentraut wrote: > Anyway, attached is the first patch for your amusement. I can't help but wonder if the cure isn't worse than the disease. I mean, I very much like the fact that our code compiles without warnings, and I'm glad you're willing to put in th

Re: [HACKERS] Storing hot members of PGPROC out of the band

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 6:45 AM, Heikki Linnakangas wrote: > While looking at GetSnapshotData(), it occurred to me that when a PGPROC > entry does not have its xid set, ie. xid == InvalidTransactionId, it's > pointless to check the subxid array for that proc. If a transaction has no > xid, none of

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Thom Brown
On 7 November 2011 11:57, Heikki Linnakangas wrote: > On 07.11.2011 13:14, Gregg Jaskiewicz wrote: >> >> http://pastebin.com/RjiRjGZc >> >> this is on fedora 12, gcc 4.6.1 > >> rangetypes.c: In function ‘tsrange_subdiff’: >> rangetypes.c:1316:11: error: ‘timestamp’ undeclared (first use in this >>

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Gregg Jaskiewicz
On 7 November 2011 11:57, Heikki Linnakangas wrote: > Looks like the range types patch was broken for float timestamps. I'll go > fix that. Thanks for the report! yup, no probs. -- GJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] git trunk doesn't build

2011-11-07 Thread Heikki Linnakangas
On 07.11.2011 13:14, Gregg Jaskiewicz wrote: http://pastebin.com/RjiRjGZc this is on fedora 12, gcc 4.6.1 rangetypes.c: In function ‘tsrange_subdiff’: rangetypes.c:1316:11: error: ‘timestamp’ undeclared (first use in this function) rangetypes.c:1316:11: note: each undeclared identifier is rep

Re: [HACKERS] Storing hot members of PGPROC out of the band

2011-11-07 Thread Heikki Linnakangas
On 04.11.2011 00:43, Simon Riggs wrote: On Thu, Nov 3, 2011 at 6:12 PM, Pavan Deolasee wrote: When PGPROC array is allocated, we also allocate another array of PGPROC_MINIMAL structures of the same size. While accessing the ProcArray, a simple pointer mathematic can get us the corresponding PG

Re: [HACKERS] WIP: Collecting statistics on CSV file data

2011-11-07 Thread Shigeru Hanada
(2011/10/20 18:56), Etsuro Fujita wrote: > I revised the patch according to Hanada-san's comments. Attached is the > updated version of the patch. > > Changes: > >* pull up of logging "analyzing foo.bar" >* new vac_update_relstats always called >* tab-completion in psql >* add "fo

[HACKERS] git trunk doesn't build

2011-11-07 Thread Gregg Jaskiewicz
http://pastebin.com/RjiRjGZc this is on fedora 12, gcc 4.6.1 -- GJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Storing hot members of PGPROC out of the band

2011-11-07 Thread Pavan Deolasee
On Fri, Nov 4, 2011 at 4:13 AM, Simon Riggs wrote: > > If you look at your PGPROC_MINIMAL, its mostly transaction related > stuff, so I would rename it PGXACT or similar. Yeah, that looks good too. Though I am not sure if all fields are related to transaction state and whether we would need to a