Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Dmitriy Igrishin
2011/11/24 Alexander Shulgin > > Excerpts from Dmitriy Igrishin's message of Thu Nov 24 09:19:02 +0200 2011: > > > > > If host part of the URI points to localhost, the UNIX domain socket > would > > > be considered by libpq just as if you would pass "-h localhost -p > 5433". > > > > > But what if

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Martijn van Oosterhout
On Thu, Nov 24, 2011 at 08:59:56AM +0200, Alexander Shulgin wrote: > > How would you specifiy a local port/UNIX domain socket? > > Missed that in my previous reply. > > If host part of the URI points to localhost, the UNIX domain socket would be > considered by libpq just as if you would pass "-

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Alexander Shulgin
Excerpts from Dmitriy Igrishin's message of Thu Nov 24 09:19:02 +0200 2011: > > > If host part of the URI points to localhost, the UNIX domain socket would > > be considered by libpq just as if you would pass "-h localhost -p 5433". > > > But what if the user wants to connect exactly via socket o

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Dmitriy Igrishin
Hey Alexander, 2011/11/24 Alexander Shulgin > > Excerpts from Florian Weimer's message of Wed Nov 23 13:04:47 +0200 2011: > > > > * Alexander Shulgin: > > > > > This, in my opinion, is very similar to what we would like to achieve > with the URI syntax, so the above could also be specified using

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Alexander Shulgin
Excerpts from Florian Weimer's message of Wed Nov 23 13:04:47 +0200 2011: > > * Alexander Shulgin: > > > This, in my opinion, is very similar to what we would like to achieve with > > the URI syntax, so the above could also be specified using a URI parameter > > like this: > > > > psql -d po

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Alexander Shulgin
Excerpts from Florian Weimer's message of Wed Nov 23 13:04:47 +0200 2011: > > * Alexander Shulgin: > > > This, in my opinion, is very similar to what we would like to achieve with > > the URI syntax, so the above could also be specified using a URI parameter > > like this: > > > > psql -d po

Re: [HACKERS] pg_upgrade relation OID mismatches

2011-11-23 Thread Bruce Momjian
Bruce Momjian wrote: > OK, that is a heap table. My only guess is that the heap is being > created without binary_upgrade_next_heap_pg_class_oid being set. > Looking at the code, I can't see how the heap could be created without > this happening. Another idea is that pg_dumpall isn't output the p

Re: [HACKERS] logging in high performance systems.

2011-11-23 Thread Greg Smith
On 11/23/2011 09:28 PM, Theo Schlossnagle wrote: The second thing I did was write a sample use of those hooks to implement a completely non-blocking fifo logger. (if it would block, it drops the log line). The concept is that we could run this without risk of negative performance impact due to s

Re: [HACKERS] Obstacles to user-defined range canonicalization functions

2011-11-23 Thread David E. Wheeler
On Nov 23, 2011, at 10:33 PM, Tom Lane wrote: > Now you could argue that for performance reasons everybody should write > their canonicalization functions in C anyway, but I'm not sure I buy > that --- at the very least, it'd be nice to write the functions in > something higher-level while prototy

Re: [HACKERS] Obstacles to user-defined range canonicalization functions

2011-11-23 Thread Florian Pflug
On Nov24, 2011, at 04:33 , Tom Lane wrote: > One possibility that just came to me is to decree that every discrete > range type has to be based on an underlying continuous range type (with > all the same properties except no canonicalization function), and then > the discrete range's canonicalizati

[HACKERS] Obstacles to user-defined range canonicalization functions

2011-11-23 Thread Tom Lane
I got religion this evening about the potential usefulness of user-defined canonicalization functions --- the example that did it for me was thinking about a range type over timestamp that quantizes boundaries to hours, or half hours, or 15 minutes, or any scheduling unit that is standard in a part

[HACKERS] logging in high performance systems.

2011-11-23 Thread Theo Schlossnagle
We have a need for logging in systems where it isn't feasible to log to disk as it negatively impacts performance. I'd like to be able to creatively solve this problem without modifying the core, but today I cannot. So... here's my first whack at solving this with some flexibility. The first thi

[HACKERS] PL/Python SQL error code pass-through

2011-11-23 Thread Mika Eloranta
Hi all, Here's a little SQL snippet that exposes an apparent regression in the 9.1.x PL/Python behavior: ---clip--- # cat foo.sql \set VERBOSITY 'verbose' CREATE table bar (a INTEGER CONSTRAINT hello CHECK (a > 1)); CREATE OR REPLACE FUNCTION foo () RETURNS integer AS $$ plpy.execute("INS

Re: [HACKERS] [JDBC] Optimize postgres protocol for fixed size arrays

2011-11-23 Thread Kevin Grittner
Oliver Jowett wrote: > Can we get a mechanism for minor protocol changes in this future > version? Something as simple as exchanging a list of protocol > features during the initial handshake (then use only features that > are present on both sides) would be enough. The difficulty of > making an

Re: [HACKERS] [JDBC] Optimize postgres protocol for fixed size arrays

2011-11-23 Thread Oliver Jowett
On 24 November 2011 05:36, Tom Lane wrote: > Now it's possible we could do that without formally calling it a > protocol version change, but I don't care at all for the idea of coming > up with one-off hacks every time somebody decides that some feature is > important enough that they have to hav

Re: [HACKERS] FlexLocks

2011-11-23 Thread Kevin Grittner
"Kevin Grittner" wrote: > Robert Haas wrote: >> Updated patches attached. > > I have to admit I don't have my head around the extraWaits issue, > so I can't personally vouch for that code, although I have no > reason to doubt it, either. Everything else was something that I at > least *think*

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 10:47 PM, Robert Haas wrote: > On Wed, Nov 23, 2011 at 5:43 PM, Simon Riggs wrote: >> Computing two cutoffs is overkill for the rare event of pruning a >> shared catalog page. I did think of that already and its not a good >> solution. I'm tempted by the idea of getting ri

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 5:43 PM, Simon Riggs wrote: > Computing two cutoffs is overkill for the rare event of pruning a > shared catalog page. I did think of that already and its not a good > solution. I'm tempted by the idea of getting rid of multiple databases > altogether. The hassle of support

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 9:55 PM, Tom Lane wrote: > Simon Riggs writes: >> My patch actually improves the speed of snapshots, rather than slowing >> them as Tom's would. > > It can be arbitrarily fast if it doesn't have to get the right answer. (LOL) - laughing with you > Unfortunately, you're n

[HACKERS] Snapshot build updates

2011-11-23 Thread Magnus Hagander
The snapshots on the ftpsite have been down for a number of days, since hub.org upgraded the machine it used to be on and git stopped working there. Since we were planning to move it anyway, we didn't bother doing anything about it at the time. The snapshots are now auto-generated by buildfarm ani

Re: [HACKERS] [Review] Include detailed information about a row failing a CHECK constraint into the error message

2011-11-23 Thread Robert Haas
On Mon, Nov 21, 2011 at 8:59 AM, Jan Kundrát wrote: > What is the suggested way to go form here? Shall I update the unit tests? Yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Tom Lane
Simon Riggs writes: > My patch actually improves the speed of snapshots, rather than slowing > them as Tom's would. It can be arbitrarily fast if it doesn't have to get the right answer. Unfortunately, you're not producing the right answer. You can not exclude XIDs in other databases from the sn

Re: [HACKERS] range_adjacent and discrete ranges

2011-11-23 Thread Tom Lane
I wrote: > I did a little bit of performance testing on an x86_64 machine (Fedora 14), > and found that the time to execute a clause like > WHERE int4range(1,2) -|- int4range(x, 1000) > (x being an integer Var) grows from 0.37 us to 0.56 us if we adopt the > patched version of range_adjac

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 8:45 PM, Robert Haas wrote: > I've wondered a few times whether we could get rid of the > RecentGlobalXmin computation from GetSnapshotData() altogether. You have to calculate an xmin, so its unavoidable. My patch actually improves the speed of snapshots, rather than slo

Re: [HACKERS] range_adjacent and discrete ranges

2011-11-23 Thread Tom Lane
I wrote: > Attached is a draft patch for this. It passes regression tests but I've > not tried to exercise it with a canonical function that actually does > something different. I hacked up int4range_canonical to produce []-style ranges, and confirmed that this version of range_adjacent seems to

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 3:15 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Nov 23, 2011 at 1:30 PM, Tom Lane wrote: >>> What I think might make more sense is to keep two variables, >>> RecentGlobalXmin with its current meaning and RecentDatabaseWideXmin >>> which considers only xmins of t

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 8:15 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Nov 23, 2011 at 1:30 PM, Tom Lane wrote: >>> What I think might make more sense is to keep two variables, >>> RecentGlobalXmin with its current meaning and RecentDatabaseWideXmin >>> which considers only xmins of t

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 23, 2011 at 1:30 PM, Tom Lane wrote: >> What I think might make more sense is to keep two variables, >> RecentGlobalXmin with its current meaning and RecentDatabaseWideXmin >> which considers only xmins of transactions in the current database. >> Then HOT cleanup

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 7:57 PM, Robert Haas wrote: > On Wed, Nov 23, 2011 at 1:30 PM, Tom Lane wrote: >> What I think might make more sense is to keep two variables, >> RecentGlobalXmin with its current meaning and RecentDatabaseWideXmin >> which considers only xmins of transactions in the curre

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 1:30 PM, Tom Lane wrote: > What I think might make more sense is to keep two variables, > RecentGlobalXmin with its current meaning and RecentDatabaseWideXmin > which considers only xmins of transactions in the current database. > Then HOT cleanup could select the appropria

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 6:15 PM, Tom Lane wrote: >> The real question is do we favour HOT cleanup on those small 8 tables, >> or do we favour HOT cleanup of every other table? > > No, the real question is why not think a little harder and see if we can > come up with a solution that doesn't invol

Re: [HACKERS] Inlining comparators as a performance optimisation

2011-11-23 Thread Robert Haas
On Tue, Nov 22, 2011 at 8:09 PM, Peter Geoghegan wrote: > I wonder, is it worth qualifying that the "Sort Method" was a > "quicksort (fast path)" sort within explain analyze output? This is a > rather large improvement, so It might actually be something that > people look out for, as it might be t

Re: [HACKERS] plpython SPI cursors

2011-11-23 Thread Jan Urbański
On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory (that's what plpy.execute does). J I foun

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Tom Lane
Simon Riggs writes: > On Tue, Nov 22, 2011 at 11:40 PM, Simon Riggs wrote: >>> I think this is unsafe for shared catalogs. >> I think so too. Thats why it uses IsMVCCSnapshot() to confirm when it >> is safe to do so. > Ah, you mean access to shared catalogs using MVCC snapshots. [ having now r

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Tom Lane
Simon Riggs writes: > On Wed, Nov 23, 2011 at 5:01 PM, Tom Lane wrote: >> Simon Riggs writes: >>> Normal catalog access does not use HOT and never has. >> You are mistaken. > Normal catalog access against shared catalogs via heap_scan does not > use HOT cleanup, because it uses SnapshotNow. N

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 5:01 PM, Tom Lane wrote: > Simon Riggs writes: >> On Wed, Nov 23, 2011 at 3:20 PM, Alvaro Herrera >> wrote: >>> Maybe not pg_database or pg_tablespace and such, but I'm not so sure >>> about pg_shdepend. (Do we record pg_shdepend entries for temp tables?) > >> Normal cata

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Pavan Deolasee
On Wed, Nov 23, 2011 at 9:44 PM, Simon Riggs wrote: > On Wed, Nov 23, 2011 at 3:20 PM, Alvaro Herrera > wrote: >> >> Excerpts from Robert Haas's message of mié nov 23 12:15:55 -0300 2011: >> >>> > And it effects shared catalogs only, which are all low traffic anyway. >>> >>> I think "low traffic"

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Tom Lane
Simon Riggs writes: > On Wed, Nov 23, 2011 at 3:20 PM, Alvaro Herrera > wrote: >> Maybe not pg_database or pg_tablespace and such, but I'm not so sure >> about pg_shdepend. (Do we record pg_shdepend entries for temp tables?) > Normal catalog access does not use HOT and never has. You are mistak

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mié nov 23 13:14:04 -0300 2011: > On Wed, Nov 23, 2011 at 3:20 PM, Alvaro Herrera > wrote: > > > > Excerpts from Robert Haas's message of mié nov 23 12:15:55 -0300 2011: > > > >> > And it effects shared catalogs only, which are all low traffic anyway. > >> >

Re: [HACKERS] [JDBC] Optimize postgres protocol for fixed size arrays

2011-11-23 Thread Tom Lane
Merlin Moncure writes: > On Tue, Nov 22, 2011 at 6:52 PM, Tom Lane wrote: >> The only way that anything like this will go in is as part of a protocol >> version bump, > Wire format changes can only be made with a protocol version bump? Is > this a new policy? In the past they were just made...f

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 3:20 PM, Alvaro Herrera wrote: > > Excerpts from Robert Haas's message of mié nov 23 12:15:55 -0300 2011: > >> > And it effects shared catalogs only, which are all low traffic anyway. >> >> I think "low traffic" is the key point.  I understand that you're not >> changing th

Re: [HACKERS] [JDBC] Optimize postgres protocol for fixed size arrays

2011-11-23 Thread Merlin Moncure
On Tue, Nov 22, 2011 at 6:52 PM, Tom Lane wrote: > Oliver Jowett writes: >> On 23 November 2011 10:47, Mikko Tiihonen >> wrote: >>> Here is a patch that adds a new flag to the protocol that is set when all >>> elements of the array are of same fixed size. > >> How does a client detect that this

Re: [HACKERS] Permissions checks for range-type support functions

2011-11-23 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Robert Haas's message of mié nov 23 11:01:50 -0300 2011: >> It's always seemed mildly insane to me that we don't distinguish >> between different flavors of "internal". That seems like an accident >> waiting to happen. > Well, before we had INTERNAL, there

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 10:35 AM, Alvaro Herrera wrote: > That's because the owner is "pinned" (i.e. the bootstrap user).  Try > with a different user.  I see new rows with both temp and non-temp > tables. Oh, wow. I had no idea it worked like that. You learn something new every day. -- Rober

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 23 12:28:38 -0300 2011: > Hmm, I'm not seeing any increase in the number of entries in > pg_shdepend when I create either a temporary or permanent table: > > rhaas=# select sum(1) from pg_shdepend; > sum > - >2 > (1 row) > > rhaas=# create

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 10:20 AM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié nov 23 12:15:55 -0300 2011: >> > And it effects shared catalogs only, which are all low traffic anyway. >> >> I think "low traffic" is the key point.  I understand that you're not >> changing the V

Re: [HACKERS] Permissions checks for range-type support functions

2011-11-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 23 11:01:50 -0300 2011: > On Tue, Nov 22, 2011 at 6:38 PM, Tom Lane wrote: > > 2. The ANALYZE option is flat out dangerous, because it allows any > > function with the signature "f(internal) returns bool" to be called as > > though it's a typanalyze

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 23 12:15:55 -0300 2011: > > And it effects shared catalogs only, which are all low traffic anyway. > > I think "low traffic" is the key point. I understand that you're not > changing the VACUUM behavior, but you are making heap_page_prune_opt() > n

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Wed, Nov 23, 2011 at 9:25 AM, Simon Riggs wrote: >> Yeah.  This change would have the disadvantage of disabling HOT >> cleanup for shared catalogs; I'm not sure whether that's a good >> decision. > > No, it disables cleanup when being read. They are still VACUUMed normally. > > Note that non-MV

Re: [HACKERS] Inlining comparators as a performance optimisation

2011-11-23 Thread Simon Riggs
On Fri, Nov 18, 2011 at 2:11 PM, Tom Lane wrote: > Simon Riggs writes: >> On Fri, Nov 18, 2011 at 5:20 AM, Robert Haas wrote: >>> I think that we should really consider doing with this patch what Tom >>> suggested upthread; namely, looking for a mechanism to allow >>> individual datatypes to off

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Simon Riggs
On Wed, Nov 23, 2011 at 2:00 PM, Robert Haas wrote: > On Tue, Nov 22, 2011 at 7:25 PM, Simon Riggs wrote: >> On Tue, Nov 22, 2011 at 11:40 PM, Simon Riggs wrote: I think this is unsafe for shared catalogs. >>> I think so too. Thats why it uses IsMVCCSnapshot() to confirm when it >>> is safe

Re: [HACKERS] Permissions checks for range-type support functions

2011-11-23 Thread Robert Haas
On Tue, Nov 22, 2011 at 6:38 PM, Tom Lane wrote: > 2. The ANALYZE option is flat out dangerous, because it allows any > function with the signature "f(internal) returns bool" to be called as > though it's a typanalyze function.  There are a couple of such functions > in the catalogs already, and e

Re: [HACKERS] Not HOT enough

2011-11-23 Thread Robert Haas
On Tue, Nov 22, 2011 at 7:25 PM, Simon Riggs wrote: > On Tue, Nov 22, 2011 at 11:40 PM, Simon Riggs wrote: >>> I think this is unsafe for shared catalogs. >> I think so too. Thats why it uses IsMVCCSnapshot() to confirm when it >> is safe to do so. > Ah, you mean access to shared catalogs using M

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Florian Weimer
* Alexander Shulgin: > This, in my opinion, is very similar to what we would like to achieve with > the URI syntax, so the above could also be specified using a URI parameter > like this: > > psql -d postgresql://example.net:5433/mydb How would you specifiy a local port/UNIX domain socket? W

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-23 Thread Michael Meskes
> It was proposed a while ago for libpq to support URI syntax for specifying > the connection information: > ... > Now we're going to actually implement this. Do you know that we had this feature (more or less) in libpq for years but it was removed quite a while ago. It should still be there in t