Re: [HACKERS] ExecStoreTuple going into infinite loop

2012-06-06 Thread Merlin Moncure
On Wed, Jun 6, 2012 at 5:36 AM, Atri Sharma atri.j...@gmail.com wrote: Hi all, I am trying to build and store multiple tuples.The code is: ExecClearTuple(slot); /The code for fetching the data from which tuple will be formed../ for(;xy;x++){ tuple =

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-06 Thread Merlin Moncure
On Wed, Jun 6, 2012 at 2:53 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Wed, 6 Jun 2012, Ants Aasma wrote: On Wed, Jun 6, 2012 at 2:27 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: I've quickly tested your lockfree-getbuffer.patch patch with the test case you provided and I barely

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-04 Thread Merlin Moncure
On Mon, Jun 4, 2012 at 9:20 AM, Ants Aasma a...@cybertec.at wrote: On Mon, Jun 4, 2012 at 5:12 PM, Robert Haas robertmh...@gmail.com wrote: Note sure about the rest of this patch, but this part is definitely bogus: +#if !defined(pg_atomic_fetch_and_set) +#define pg_atomic_fetch_and_set(dst,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-04 Thread Merlin Moncure
On Mon, Jun 4, 2012 at 10:17 AM, Merlin Moncure mmonc...@gmail.com wrote: What happens (in the very unlikely, but possible case?) if another backend races to the buffer you've pointed at with 'victim'?  It looks like multiple backends share the clock sweep now, but don't you need to need

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-04 Thread Merlin Moncure
On Mon, Jun 4, 2012 at 10:42 AM, Ants Aasma a...@cybertec.at wrote: On Mon, Jun 4, 2012 at 6:38 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Jun 4, 2012 at 10:17 AM, Merlin Moncure mmonc...@gmail.com wrote: What happens (in the very unlikely, but possible case?) if another backend

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 7:47 AM, Florian Pflug f...@phlo.org wrote: On May31, 2012, at 20:50 , Robert Haas wrote: Suppose we introduce two new buffer flags, BUF_NAILED and BUF_NAIL_REMOVAL.  When we detect excessive contention on the buffer header spinlock, we set BUF_NAILED.  Once we do that,

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 8:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: A potential issue with this line of thinking is that your pin delay queue could get highly pressured by outer portions of the query (as in the OP's case)  that will get little

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 3:40 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 1, 2012 at 3:16 PM, Florian Pflug f...@phlo.org wrote: Ok, now you've lost me. If the read() blocks, how on earth can a few additional pins/unpins ever account for any meaningful execution time? It seems to

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 10:23 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Thanks.  How did you generate this perf report?  It's cool, because I haven't figured out how to make perf generate a report that is easily email-able, and it seems you have.

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 11:54 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Oh, ho.  So from this we can see that the problem is that we're getting huge amounts of spinlock contention when pinning and unpinning index pages. It would be nice to have

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-31 Thread Merlin Moncure
On Thu, May 31, 2012 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 31, 2012 at 2:03 PM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, May 31, 2012 at 11:54 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 31 May 2012, Robert Haas wrote: Oh, ho.  So from this we

Re: [HACKERS] hash index concurrency

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 3:49 AM, Simon Riggs si...@2ndquadrant.com wrote: On 30 May 2012 04:54, Robert Haas robertmh...@gmail.com wrote: This was a hobby horse of mine a couple of years ago, but I never got much traction.  The main question I have is, what do we even want hash indexes to be?  

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-30 Thread Merlin Moncure
On Sun, May 27, 2012 at 1:45 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Hi, I did another test using the same data and the same code, which I've provided before and the performance of the single thread seems to be degrading quadratically with the number of threads. Here are the

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 10:42 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Here is the actual explain analyze of the query on the smaller dataset which I have been using for the recent testing. test=# explain analyze create table _tmp0 as select * from  ( select *,        (select

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 12:11 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, May 30, 2012 at 12:58 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Here is the one to one comparison of the 'bogged' **  QUERY PLAN

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 1:45 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Wed, 30 May 2012, Merlin Moncure wrote: Hm, why aren't we getting a IOS?  Just for kicks (assuming this is test data), can we drop the index on just transitid, leaving the index on transitid, healpixid

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 3:15 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, May 30, 2012 at 11:45 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Wed, 30 May 2012, Merlin Moncure wrote: Hm, why aren't we getting a IOS?  Just for kicks (assuming this is test data), can we drop

Re: [HACKERS] Early hint bit setting

2012-05-30 Thread Merlin Moncure
On Wed, May 30, 2012 at 4:42 PM, Ants Aasma a...@cybertec.at wrote: I was thinking about what is the earliest time where we could set hint bits. This would be just after the commit has been made visible. When the transaction completes and commit confirmation is sent to the client the backend

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-25 Thread Merlin Moncure
On Thu, May 24, 2012 at 6:26 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 24 May 2012, Jeff Janes wrote: Add #define LWLOCK_STATS near the top of: src/backend/storage/lmgr/lwlock.c and recompile and run a reduced-size workload.  When the processes exits, they will dump a lot of

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-25 Thread Merlin Moncure
On Fri, May 25, 2012 at 8:06 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, May 24, 2012 at 6:26 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 24 May 2012, Jeff Janes wrote: Add #define LWLOCK_STATS near the top of: src/backend/storage/lmgr/lwlock.c and recompile and run

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-25 Thread Merlin Moncure
On Fri, May 25, 2012 at 10:22 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: Hm, what if BufTableHashPartition() was pseudo randomized so that different backends would not get the same buffer partition for a particular tag? Huh?  You have to make sure

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-25 Thread Merlin Moncure
On Fri, May 25, 2012 at 11:17 AM, Stephen Frost sfr...@snowman.net wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: Right -- duh.  Well, hm.  Is this worth fixing?  ISTM there's a bit of 'optimizing for pgbench-itis' in the buffer partitions -- they seem optimized to lever the mostly random

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-25 Thread Merlin Moncure
On Fri, May 25, 2012 at 11:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Fri, May 25, 2012 at 11:17 AM, Stephen Frost sfr...@snowman.net wrote: Didn't we implement a system whereby this is exactly what we intend to happen on the read side

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 8:24 AM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: Hi, I've been running some tests on pg 9.2beta1 and in particular a set of queries like create table _tmp0 as select * from (        select *, (select healpixid from idt_match as m where                            

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 1:43 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 24, 2012 at 2:19 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: On Thu, 24 May 2012, Robert Haas wrote: On Thu, May 24, 2012 at 1:42 PM, Sergey Koposov kopo...@ast.cam.ac.uk wrote: I guess there is nothing

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-05-24 Thread Merlin Moncure
On Thu, May 24, 2012 at 3:35 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 24, 2012 at 3:46 PM, Merlin Moncure mmonc...@gmail.com wrote: hm, looking at the code some more, it looks like the whole point of the strategy system is to do that.  ISTM bulk insert type queries would

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-23 Thread Merlin Moncure
On Tue, May 22, 2012 at 11:33 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, May 21, 2012 at 2:29 PM, Merlin Moncure mmonc...@gmail.com wrote: See here: http://www.devheads.net/database/postgresql/performance/index-all-necessary-columns-postgres-vs-mssql.htm for a 'in the wild' gripe

Re: [HACKERS] heap metapages

2012-05-21 Thread Merlin Moncure
On Mon, May 21, 2012 at 12:56 PM, Robert Haas robertmh...@gmail.com wrote: At dinner on Friday night at PGCon, the end of the table that included Tom Lane, Stephen Frost, and myself got to talking about the idea of including some kind of metapage in every relation, including heap relations.  

Re: [HACKERS] Why is indexonlyscan so darned slow?

2012-05-21 Thread Merlin Moncure
On Mon, May 21, 2012 at 4:17 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, May 21, 2012 at 1:42 PM, Josh Berkus j...@agliodbs.com wrote: Earlier you said that this should be an ideal setup for IOS.  But it isn't really--the ideal set up is one in which the alternative to an IOS is a

Re: [HACKERS] Draft release notes complete

2012-05-14 Thread Merlin Moncure
On Wed, May 9, 2012 at 10:11 PM, Bruce Momjian br...@momjian.us wrote: I have completed my draft of the 9.2 release notes, and committed it to git. The beta release announcement is on postgresql.org with a direct link to the release notes. The notes lead off with: NARRATIVE HERE. Major

Re: [HACKERS] Temporary tables under hot standby

2012-05-07 Thread Merlin Moncure
On Tue, Apr 24, 2012 at 10:55 PM, Noah Misch n...@leadboat.com wrote: A key barrier to migrations from trigger-based replication to WAL-based replication is the lack of temporary tables under hot standby.  I'd like to close that gap; the changes needed will also reduce the master-side cost of

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 4:11 AM, Simon Riggs si...@2ndquadrant.com wrote: which seems to me to be actually harder than just rewriting as derived table and isn't an option on Microstrategy etc, hence my observation that GTTs don't help HS much. What I would like to see, one day, is for temp

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) Yeah -- the syntax isn't great, but IMO it's more generally usable than what you're proposing because it's a scalar returning

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:01 AM, Andrew Dunstan and...@dunslane.net wrote: On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncuremmonc...@gmail.com: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehulepavel.steh...@gmail.com  wrote: Hello (1 row) This works the same indeed, just

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:44 AM, Andrew Dunstan and...@dunslane.net wrote: Why would you always need FROM? that was coming from Hannu's original example: insert into test2 select * from json_to_record(jrec json) as (like test2); how do you work it so you can call: select json_to_record(jrec

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 10:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe some keyword can help to us. What do you think about new operator TYPE that can returns regtype value and can be used together with polymorphic functions. Doesn't have any more attraction for me than the proposed LIKE

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 11:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that a saner implementation would involve splitting the userlock lockmethod into two, one transactional and one not.  That gets rid of the when-to-release kluges, but instead we have to think of a way for

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-02 Thread Merlin Moncure
On Wed, May 2, 2012 at 12:06 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2012-05-02 at 13:40 +0200, Hannu Krosing wrote: How hard would it be to add support for LIKE syntax, similar to table def in field list declaration for generic record functions What I'dd like to be able to do is

Re: [HACKERS] JSON in 9.2 - Could we have just one to_json() function instead of two separate versions ?

2012-05-01 Thread Merlin Moncure
On Tue, May 1, 2012 at 7:02 AM, Hannu Krosing ha...@2ndquadrant.com wrote: Hi hackers After playing around with array_to_json() and row_to_json() functions a bit it I have a question - why do we even have 2 variants *_to_json() Collapsing array_to_json() and row_to_json() into just to_json()

Re: [HACKERS] JSON in 9.2 - Could we have just one to_json() function instead of two separate versions ?

2012-05-01 Thread Merlin Moncure
On Tue, May 1, 2012 at 10:49 AM, Joey Adams joeyadams3.14...@gmail.com wrote: On Tue, May 1, 2012 at 8:02 AM, Hannu Krosing ha...@2ndquadrant.com wrote: Hi hackers After playing around with array_to_json() and row_to_json() functions a bit it I have a question - why do we even have 2 variants

Re: [HACKERS] JSON in 9.2 - Could we have just one to_json() function instead of two separate versions ?

2012-05-01 Thread Merlin Moncure
On Tue, May 1, 2012 at 11:22 AM, Andrew Dunstan and...@dunslane.net wrote: On Tue, May 1, 2012 at 9:05 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, May 1, 2012 at 10:49 AM, Joey Adams joeyadams3.14...@gmail.com wrote: On Tue, May 1, 2012 at 8:02 AM, Hannu Krosing ha

Re: [HACKERS] Future In-Core Replication

2012-04-30 Thread Merlin Moncure
On Mon, Apr 30, 2012 at 12:38 PM, Bruce Momjian br...@momjian.us wrote: For example, you said that MM replication alone is not a solution for large data or the general case.  Why is that?  Is the goal of your work really to do logical replciation, which allows for major version upgrades?  Is

Re: [HACKERS] Future In-Core Replication

2012-04-30 Thread Merlin Moncure
On Mon, Apr 30, 2012 at 2:38 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Apr 30, 2012 at 2:33 PM, Merlin Moncure mmonc...@gmail.com wrote: On Mon, Apr 30, 2012 at 12:38 PM, Bruce Momjian br...@momjian.us wrote: For example, you said that MM replication alone is not a solution

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-27 Thread Merlin Moncure
On Fri, Apr 27, 2012 at 1:57 PM, Robert Haas robertmh...@gmail.com wrote: I think there is no point at all in having a discussion about this unless we can first agree that the overwhelming majority of people who have commented on this issue on this list are unhappy with the current default

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-26 Thread Merlin Moncure
On Wed, Apr 25, 2012 at 9:28 AM, Merlin Moncure mmonc...@gmail.com wrote: That patch actually has more than one optimization in it, I think, but the basic idea is that if we could figure out a way to set HEAP_XMIN_COMMITTED when loading data into a table created or truncated within the same

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-25 Thread Merlin Moncure
On Tue, Apr 24, 2012 at 3:40 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Apr 21, 2012 at 7:08 PM, Greg Stark st...@mit.edu wrote: The earlier consensus was to move all the hint bits to a dedicated area and exclude them from the checksum. I think double-write buffers seem to have

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Merlin Moncure
On Wed, Apr 25, 2012 at 11:53 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 25, 2012 at 12:30 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Apr 25, 2012 at 5:19 PM, Robert Haas robertmh...@gmail.com wrote: Oh, we're talking about different things, and I'm slightly confused.

Re: [HACKERS] Desperately need mentors for GSOC FDW projects

2012-04-18 Thread Merlin Moncure
On Wed, Apr 18, 2012 at 12:15 PM, Josh Berkus j...@agliodbs.com wrote: All, We have 2 fairly promising projects for FDW work for this Google Summer of code.  One is for a Firebird FDW, and the more promising one is for a document collection FDW from a previous successful GSOC student. The

Re: [HACKERS] JSON for PG 9.2

2012-04-16 Thread Merlin Moncure
On Mon, Apr 16, 2012 at 9:10 AM, Andrew Dunstan and...@dunslane.net wrote: On 04/16/2012 09:34 AM, Hannu Krosing wrote: based on Abhijit's feeling and some discussion offline, the consensus seems to be to remove query_to_json. The only comment I have here is that query_to_json could have

Re: [HACKERS] JSON for PG 9.2

2012-04-16 Thread Merlin Moncure
On Mon, Apr 16, 2012 at 11:19 AM, Hannu Krosing ha...@2ndquadrant.com wrote: If doing something in 9.3 then what I would like is some way to express multiple queries. Basically a variant of query_to_json(query text[]) where queries would be evaluated in order and then all the results

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-11 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 10:41 PM, Atri Sharma atri.j...@gmail.com wrote: Well. maybe I spoke too soon...JNI is probably the best route.  Since SPI is off the table, all we're really pulling in from pl/java is the (non-trivial) proper installation of a jvm into a postgres process. pl/java is

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-10 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 8:42 AM, Andrew Dunstan and...@dunslane.net wrote: I am considering two paths for doing this: The first one takes the help of the SPI(Server Programming Interface) and the second one directly connects through Pl/Java and JNI(Java Native Interface). I'd say forget SPI

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-10 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 9:15 AM, Andrew Dunstan and...@dunslane.net wrote: On 04/10/2012 09:48 AM, Merlin Moncure wrote: On Tue, Apr 10, 2012 at 8:42 AM, Andrew Dunstanand...@dunslane.net  wrote: I am considering two paths for doing this: The first one takes the help of the SPI(Server

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-10 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 9:47 AM, Andrew Dunstan and...@dunslane.net wrote: I don't understand what the heck you're talking about, TBH. From a user perspective there is nothing to work out. It will look like any other FDW. yes, that is correct. The implementor of the FDW handler will have to

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-10 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 10:36 AM, Tom Lane t...@sss.pgh.pa.us wrote: Atri Sharma atri.j...@gmail.com writes: On Tue, Apr 10, 2012 at 8:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hm?  SPI doesn't know anything about Java either. We plan to call SQL through SPI from the FDW,which in turn would

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-10 Thread Merlin Moncure
On Tue, Apr 10, 2012 at 11:07 AM, Merlin Moncure mmonc...@gmail.com wrote:  I agree that JNI isn't required -- we're going to have to study the pl/java system a bit to determine the best way to hook in.  This could end up getting us into the 'biting of more than can chew' territory admittedly

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-09 Thread Merlin Moncure
On Sun, Apr 8, 2012 at 8:56 AM, Dave Cramer p...@fastcrypt.com wrote: Hi Atri, Is there some JDBC API that supports this in newer versions of the API ? Didn't parse that question. My understanding is that the only JDBC features needed are what's already there, to make connections to databases

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-09 Thread Merlin Moncure
On Mon, Apr 9, 2012 at 10:47 AM, Dave Cramer p...@fastcrypt.com wrote: How will the user access this? Will it be a normal query through the existing API ? Will it be a private postgresql API ? How will they set it up ? It appears complicated as you have to setup PL/Java as well Yeah -- it

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-09 Thread Merlin Moncure
On Mon, Apr 9, 2012 at 11:14 AM, Dave Cramer p...@fastcrypt.com wrote: So I'm confused, once they link a file to an FDW can't you just read it with an normal select ? What additional functionality will this provide ? Dave The basic objective is to expose the JDBC to postgres for grabbing

Re: [HACKERS] [JDBC] Regarding GSoc Application

2012-04-09 Thread Merlin Moncure
On Mon, Apr 9, 2012 at 12:25 PM, Atri Sharma atri.j...@gmail.com wrote: On Mon, Apr 9, 2012 at 10:15 PM, Andrew Dunstan and...@dunslane.net wrote: On 04/09/2012 12:14 PM, Dave Cramer wrote: So I'm confused, once they link a file to an FDW can't you just read it with an normal select ? What

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-04-09 Thread Merlin Moncure
On Mon, Apr 9, 2012 at 1:14 PM, Bruce Momjian br...@momjian.us wrote: Well, I assume they reimplemented libpq so that java would not rely on a platform-specific library like libpq. yes, that is correct. jdbc for postgres is a complete implementation of the client side protocol. this has some

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-03-30 Thread Merlin Moncure
On Thu, Mar 29, 2012 at 10:04 AM, Andrew Dunstan and...@dunslane.net wrote: 1. I've been in discussion with some people about adding simple JSON extract functions. We already have some (i.e. xpath()) for XML. I've built a couple of applications that push data in and out of xml via manual

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-23 Thread Merlin Moncure
On Wed, Mar 21, 2012 at 10:51 PM, Atri Sharma atri.j...@gmail.com wrote: Please let me know how to proceed further. Waiting for your reply, sure -- let's take this discussion off line. send me a private mail and we'll discuss if/how we can get this off the ground. merlin -- Sent via

Re: [HACKERS] query cache

2012-03-23 Thread Merlin Moncure
On Fri, Mar 23, 2012 at 12:03 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 23, 2012 at 12:29 PM, Greg Stark st...@mit.edu wrote: On Fri, Mar 23, 2012 at 3:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: The complication, opportunities for bugs, and general slowdown associated with that

Re: [HACKERS] Apology to the community

2012-03-23 Thread Merlin Moncure
On Fri, Mar 23, 2012 at 1:43 PM, Joshua D. Drake j...@commandprompt.com wrote: Hello, It has been brought to my attention a few times over the last year that I have been over the top in my presentation of myself and have in fact alienated and offended many of the community. To be honest I am

Re: [HACKERS] HOT updates REDIRECT line pointers

2012-03-22 Thread Merlin Moncure
On Wed, Mar 21, 2012 at 8:28 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 21, 2012 at 9:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: It strikes me that it likely wouldn't be any worse than, oh, say, flipping the default value of standard_conforming_strings, Really?  It's taking away

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-20 Thread Merlin Moncure
On Tue, Mar 20, 2012 at 6:58 AM, Atri Sharma atri.j...@gmail.com wrote: -Original Message- From: Merlin Moncure [mailto:mmonc...@gmail.com] Sent: 20 March 2012 03:15 To: Heikki Linnakangas Cc: Alvaro Herrera; Atri Sharma; Daniel Farina; Andrew Dunstan; Dave Page; Pg Hackers Subject

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-20 Thread Merlin Moncure
On Tue, Mar 20, 2012 at 11:57 AM, Atri Sharma atri.j...@gmail.com wrote: -Original Message- From: Merlin Moncure [mailto:mmonc...@gmail.com] Sent: 20 March 2012 20:52 To: Atri Sharma Cc: Heikki Linnakangas; Alvaro Herrera; Daniel Farina; Andrew Dunstan; Dave Page; Pg Hackers

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-20 Thread Merlin Moncure
On Tue, Mar 20, 2012 at 12:34 PM, Atri Sharma atri.j...@gmail.com wrote: -Original Message- From: Merlin Moncure [mailto:mmonc...@gmail.com] Sent: 20 March 2012 22:50 To: Atri Sharma Cc: Heikki Linnakangas; Alvaro Herrera; Daniel Farina; Andrew Dunstan; Dave Page; Pg Hackers

Re: [HACKERS] Regarding column reordering project for GSoc 2012

2012-03-19 Thread Merlin Moncure
On Mon, Mar 19, 2012 at 2:49 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 19.03.2012 21:29, Alvaro Herrera wrote: Excerpts from Atri Sharma's message of lun mar 19 16:20:09 -0300 2012: I was just going through PGfoundry and I think I will be able to work on the

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Merlin Moncure
On Wed, Mar 14, 2012 at 1:06 PM, Daniel Farina dan...@heroku.com wrote: For 9.3 at a minimum. The topic of LZO became mired in doubts about: * Potential Patents * The author's intention for the implementation to be GPL Since then, Google released Snappy, also an LZ77-class implementation,

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Merlin Moncure
On Wed, Mar 14, 2012 at 3:43 PM, Andrew Dunstan and...@dunslane.net wrote: On 03/14/2012 04:10 PM, Merlin Moncure wrote: there are plenty of on gpl lz based libraries out there (for example: http://www.fastlz.org/) and always have been.  they are all much faster than zlib.  the main issue

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-09 Thread Merlin Moncure
On Fri, Mar 9, 2012 at 9:36 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2012/3/6 Alvaro Herrera alvhe...@commandprompt.com: It seems to me that the only thing that needs core support is the ability to start up the daemon when postmaster is ready to accept queries, and shut the daemon down when

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Merlin Moncure
On Wed, Mar 7, 2012 at 2:15 AM, Simon Riggs si...@2ndquadrant.com wrote: We talked about this at last year's Dev meeting. And we got sidetracked into what we really want is stored procedures. Maybe we want that, but its a completely separate thing. Please lets not get distracted from a very

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-07 Thread Merlin Moncure
On Wed, Mar 7, 2012 at 2:14 PM, Simon Riggs si...@2ndquadrant.com wrote: The stored procedure route sounds attractive but its a long way off and doesn't address all of the states needs people have voiced. I'm not against doing both, I just want to do the quickest and easiest. sure, I get that,

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 9:37 AM, Robert Haas robertmh...@gmail.com wrote: But having said that, it's not apparent to me why such a thing would need to live inside the database at all.  It's very easy to visualize a task scheduler that runs as a client and requires nothing new from the core

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 3:44 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Josh Berkus j...@agliodbs.com writes: Activity and discretion beyond that could be defined in PL code, including run/don't run conditions, activities, and dependancies.  The only thing Postgres doesn't currently

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Merlin Moncure
On Tue, Mar 6, 2012 at 4:01 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Why do we need a ticker?  Just fetch the time of the task closest in the future, and sleep till that time or a notify arrives (meaning schedule change). Because that can't be done in userland (at least, not

Re: [HACKERS] Let's drop V2 protocol

2012-02-24 Thread Merlin Moncure
On Fri, Feb 24, 2012 at 7:52 AM, Marko Kreen mark...@gmail.com wrote: On Fri, Feb 24, 2012 at 02:11:45PM +0200, Marko Kreen wrote: On Fri, Feb 24, 2012 at 07:53:14PM +0900, Kyotaro HORIGUCHI wrote: - I have no idea how to do test for protocol 2... I have a urge to test with rm

Re: [HACKERS] Let's drop V2 protocol

2012-02-24 Thread Merlin Moncure
On Fri, Feb 24, 2012 at 10:46 AM, Tom Lane t...@sss.pgh.pa.us wrote: I believe it's still somewhat common among JDBC users to force V2-protocol connections as a workaround for over-eager prepared statement planning.  Although I think the issue they're trying to dodge will be fixed as of 9.2,

Re: [HACKERS] leakproof

2012-02-21 Thread Merlin Moncure
On Sun, Feb 19, 2012 at 8:20 PM, Don Baccus dhog...@pacifier.com wrote: On Feb 19, 2012, at 5:42 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Having now spent far too much time in bed with that patch, I'm feeling like the concept that we are really looking for there is what

Re: [HACKERS] SKIP LOCKED DATA

2012-02-06 Thread Merlin Moncure
On Sat, Feb 4, 2012 at 5:38 AM, Thomas Munro mu...@ip9.org wrote: On 16 January 2012 21:30, Josh Berkus j...@agliodbs.com wrote: Useful, yes.  Harder than it looks, probably.  I tried to mock up a version of this years ago for a project where I needed it, and ran into all kinds of race

Re: [HACKERS] JSON for PG 9.2

2012-02-01 Thread Merlin Moncure
On Tue, Jan 31, 2012 at 11:46 PM, Andrew Dunstan and...@dunslane.net wrote: The array(select...) locution turns out to have less flexibility than the array_agg(record-ref) locution. Less flexible maybe, but it can cleaner for exactly the type of queries that will tend to come up in exactly the

Re: [HACKERS] JSON for PG 9.2

2012-01-31 Thread Merlin Moncure
On Mon, Jan 30, 2012 at 9:37 AM, Andrew Dunstan and...@dunslane.net wrote: On 01/30/2012 09:54 AM, Abhijit Menon-Sen wrote: At 2012-01-27 09:47:05 +0530, a...@toroid.org wrote: I've started reviewing this patch, but it'll take me a bit longer to go through json.c properly. OK, I finished

Re: [HACKERS] JSON for PG 9.2

2012-01-31 Thread Merlin Moncure
On Tue, Jan 31, 2012 at 12:15 PM, Josh Berkus j...@agliodbs.com wrote: Andrew, based on Abhijit's feeling and some discussion offline, the consensus seems to be to remove query_to_json. If we do that, what would getting complete query results back from a query look like?  It's important to

Re: [HACKERS] JSON for PG 9.2

2012-01-31 Thread Merlin Moncure
On Tue, Jan 31, 2012 at 12:48 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/31/2012 01:32 PM, Merlin Moncure wrote: On Tue, Jan 31, 2012 at 12:15 PM, Josh Berkusj...@agliodbs.com  wrote: Andrew, based on Abhijit's feeling and some discussion offline, the consensus seems

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-27 Thread Merlin Moncure
On Fri, Jan 27, 2012 at 2:57 AM, Kyotaro HORIGUCHI horiguchi.kyot...@oss.ntt.co.jp wrote: Hello, This is a new version of the patch formerly known as 'alternative storage for libpq'. I took a quick look at the patch and the docs. Looks good and agree with rationale and implementation. I see

Re: [HACKERS] CLOG contention, part 2

2012-01-27 Thread Merlin Moncure
On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes jeff.ja...@gmail.com wrote: Also, I think the general approach is wrong.  The only reason to have these pages in shared memory is that we can control access to them to prevent write/write and read/write corruption.  Since these pages are never

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: This is just to confirm that the patch applies and builds and works fine (though of course it does take a long time… pity there doesn't seem to be any easy way to add progress indication like -i has). On any non server

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 10:59 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 26, 2012 at 11:41 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: This is just to confirm that the patch applies and builds and works

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Merlin Moncure
On Wed, Jan 25, 2012 at 11:24 AM, Marko Kreen mark...@gmail.com wrote: I specifically want to avoid any sort of per-connection negotation, except the max format version supported, because it will mess up multiplexed usage of single connection. Then they need to either disabled advanced formats

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Merlin Moncure
On Wed, Jan 25, 2012 at 1:24 PM, Marko Kreen mark...@gmail.com wrote: On Wed, Jan 25, 2012 at 12:54:00PM -0600, Merlin Moncure wrote: On Wed, Jan 25, 2012 at 11:24 AM, Marko Kreen mark...@gmail.com wrote: I specifically want to avoid any sort of per-connection negotation, except the max

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-25 Thread Merlin Moncure
On Wed, Jan 25, 2012 at 2:29 PM, Marko Kreen mark...@gmail.com wrote: well, I see the following cases: 1) Vserver Vapplication: server downgrades wire formats to applications version 2) Vapplication Vlibpq Vserver: since the application is reading/writing formats the server can't

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Merlin Moncure
On Tue, Jan 24, 2012 at 8:26 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jan 23, 2012 at 5:49 PM, Merlin Moncure mmonc...@gmail.com wrote: I'm not sure that you're getting anything with that user facing complexity.  The only realistic case I can see for explicit control of wire

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-24 Thread Merlin Moncure
On Tue, Jan 24, 2012 at 11:55 AM, Robert Haas robertmh...@gmail.com wrote: I do wonder whether we are making a mountain out of a mole-hill here, though.  If I properly understand the proposal on the table, which it's possible that I don't, but if I do, the new format is self-identifying: when

Re: [HACKERS] Multithread Query Planner

2012-01-23 Thread Merlin Moncure
On Fri, Jan 13, 2012 at 2:29 PM, Christopher Browne cbbro...@gmail.com wrote: On Fri, Jan 13, 2012 at 3:14 PM, Frederico zepf...@gmail.com wrote: Hi folks. Is there any restriction in create and start threads inside Postgres? I'm trying to develop a multithread planner, and some times is

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 2:00 PM, A.M. age...@themactionfaction.com wrote: One simple way clients could detect the binary encoding at startup would be to pass known test parameters and match against the returned values. If the client cannot match the response, then it should choose the text

Re: [HACKERS] Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 3:06 PM, Robert Haas robertmh...@gmail.com wrote: Not being responsible from the maintenance of any PostgreSQL drivers whatsoever, I don't have a strong feeling about which of these is the case, and I'd like us to hear from the people who do. I'm just gonna come right

Re: [HACKERS] JSON for PG 9.2

2012-01-23 Thread Merlin Moncure
On Sun, Jan 15, 2012 at 10:08 AM, Andrew Dunstan and...@dunslane.net wrote: Here's an update that adds row_to_json, plus a bit more cleanup. why not call all these functions 'to_json' and overload them? merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: GUC_REPORT for protocol tunables was: Re: [HACKERS] Optimize binary serialization format of arrays with fixed size elements

2012-01-23 Thread Merlin Moncure
On Mon, Jan 23, 2012 at 4:12 PM, A.M. age...@themactionfaction.com wrote: On Jan 23, 2012, at 4:45 PM, Merlin Moncure wrote: Prefer the version.  But why send this over and over with each bind? Wouldn't you negotiate that when connecting? Most likely, optionally, doing as much as you can from

<    4   5   6   7   8   9   10   11   12   13   >