Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 17 October 2012 11:21, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Simon Riggs si...@2ndquadrant.com writes: On 16 October 2012 15:15, Tom Lane t...@sss.pgh.pa.us wrote: What you really want is something vaguely like nextval but applied to a distinct type of object. That is, I think we

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 17 October 2012 10:19, Markus Wanner mar...@bluegap.ch wrote: On 10/17/2012 10:34 AM, Simon Riggs wrote: IMHO an API is required for give me the next allocation of numbers, essentially a bulk equivalent of nextval(). Agreed. That pretty exactly matches what I described (and what's

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 16 October 2012 18:29, Tom Lane t...@sss.pgh.pa.us wrote: Or maybe better, invent a level of indirection like a sequence access method (comparable to index access methods) that provides a compatible set of substitute functions for sequence operations. If you want to override nextval() for

Re: [HACKERS] Global Sequences

2012-10-18 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: Not sure how it is cleaner when we have to have trigger stuff hanging around to make one object pretend to be another. That also creates a chain of dependency which puts this into the future, rather than now. Yes, that part isn't cleaner at all. The

Re: [HACKERS] Global Sequences

2012-10-18 Thread Robert Haas
On Tue, Oct 16, 2012 at 1:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: In particular, the reason proposing a hook first seems backwards is that if we have a catalog-level representation that some sequences are local and others not, we should be using that to drive the determination of whether to

Re: [HACKERS] Global Sequences

2012-10-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Oct 16, 2012 at 1:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Or maybe better, invent a level of indirection like a sequence access method (comparable to index access methods) that provides a compatible set of substitute functions for sequence

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 18 October 2012 16:08, Robert Haas robertmh...@gmail.com wrote: To make it even better, add some generic options that can be passed through to the underlying handler. Agreed Or maybe better, invent a level of indirection like a sequence access method (comparable to index access methods)

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 18 October 2012 16:15, Tom Lane t...@sss.pgh.pa.us wrote: But I'd want to see a pretty bulletproof argument why overriding *only* nextval is sufficient (and always will be) before accepting a hook for just nextval. If we build an equivalent amount of functionality piecemeal it's going to

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Simon, On 10/16/2012 02:36 PM, Simon Riggs wrote: Where else would you put the hook? The hook's location as described won't change whether you decide you want 1, 2 or 3. You assume we want an API that supports all three options. In that case, yes, the hooks need to be very general. Given that

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Tom, On 10/16/2012 06:15 PM, Tom Lane wrote: I challenge you to find anything in the SQL standard that suggests that sequences have any nonlocal behavior. If anything, what you propose violates the standard, it doesn't make us follow it more closely. If you look at a distributed database as

Re: [HACKERS] Global Sequences

2012-10-17 Thread Simon Riggs
On 17 October 2012 09:10, Markus Wanner mar...@bluegap.ch wrote: Simon, On 10/16/2012 02:36 PM, Simon Riggs wrote: Where else would you put the hook? The hook's location as described won't change whether you decide you want 1, 2 or 3. You assume we want an API that supports all three

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Simon, On 10/17/2012 10:34 AM, Simon Riggs wrote: IMHO an API is required for give me the next allocation of numbers, essentially a bulk equivalent of nextval(). Agreed. That pretty exactly matches what I described (and what's implemented in Postgres-R). The API then only needs to be called

Re: [HACKERS] Global Sequences

2012-10-17 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: On 16 October 2012 15:15, Tom Lane t...@sss.pgh.pa.us wrote: What you really want is something vaguely like nextval but applied to a distinct type of object. That is, I think we first need a different kind of object called a global sequence with its

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 03:03, Christopher Browne cbbro...@gmail.com wrote: There's a necessary trade-off; you can either have it globally *strongly* ordered, and, if so, you'll have to pay a hefty coordination price, or you can have the cheaper answer of a weakly ordered sequence. The latter

Re: [HACKERS] Global Sequences

2012-10-16 Thread Yeb Havinga
On 2012-10-15 23:33, Simon Riggs wrote: So, proposal is to allow nextval() allocation to access a plugin, rather than simply write a WAL record and increment. If the plugin is loaded all sequences call it (not OIDs). +1. It is currently impossible to alter nextvals behaviour, without making

Re: [HACKERS] Global Sequences

2012-10-16 Thread Markus Wanner
On 10/16/2012 12:47 AM, Josh Berkus wrote: I'd also love to hear from the PostgresXC folks on whether this solution works for them. Postgres-R too. In Postgres-R, option 3) is implemented. Though, by default sequences work just like on a single machine, giving you monotonically increasing

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 13:26, Markus Wanner mar...@bluegap.ch wrote: Why does a Global Sequences API necessarily hook at the nextval() and setval() level? That sounds like it yields an awkward amount of duplicate work. Reading this thread, so far it looks like we agree that option 3) is the most

Re: [HACKERS] Global Sequences

2012-10-16 Thread Peter Eisentraut
On 10/15/12 5:33 PM, Simon Riggs wrote: There are a few options 1) Manual separation of the value space, so that N1 has 50% of possible values and N2 has 50%. That has problems when we reconfigure the cluster, and requires complex manual reallocation of values. So it starts good but ends

Re: [HACKERS] Global Sequences

2012-10-16 Thread Andrew Dunstan
On 10/16/2012 08:54 AM, Peter Eisentraut wrote: Option 4 is of course to use UUIDs. Yeah, I was wondering what this would really solve that using UUIDs wouldn't solve. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Global Sequences

2012-10-16 Thread Andres Freund
On Tuesday, October 16, 2012 02:58:11 PM Andrew Dunstan wrote: On 10/16/2012 08:54 AM, Peter Eisentraut wrote: Option 4 is of course to use UUIDs. Yeah, I was wondering what this would really solve that using UUIDs wouldn't solve. Large indexes over random values perform notably worse than

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 13:54, Peter Eisentraut pete...@gmx.net wrote: On 10/15/12 5:33 PM, Simon Riggs wrote: There are a few options 1) Manual separation of the value space, so that N1 has 50% of possible values and N2 has 50%. That has problems when we reconfigure the cluster, and requires

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: So, proposal is to allow nextval() allocation to access a plugin, rather than simply write a WAL record and increment. If the plugin is loaded all sequences call it (not OIDs). I think this is a fundamentally wrong way to go about doing what you want

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 15:15, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: So, proposal is to allow nextval() allocation to access a plugin, rather than simply write a WAL record and increment. If the plugin is loaded all sequences call it (not OIDs). I think this

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 16 October 2012 15:15, Tom Lane t...@sss.pgh.pa.us wrote: I think this is a fundamentally wrong way to go about doing what you want to do. It presumes that DDL-level manipulation of global sequences is exactly like local sequences; an assumption

Re: [HACKERS] Global Sequences

2012-10-16 Thread Peter Eisentraut
On 10/16/12 9:20 AM, Simon Riggs wrote: I've proposed a plugin for the allocation only. So the allocation looks like anything you want. Are you planning to provide a reference implementation of some kind? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Global Sequences

2012-10-16 Thread Daniel Farina
On Tue, Oct 16, 2012 at 5:54 AM, Peter Eisentraut pete...@gmx.net wrote: On 10/15/12 5:33 PM, Simon Riggs wrote: There are a few options 1) Manual separation of the value space, so that N1 has 50% of possible values and N2 has 50%. That has problems when we reconfigure the cluster, and

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 17:15, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 16 October 2012 15:15, Tom Lane t...@sss.pgh.pa.us wrote: I think this is a fundamentally wrong way to go about doing what you want to do. It presumes that DDL-level manipulation of

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 17:17, Peter Eisentraut pete...@gmx.net wrote: On 10/16/12 9:20 AM, Simon Riggs wrote: I've proposed a plugin for the allocation only. So the allocation looks like anything you want. Are you planning to provide a reference implementation of some kind? I'll provide hooks

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 16 October 2012 17:15, Tom Lane t...@sss.pgh.pa.us wrote: So I fully expect that we're going to need something different from bog-standard CREATE SEQUENCE. There's no point in that at all, as explained. It's sequences that need to work. We can

Re: [HACKERS] Global Sequences

2012-10-15 Thread Josh Berkus
On 10/15/12 2:33 PM, Simon Riggs wrote: 3) Lazy allocation from the value space. When a node is close to running out of values, it requests a new allocation and coordinates with all nodes to confirm the new allocation is good. (3) is similar to the way values are allocated currently, so the

Re: [HACKERS] Global Sequences

2012-10-15 Thread Daniel Farina
On Mon, Oct 15, 2012 at 2:33 PM, Simon Riggs si...@2ndquadrant.com wrote: Sequences, as defined by SQL Standard, provide a series of unique values. The current implementation on PostgreSQL isolates the generation mechanism to only a single node, as is common on many RDBMS. For sharded or

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: I'd also love to hear from the PostgresXC folks on whether this solution works for them. Postgres-R too. If it works for all three of those tools, it's liable to work for any potential new tool. AIUI, PG-XC and PG-R need an order, so they both

Re: [HACKERS] Global Sequences

2012-10-15 Thread Josh Berkus
Stephen, AIUI, PG-XC and PG-R need an order, so they both use an independent system (eg, the PG-XC GTM) to provide that ordering. You're thinking of XIDs. This is a proposal for user-defined sequences. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: AIUI, PG-XC and PG-R need an order, so they both use an independent system (eg, the PG-XC GTM) to provide that ordering. You're thinking of XIDs. This is a proposal for user-defined sequences. Right, I got that it's a proposal for user sequences.

Re: [HACKERS] Global Sequences

2012-10-15 Thread Michael Paquier
On Tue, Oct 16, 2012 at 10:30 AM, Josh Berkus j...@agliodbs.com wrote: Stephen, AIUI, PG-XC and PG-R need an order, so they both use an independent system (eg, the PG-XC GTM) to provide that ordering. You're thinking of XIDs. This is a proposal for user-defined sequences. XC also

Re: [HACKERS] Global Sequences

2012-10-15 Thread Christopher Browne
On Mon, Oct 15, 2012 at 5:33 PM, Simon Riggs si...@2ndquadrant.com wrote: Sequences, as defined by SQL Standard, provide a series of unique values. The current implementation on PostgreSQL isolates the generation mechanism to only a single node, as is common on many RDBMS. I remember

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
* Christopher Browne (cbbro...@gmail.com) wrote: There's a necessary trade-off; you can either have it globally *strongly* ordered, and, if so, you'll have to pay a hefty coordination price, or you can have the cheaper answer of a weakly ordered sequence. The latter leaves me feeling rather