Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 18 October 2012 16:15, Tom Lane wrote: > But I'd want to see a pretty > bulletproof argument why overriding *only* nextval is sufficient > (and always will be) before accepting a hook for just nextval. If we > build an equivalent amount of functionality piecemeal it's going to > be a lot ugli

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 18 October 2012 16:08, Robert Haas wrote: > To make > it even better, add some generic options that can be passed through to > the underlying handler. Agreed >> Or maybe better, invent a level of indirection like a "sequence access >> method" (comparable to index access methods) that provide

Re: [HACKERS] Global Sequences

2012-10-18 Thread Tom Lane
Robert Haas writes: > On Tue, Oct 16, 2012 at 1:29 PM, Tom Lane wrote: >> Or maybe better, invent a level of indirection like a "sequence access >> method" (comparable to index access methods) that provides a compatible >> set of substitute functions for sequence operations. If you want to >> ov

Re: [HACKERS] Global Sequences

2012-10-18 Thread Robert Haas
On Tue, Oct 16, 2012 at 1:29 PM, Tom Lane wrote: > In particular, the reason proposing a hook first seems backwards is that > if we have a catalog-level representation that some sequences are local > and others not, we should be using that to drive the determination of > whether to call a substitu

Re: [HACKERS] Global Sequences

2012-10-18 Thread Dimitri Fontaine
Simon Riggs writes: > Not sure how it is cleaner when we have to have trigger stuff hanging > around to make one object pretend to be another. That also creates a > chain of dependency which puts this into the future, rather than now. Yes, that part isn't cleaner at all. The part where we have a

Re: [HACKERS] Global Sequences

2012-10-18 Thread Simon Riggs
On 16 October 2012 18:29, Tom Lane wrote: > Or maybe better, invent a level of indirection like a "sequence access > method" (comparable to index access methods) that provides a compatible > set of substitute functions for sequence operations. If you want to > override nextval() for a sequence,

Re: [HACKERS] Global Sequences

2012-10-17 Thread Simon Riggs
On 17 October 2012 10:19, Markus Wanner wrote: > On 10/17/2012 10:34 AM, Simon Riggs wrote: >> IMHO an API is required for "give me the next allocation of numbers", >> essentially a bulk equivalent of nextval(). > > Agreed. That pretty exactly matches what I described (and what's > implemented in

Re: [HACKERS] Global Sequences

2012-10-17 Thread Simon Riggs
On 17 October 2012 11:21, Dimitri Fontaine wrote: > Simon Riggs writes: >> On 16 October 2012 15:15, Tom Lane wrote: >>> What you really want is something vaguely like nextval but applied to >>> a distinct type of object. That is, I think we first need a different >>> kind of object called a "g

Re: [HACKERS] Global Sequences

2012-10-17 Thread Dimitri Fontaine
Simon Riggs writes: > On 16 October 2012 15:15, Tom Lane wrote: >> What you really want is something vaguely like nextval but applied to >> a distinct type of object. That is, I think we first need a different >> kind of object called a "global sequence" with its own DDL operations. >> > hence a

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Simon, On 10/17/2012 10:34 AM, Simon Riggs wrote: > IMHO an API is required for "give me the next allocation of numbers", > essentially a bulk equivalent of nextval(). Agreed. That pretty exactly matches what I described (and what's implemented in Postgres-R). The API then only needs to be called

Re: [HACKERS] Global Sequences

2012-10-17 Thread Simon Riggs
On 17 October 2012 09:10, Markus Wanner wrote: > Simon, > > On 10/16/2012 02:36 PM, Simon Riggs wrote: >> Where else would you put the hook? The hook's location as described >> won't change whether you decide you want 1, 2 or 3. > > You assume we want an API that supports all three options. In tha

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Tom, On 10/16/2012 06:15 PM, Tom Lane wrote: > I challenge you to find anything in the SQL standard that suggests that > sequences have any nonlocal behavior. If anything, what you propose > violates the standard, it doesn't make us follow it more closely. If you look at a distributed database a

Re: [HACKERS] Global Sequences

2012-10-17 Thread Markus Wanner
Simon, On 10/16/2012 02:36 PM, Simon Riggs wrote: > Where else would you put the hook? The hook's location as described > won't change whether you decide you want 1, 2 or 3. You assume we want an API that supports all three options. In that case, yes, the hooks need to be very general. Given tha

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs writes: > On 16 October 2012 17:15, Tom Lane wrote: >> So I fully >> expect that we're going to need something different from bog-standard >> CREATE SEQUENCE. > There's no point in that at all, as explained. It's sequences that > need to work. We can already call my_nextval() rather

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 17:17, Peter Eisentraut wrote: > On 10/16/12 9:20 AM, Simon Riggs wrote: >> I've proposed a plugin for the allocation only. So the allocation >> looks like anything you want. > > Are you planning to provide a reference implementation of some kind? I'll provide hooks and a stub

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 17:15, Tom Lane wrote: > Simon Riggs writes: >> On 16 October 2012 15:15, Tom Lane wrote: >>> I think this is a fundamentally wrong way to go about doing what you >>> want to do. It presumes that DDL-level manipulation of global sequences >>> is exactly like local sequences;

Re: [HACKERS] Global Sequences

2012-10-16 Thread Daniel Farina
On Tue, Oct 16, 2012 at 5:54 AM, Peter Eisentraut wrote: > On 10/15/12 5:33 PM, Simon Riggs wrote: >> There are a few options >> 1) Manual separation of the value space, so that N1 has 50% of >> possible values and N2 has 50%. That has problems when we reconfigure >> the cluster, and requires comp

Re: [HACKERS] Global Sequences

2012-10-16 Thread Peter Eisentraut
On 10/16/12 9:20 AM, Simon Riggs wrote: > I've proposed a plugin for the allocation only. So the allocation > looks like anything you want. Are you planning to provide a reference implementation of some kind? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs writes: > On 16 October 2012 15:15, Tom Lane wrote: >> I think this is a fundamentally wrong way to go about doing what you >> want to do. It presumes that DDL-level manipulation of global sequences >> is exactly like local sequences; an assumption that is obviously false. > The mes

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 15:15, Tom Lane wrote: > Simon Riggs writes: >> So, proposal is to allow nextval() allocation to access a plugin, >> rather than simply write a WAL record and increment. If the plugin is >> loaded all sequences call it (not OIDs). > > I think this is a fundamentally wrong way t

Re: [HACKERS] Global Sequences

2012-10-16 Thread Tom Lane
Simon Riggs writes: > So, proposal is to allow nextval() allocation to access a plugin, > rather than simply write a WAL record and increment. If the plugin is > loaded all sequences call it (not OIDs). I think this is a fundamentally wrong way to go about doing what you want to do. It presumes

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 13:54, Peter Eisentraut wrote: > On 10/15/12 5:33 PM, Simon Riggs wrote: >> There are a few options >> 1) Manual separation of the value space, so that N1 has 50% of >> possible values and N2 has 50%. That has problems when we reconfigure >> the cluster, and requires complex man

Re: [HACKERS] Global Sequences

2012-10-16 Thread Andres Freund
On Tuesday, October 16, 2012 02:58:11 PM Andrew Dunstan wrote: > On 10/16/2012 08:54 AM, Peter Eisentraut wrote: > > Option 4 is of course to use UUIDs. > > Yeah, I was wondering what this would really solve that using UUIDs > wouldn't solve. Large indexes over random values perform notably worse

Re: [HACKERS] Global Sequences

2012-10-16 Thread Andrew Dunstan
On 10/16/2012 08:54 AM, Peter Eisentraut wrote: Option 4 is of course to use UUIDs. Yeah, I was wondering what this would really solve that using UUIDs wouldn't solve. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Global Sequences

2012-10-16 Thread Peter Eisentraut
On 10/15/12 5:33 PM, Simon Riggs wrote: > There are a few options > 1) Manual separation of the value space, so that N1 has 50% of > possible values and N2 has 50%. That has problems when we reconfigure > the cluster, and requires complex manual reallocation of values. So it > starts good but ends

Re: [HACKERS] Global Sequences

2012-10-16 Thread Simon Riggs
On 16 October 2012 13:26, Markus Wanner wrote: > Why does a "Global Sequences" API necessarily hook at the nextval() and > setval() level? That sounds like it yields an awkward amount of > duplicate work. Reading this thread, so far it looks like we agree that > option 3) is the most feasible opt

Re: [HACKERS] Global Sequences

2012-10-16 Thread Markus Wanner
On 10/16/2012 12:47 AM, Josh Berkus wrote: > I'd also love to hear from the PostgresXC folks on whether this solution > works for them. Postgres-R too. In Postgres-R, option 3) is implemented. Though, by default sequences work just like on a single machine, giving you monotonically increasing seq

Re: [HACKERS] Global Sequences

2012-10-16 Thread Yeb Havinga
On 2012-10-15 23:33, Simon Riggs wrote: So, proposal is to allow nextval() allocation to access a plugin, rather than simply write a WAL record and increment. If the plugin is loaded all sequences call it (not OIDs). +1. It is currently impossible to alter nextvals behaviour, without making chan

Re: [HACKERS] Global Sequences

2012-10-15 Thread Simon Riggs
On 16 October 2012 03:03, Christopher Browne wrote: > There's a necessary trade-off; you can either have it globally > *strongly* ordered, and, if so, you'll have to pay a hefty > coordination price, or you can have the cheaper answer of a weakly > ordered sequence. The latter leaves me feeling

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
* Christopher Browne (cbbro...@gmail.com) wrote: > There's a necessary trade-off; you can either have it globally > *strongly* ordered, and, if so, you'll have to pay a hefty > coordination price, or you can have the cheaper answer of a weakly > ordered sequence. The latter leaves me feeling rathe

Re: [HACKERS] Global Sequences

2012-10-15 Thread Christopher Browne
On Mon, Oct 15, 2012 at 5:33 PM, Simon Riggs wrote: > Sequences, as defined by SQL Standard, provide a series of unique > values. The current implementation on PostgreSQL isolates the > generation mechanism to only a single node, as is common on many > RDBMS. I remember constructing at least the

Re: [HACKERS] Global Sequences

2012-10-15 Thread Michael Paquier
On Tue, Oct 16, 2012 at 10:30 AM, Josh Berkus wrote: > Stephen, > > > AIUI, PG-XC and PG-R need an order, so they both use an independent > > system (eg, the PG-XC GTM) to provide that ordering. > > You're thinking of XIDs. This is a proposal for user-defined sequences. > XC also provides global

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > > AIUI, PG-XC and PG-R need an order, so they both use an independent > > system (eg, the PG-XC GTM) to provide that ordering. > > You're thinking of XIDs. This is a proposal for user-defined sequences. Right, I got that it's a proposal for user sequenc

Re: [HACKERS] Global Sequences

2012-10-15 Thread Josh Berkus
Stephen, > AIUI, PG-XC and PG-R need an order, so they both use an independent > system (eg, the PG-XC GTM) to provide that ordering. You're thinking of XIDs. This is a proposal for user-defined sequences. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] Global Sequences

2012-10-15 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: > I'd also love to hear from the PostgresXC folks on whether this solution > works for them. Postgres-R too. If it works for all three of those > tools, it's liable to work for any potential new tool. AIUI, PG-XC and PG-R need an order, so they bot

Re: [HACKERS] Global Sequences

2012-10-15 Thread Daniel Farina
On Mon, Oct 15, 2012 at 2:33 PM, Simon Riggs wrote: > Sequences, as defined by SQL Standard, provide a series of unique > values. The current implementation on PostgreSQL isolates the > generation mechanism to only a single node, as is common on many > RDBMS. > > For sharded or replicated systems

Re: [HACKERS] Global Sequences

2012-10-15 Thread Josh Berkus
On 10/15/12 2:33 PM, Simon Riggs wrote: > 3) Lazy allocation from the value space. When a node is close to > running out of values, it requests a new allocation and coordinates > with all nodes to confirm the new allocation is good. > > (3) is similar to the way values are allocated currently, so

[HACKERS] Global Sequences

2012-10-15 Thread Simon Riggs
Sequences, as defined by SQL Standard, provide a series of unique values. The current implementation on PostgreSQL isolates the generation mechanism to only a single node, as is common on many RDBMS. For sharded or replicated systems it forces people to various hackish mechanisms in user space for