Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 11, 2016 at 10:19:16AM +0100, Oleg Bartunov wrote: > Our XTM is the yet another example of infrastructure we need to work on > clustering. Should we wait other smart guy starts thinking on distributed > transactions ?  We described in https://wiki.postgresql.org/wiki/DTM our  API, >

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 11, 2016 at 9:09 AM, Bruce Momjian wrote: > > > > 3. I have tried to encourage others to get involved, with limited > success. I do think the FDW is perhaps the only reasonable way to get > _built-in_ sharding. The external sharding solutions are certainly >

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 11, 2016 at 04:30:13PM +0800, Craig Ringer wrote: > ... eventually. > > Sometimes the bug reports start. Occasionally you get a "thanks, this looks > interesting/handy". But usually just bug reports or complaints that whatever > you built isn't good enough to meet some random person's

Re: [HACKERS] The plan for FDW-based sharding

On 11 March 2016 at 16:09, Bruce Momjian wrote: > Ideally everything would have a well-defined plan, it is > sometimes hard to do. BDR helped for logical decoding etc - having something concrete really helped shape and guide each part of it as it was (or is/will be, in some

Re: [HACKERS] The plan for FDW-based sharding

I have read the recent comments on this thread with great interest. I am glad people have expressed their concerns, rather than remain silent. Now that the responses have decreased, I can reply. I saw several concerns: 1. My motivation for starting this thread was to decrease interest in

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 8, 2016 at 6:40 AM, Craig Ringer wrote: > Either that, or bless experimental features/API as an official concept. > I'd quite like that myself - stuff that's in Pg, but documented as "might > change or go away in the next release, experimental feature". As

Re: [HACKERS] The plan for FDW-based sharding

On 7 March 2016 at 23:02, Robert Haas wrote: > On Fri, Mar 4, 2016 at 11:17 PM, Craig Ringer > wrote: > > If FDW-based sharding works, I'm happy enough, I have no horse in this > race. > > If it doesn't work I don't much care either. What I'm

Re: [HACKERS] The plan for FDW-based sharding

On Mon, Mar 7, 2016 at 6:13 AM, Craig Ringer wrote: > On 5 March 2016 at 23:41, Kevin Grittner wrote: >> The only place you *need* to vary from commit order for correctness >> is when there are overlapping SERIALIZABLE transactions, one >> modifies data

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 11:17 PM, Craig Ringer wrote: > If FDW-based sharding works, I'm happy enough, I have no horse in this race. > If it doesn't work I don't much care either. What I'm worried about is it if > works like partitioning using inheritance works - horribly

Re: [HACKERS] The plan for FDW-based sharding

On 5 March 2016 at 23:41, Kevin Grittner wrote: > > > I'd be really interested in some ideas on how that information might be > > usefully accessed. If we could write info on when to apply commits to the > > xlog in serializable mode that'd be very handy, especially when

Re: [HACKERS] The plan for FDW-based sharding

On 03/07/2016 04:28 AM, Robert Haas wrote: On Fri, Mar 4, 2016 at 10:54 PM, Craig Ringer wrote: I've got to say that this is somewhat reminicient of the discussions around in-core pooling, where argument 1 is applied to justify excluding pooling from core/contrib. I

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 10:54 PM, Craig Ringer wrote: > I've got to say that this is somewhat reminicient of the discussions around > in-core pooling, where argument 1 is applied to justify excluding pooling > from core/contrib. > > I don't have a strong position on whether

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 10:23 PM, Craig Ringer wrote: > I can imagine that many such hooks would have little use beyond PPAS, but > I'm somewhat curious as to if any would have wider applications. It's not > unusual for me to be working on something and think "gee, I wish

Re: [HACKERS] The plan for FDW-based sharding

On 6 Mar 2016 8:27 p.m., "Peter Geoghegan" wrote: > > On Fri, Mar 4, 2016 at 4:41 PM, Robert Haas wrote: > > Yeah, I agree with that. I am utterly mystified by why Bruce keeps > > beating this drum, and am frankly pretty annoyed about it. In the > >

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 4:41 PM, Robert Haas wrote: > Yeah, I agree with that. I am utterly mystified by why Bruce keeps > beating this drum, and am frankly pretty annoyed about it. In the > first place, he seems to think that he invented the idea of using FDWs > for

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 10:10 PM, Craig Ringer wrote: > On 28 February 2016 at 06:38, Kevin Grittner wrote: >> What I sketched out with the "apparent order of execution" >> ordering of the transactions (basically, commit order except >> when one

Re: [HACKERS] The plan for FDW-based sharding

On 2 March 2016 at 03:02, Bruce Momjian wrote: > On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: > > Note that I am not saying that other discussed approaches are any > > better, I am saying that we should know approximately what we > > actually want and not just

Re: [HACKERS] The plan for FDW-based sharding

On 2 March 2016 at 00:03, Robert Haas wrote: > > True. There is an API, though, and having pluggable WAL support seems > desirable too. At the same time, I don't think we know of anyone > maintaining a non-core index AM ... and there are probably good > reasons for

Re: [HACKERS] The plan for FDW-based sharding

On 28 February 2016 at 06:38, Kevin Grittner wrote: > > > For logical replay, applying in batches is actually a good thing since it > > allows parallelism. We can remove them all from the target's procarray > all > > at once to avoid intermediate states becoming visible. So

Re: [HACKERS] The plan for FDW-based sharding

On 27 February 2016 at 15:29, Konstantin Knizhnik wrote: > Two reasons: > 1. There is no ideal implementation of DTM which will fit all possible > needs and be efficient for all clusters. > 2. Even if such implementation exists, still the right way of it >

Re: [HACKERS] The plan for FDW-based sharding

On 27 February 2016 at 11:54, Robert Haas wrote: > I could submit a patch adding > hooks to core to enable all of the things (or even just some of the > things) that EnterpriseDB has changed in Advanced Server, and that > patch would be rejected so fast it would make

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Mar 4, 2016 at 8:27 PM, Joshua D. Drake wrote: > This does not sound like Bruce at all. Bruce is a lot of things, stubborn, > sometimes temperamental, a lot of times like you... a hot head but he does > not take credit for other people's work in my experience. On

Re: [HACKERS] The plan for FDW-based sharding

On 03/04/2016 04:41 PM, Robert Haas wrote: As far as I understand it, Bruce came in near the end of that conversation and now wants to claim credit for something that doesn't really exist yet and, to the extent that it does exist, wasn't even his idea. Robert, This does not sound like Bruce

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 12:07 PM, Konstantin Knizhnik wrote: > In the article them used anotion "wait": > > if T.SnapshotTime>GetClockTime() > then wait until T.SnapshotTime > Originally we really do sleep here, but then we think that instead of >

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Mar 2, 2016 at 1:53 PM, Josh berkus wrote: > One of the things which causes bad reactions and arguments, Bruce, is that a > lot of your posts and presentations detailing plans for the FDW approach > carry the subtext that all four of the other approaches are dead ends

Re: [HACKERS] The plan for FDW-based sharding

On Mar 3, 2016 4:47 AM, "Michael Paquier" wrote: > > On Wed, Mar 2, 2016 at 6:54 PM, Alexander Korotkov > wrote: > > If FDWs existed then Postgres XC/XL were being developed then I believe they > > would try to build full-featured prototype

Re: [HACKERS] The plan for FDW-based sharding

> On Wed, Mar 2, 2016 at 6:54 PM, Alexander Korotkov > wrote: >> If FDWs existed then Postgres XC/XL were being developed then I believe they >> would try to build full-featured prototype of FDW based sharding. If this >> prototype succeed then we could make a full

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Mar 2, 2016 at 6:54 PM, Alexander Korotkov wrote: > If FDWs existed then Postgres XC/XL were being developed then I believe they > would try to build full-featured prototype of FDW based sharding. If this > prototype succeed then we could make a full roadmap.

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Mar 2, 2016 at 9:53 PM, Josh berkus wrote: > On 02/24/2016 01:22 AM, Konstantin Knizhnik wrote: > >> Sorry, but based on this plan it is possible to make a conclusion that >> there are only two possible cluster solutions for Postgres: >> XC/XL and FDW-based. From my

Re: [HACKERS] The plan for FDW-based sharding

On 02/24/2016 01:22 AM, Konstantin Knizhnik wrote: Sorry, but based on this plan it is possible to make a conclusion that there are only two possible cluster solutions for Postgres: XC/XL and FDW-based. From my point of view there are much more possible alternatives. Definitely. Currently

Re: [HACKERS] The plan for FDW-based sharding

On 01.03.2016 22:02, Bruce Momjian wrote: On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: Note that I am not saying that other discussed approaches are any better, I am saying that we should know approximately what we actually want and not just beat FDWs with a hammer and hope

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 10:11 PM, Bruce Momjian wrote: > On Tue, Mar 1, 2016 at 02:02:44PM -0500, Bruce wrote: > > On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: > > > Note that I am not saying that other discussed approaches are any > > > better, I am saying

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 7:03 PM, Robert Haas wrote: > On Tue, Mar 1, 2016 at 10:37 AM, Bruce Momjian wrote: > > On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: > >> > Two reasons: > >> > 1. There is no ideal implementation of DTM which will

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Mar 2, 2016 at 4:36 AM, Tomas Vondra wrote: Hi, > > On 03/01/2016 08:02 PM, Bruce Momjian wrote: > >> On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: >> >>> Note that I am not saying that other discussed approaches are any >>> better, I am

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 7:03 PM, Robert Haas wrote: > On Tue, Mar 1, 2016 at 10:37 AM, Bruce Momjian wrote: > > On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: > >> > Two reasons: > >> > 1. There is no ideal implementation of DTM which will

Re: [HACKERS] The plan for FDW-based sharding

Hi, On 03/01/2016 08:02 PM, Bruce Momjian wrote: On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: Note that I am not saying that other discussed approaches are any better, I am saying that we should know approximately what we actually want and not just beat FDWs with a hammer and

Re: [HACKERS] The plan for FDW-based sharding

On 03/01/2016 09:19 PM, Petr Jelinek wrote: Since this thread heavily discusses the XTM, I have question about the XTM as proposed because one thing is very unclear to me - what happens when user changes the XTM plugin on the server? I didn't see any xid handover API which makes me wonder if

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 02:02:44PM -0500, Bruce wrote: > On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: > > Note that I am not saying that other discussed approaches are any > > better, I am saying that we should know approximately what we > > actually want and not just beat FDWs

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 07:56:58PM +0100, Petr Jelinek wrote: > Note that I am not saying that other discussed approaches are any > better, I am saying that we should know approximately what we > actually want and not just beat FDWs with a hammer and hope sharding > will eventually emerge and call

Re: [HACKERS] The plan for FDW-based sharding

On 27/02/16 04:54, Robert Haas wrote: On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik wrote: We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are two general ways of dealing

Re: [HACKERS] The plan for FDW-based sharding

On 01/03/16 18:18, Konstantin Knizhnik wrote: On 01.03.2016 19:03, Robert Haas wrote: On Tue, Mar 1, 2016 at 10:37 AM, Bruce Momjian wrote: On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: Two reasons: 1. There is no ideal implementation of DTM which will fit

Re: [HACKERS] The plan for FDW-based sharding

On 01.03.2016 19:03, Robert Haas wrote: On Tue, Mar 1, 2016 at 10:37 AM, Bruce Momjian wrote: On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: Two reasons: 1. There is no ideal implementation of DTM which will fit all possible needs and be efficient for all

Re: [HACKERS] The plan for FDW-based sharding

Thank you very much for you comments. On 01.03.2016 18:19, Robert Haas wrote: On Sat, Feb 27, 2016 at 2:29 AM, Konstantin Knizhnik wrote: How do you prevent clock skew from causing serialization anomalies? If node receives message from "feature" it just needs to

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 10:37 AM, Bruce Momjian wrote: > On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: >> > Two reasons: >> > 1. There is no ideal implementation of DTM which will fit all possible >> > needs >> > and be efficient for all clusters. >> >> Hmm, what

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Mar 1, 2016 at 10:19:45AM -0500, Robert Haas wrote: > > Two reasons: > > 1. There is no ideal implementation of DTM which will fit all possible needs > > and be efficient for all clusters. > > Hmm, what is the reasoning behind that statement? I mean, it is > certainly true that there

Re: [HACKERS] The plan for FDW-based sharding

On Sat, Feb 27, 2016 at 2:29 AM, Konstantin Knizhnik wrote: >> How do you prevent clock skew from causing serialization anomalies? > > If node receives message from "feature" it just needs to wait until this > future arrive. > Practically we just "adjust" system time in

Re: [HACKERS] The plan for FDW-based sharding

On 27 February 2016 at 22:38, Kevin Grittner wrote: > That could be part of a solution. What I sketched out with the > "apparent order of execution" ordering of the transactions > (basically, commit order except when one SERIALIZABLE transaction > needs to be dragged in

Re: [HACKERS] The plan for FDW-based sharding

On 02/27/2016 11:38 PM, Kevin Grittner wrote: Is this an implementation of some particular formal technique? If so, do you have a reference to a paper on it? I get the sense that there has been a lot written about distributed transactions, and that it would be a mistake to ignore it, but I

Re: [HACKERS] The plan for FDW-based sharding

On Sat, Feb 27, 2016 at 3:57 PM, Simon Riggs wrote: > On 27 February 2016 at 17:54, Kevin Grittner wrote: >> >> On a single database SSI can see whether a read has >> caused such a problem. If you replicate the transactions to >> somewhere else and read

Re: [HACKERS] The plan for FDW-based sharding

On 27 February 2016 at 17:54, Kevin Grittner wrote: > On a single database SSI can see whether a read has > caused such a problem. If you replicate the transactions to > somewhere else and read them SSI cannot tell whether there is an > anomaly OK, I thought you were saying

Re: [HACKERS] The plan for FDW-based sharding

On Sat, Feb 27, 2016 at 1:14 PM, Konstantin Knizhnik wrote: > We do not try to preserve transaction commit order at all nodes. > But in principle it can be implemented using XTM API: it allows to redefine > function which actually sets transaction status. pg_dtm

Re: [HACKERS] The plan for FDW-based sharding

Neither pg_dtm, neither pg_tsdtm supports serializable isolation level. We implemented distributed snapshot isolation - repeatable-read isolation level. We also do not support read-committed isolation level now. We do not try to preserve transaction commit order at all nodes. But in principle it

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 5:37 PM, Simon Riggs wrote: > On 26 February 2016 at 22:48, Kevin Grittner wrote: >> if we want logical >> replication to be free of serialization anomalies for those using >> serializable transactions, we need to support

Re: [HACKERS] The plan for FDW-based sharding

On 27/02/16 09:19, Konstantin Knizhnik wrote: On 02/27/2016 06:54 AM, Robert Haas wrote: [...] So maybe the goal for the GTM isn't to provide true serializability across the cluster but some lesser degree of transaction isolation. But then exactly which serialization anomalies are we

Re: [HACKERS] The plan for FDW-based sharding

On 02/27/2016 06:54 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik wrote: We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are two general ways of

Re: [HACKERS] The plan for FDW-based sharding

On 02/27/2016 06:57 AM, Robert Haas wrote: On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik wrote: pg_tsdtm is based on another approach: it is using system time as CSN and doesn't require arbiter. In theory there is no limit for scalability. But differences in

Re: [HACKERS] The plan for FDW-based sharding

On Sat, Feb 27, 2016 at 1:49 AM, Konstantin Knizhnik wrote: > pg_tsdtm is based on another approach: it is using system time as CSN and > doesn't require arbiter. In theory there is no limit for scalability. But > differences in system time and necessity to use more

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 10:56 PM, Konstantin Knizhnik wrote: > We do not have formal prove that proposed XTM is "general enough" to handle > all possible transaction manager implementations. > But there are two general ways of dealing with isolation: snapshot based and

Re: [HACKERS] The plan for FDW-based sharding

On 26 February 2016 at 22:48, Kevin Grittner wrote: > On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik > wrote: > > > pg_tsdtm is based on another approach: it is using system time > > as CSN > > Which brings up an interesting point, if we want

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 2:19 PM, Konstantin Knizhnik wrote: > pg_tsdtm is based on another approach: it is using system time > as CSN Which brings up an interesting point, if we want logical replication to be free of serialization anomalies for those using

Re: [HACKERS] The plan for FDW-based sharding

On 02/26/2016 09:30 PM, Alvaro Herrera wrote: Konstantin Knizhnik wrote: Yes, it is certainly possible to develop cluster by cloning PostgreSQL. But it cause big problems both for developers, which have to permanently synchronize their branch with master, and, what is more important, for

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 03:30:29PM -0300, Alvaro Herrera wrote: > That's not the point, though. I don't think a Postgres clone with a GTM > solves any particular problem that's not already solved by the existing > forks. However, if you have a clone at home and you make a GTM work on > it, then

Re: [HACKERS] The plan for FDW-based sharding

Konstantin Knizhnik wrote: > Yes, it is certainly possible to develop cluster by cloning PostgreSQL. > But it cause big problems both for developers, which have to permanently > synchronize their branch with master, > and, what is more important, for customers, which can not use standard >

Re: [HACKERS] The plan for FDW-based sharding

We do not have formal prove that proposed XTM is "general enough" to handle all possible transaction manager implementations. But there are two general ways of dealing with isolation: snapshot based and CSN based. pg_dtm and pg_tsdtm prove that both of them can be implemented using XTM. If you

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 10:00 PM, Joshua D. Drake wrote: > Robert, this is all a game. It is a game of who wins the intellectual prize > to whatever problem. Who gets the market or mind share and who gets to > pretend they win the Oscar for coolest design. JD, I don't

Re: [HACKERS] The plan for FDW-based sharding

On 02/26/2016 08:06 AM, Robert Haas wrote: On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov wrote: Right now tm is hardcoded and it's doesn't matter "if other people might need" at all. We at least provide developers ("other people") ability to work on their

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 7:21 PM, Oleg Bartunov wrote: > Right now tm is hardcoded and it's doesn't matter "if other people might > need" at all. We at least provide developers ("other people") ability to > work on their implementations and the patch is safe and doesn't

Re: [HACKERS] The plan for FDW-based sharding

On Fri, Feb 26, 2016 at 3:50 PM, Robert Haas wrote: > On Wed, Feb 24, 2016 at 3:05 PM, Oleg Bartunov > wrote: > > I already several times pointed, that we need XTM to be able to continue > > development in different directions, since there is no clear

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 3:05 PM, Oleg Bartunov wrote: > I already several times pointed, that we need XTM to be able to continue > development in different directions, since there is no clear winner. > Moreover, I think there is no fits-all solution and while I agree we need

Re: [HACKERS] The plan for FDW-based sharding

On Thu, Feb 25, 2016 at 01:53:12PM +0900, Michael Paquier wrote: > > Well, as far as I know XC doesn't support data redistribution between > > nodes and I saw good benchmarks of that, as well as XL. > > XC does support that in 1.2 with a very basic approach (coded that > years ago), though it

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 11:34 PM, Bruce Momjian wrote: > On Wed, Feb 24, 2016 at 12:17:28PM +0300, Alexander Korotkov wrote: >> Hi, Bruce! >> >> The important point for me is to distinguish different kind of plans: >> implementation plan and research plan. >> If we're talking

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 01:02:21PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Wed, Feb 24, 2016 at 01:08:29AM +, Simon Riggs wrote: > > > > It's never been our policy to try to include major projects in single code > > > drops. Any move of XL/XC code into PostgreSQL core would

Re: [HACKERS] The plan for FDW-based sharding

Bruce Momjian wrote: > On Wed, Feb 24, 2016 at 01:08:29AM +, Simon Riggs wrote: > > It's never been our policy to try to include major projects in single code > > drops. Any move of XL/XC code into PostgreSQL core would need to be done > > piece > > by piece across many releases. XL is

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 09:34:37AM -0500, Bruce Momjian wrote: > > I have nothing against particular FDW advances. However, it's unclear for me > > that FDW should be the only sharding approach. > > It's unproven that FDW can do work that Postgres XC/XL does. With FDW we can > > have some 

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 12:22:20PM +0300, Konstantin Knizhnik wrote: > Sorry, but based on this plan it is possible to make a conclusion > that there are only two possible cluster solutions for Postgres: > XC/XL and FDW-based. From my point of view there are much more > possible alternatives. >

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 12:35:15PM +0300, Oleg Bartunov wrote: > I have nothing against particular FDW advances. However, it's unclear for > me that FDW should be the only sharding approach. > It's unproven that FDW can do work that Postgres XC/XL does. With FDW we > can have some 

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 12:17:28PM +0300, Alexander Korotkov wrote: > Hi, Bruce! > > The important point for me is to distinguish different kind of plans: > implementation plan and research plan. > If we're talking about implementation plan then it should be proven that > proposed approach works

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 12:17 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > Hi, Bruce! > > The important point for me is to distinguish different kind of plans: > implementation plan and research plan. > If we're talking about implementation plan then it should be proven that >

Re: [HACKERS] The plan for FDW-based sharding

Sorry, but based on this plan it is possible to make a conclusion that there are only two possible cluster solutions for Postgres: XC/XL and FDW-based. From my point of view there are much more possible alternatives. Our main idea with XTM (eXtensible Transaction Manager API) was to make it

Re: [HACKERS] The plan for FDW-based sharding

Hi, Bruce! The important point for me is to distinguish different kind of plans: implementation plan and research plan. If we're talking about implementation plan then it should be proven that proposed approach works in this case. I.e research should be already done. If we're talking about

Re: [HACKERS] The plan for FDW-based sharding

On Wed, Feb 24, 2016 at 01:08:29AM +, Simon Riggs wrote: > On 23 February 2016 at 16:43, Bruce Momjian wrote: > > There was discussion at the FOSDEM/PGDay Developer Meeting > (https://wiki.postgresql.org/wiki/FOSDEM/PGDay_2016_Developer_Meeting) > about sharding

Re: [HACKERS] The plan for FDW-based sharding

On 23 February 2016 at 16:43, Bruce Momjian wrote: > There was discussion at the FOSDEM/PGDay Developer Meeting > (https://wiki.postgresql.org/wiki/FOSDEM/PGDay_2016_Developer_Meeting) > about sharding so I wanted to outline where I think we are going with > sharding and FDWs.

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Feb 23, 2016 at 09:54:46AM -0700, David G. Johnston wrote: > On Tue, Feb 23, 2016 at 9:43 AM, Bruce Momjian wrote: > > 4. Cross-node read-write queries: > > This will require a global snapshot manager and global snapshot manager. > > > Probably meant "global

Re: [HACKERS] The plan for FDW-based sharding

On Tue, Feb 23, 2016 at 9:43 AM, Bruce Momjian wrote: > 4. Cross-node read-write queries: > > This will require a global snapshot manager and global snapshot manager. > Probably meant "global transaction manager" ​David J.​