On Wed, Mar 2, 2016 at 1:53 PM, Josh berkus <j...@agliodbs.com> wrote: > One of the things which causes bad reactions and arguments, Bruce, is that a > lot of your posts and presentations detailing plans for the FDW approach > carry the subtext that all four of the other approaches are dead ends and > not worth considering. Given that the other approaches, whatever their > limitations, have working code in the field and the FDW approach does not, > that's more than a little offensive.
Yeah, I agree with that. I am utterly mystified by why Bruce keeps beating this drum, and am frankly pretty annoyed about it. In the first place, he seems to think that he invented the idea of using FDWs for sharding in PostgreSQL, but I don't think that's true. I think it was partly my idea, and partly something that the NTT folks have been working on for years (cf, e.g., cb1ca4d800621dcae67ca6c799006de99fa4f0a5). As far as I understand it, Bruce came in near the end of that conversation and now wants to claim credit for something that doesn't really exist yet and, to the extent that it does exist, wasn't even his idea. In the second place, the only thing that these repeated emails and development meeting discussions of the topic actually accomplish is to be piss people off. I do believe that enhancing the foreign data wrapper interface can be part of a horizontal scalability story for PostgreSQL, but as long as nobody is objecting to the individual enhancements, which I don't see anybody doing, then why the heck do we have to keep arguing about this big picture story? It doesn't matter at all, and it doesn't even really exist, yet somehow Bruce keeps bringing it up, which I think serves no useful purpose whatsoever. > If we want to move forwards on serious work on FDW-based sharding, the folks > working on it should stop treating it as a "fait accompli" that this is the > Chosen Way for the PostgreSQL project. Otherwise, you'll spend all of your > time arguing that point instead of working on features that matter. The only person treating it that way is Bruce. > In contrast, this FDW plan *still* feels very much like a small group made > up of employees of only two companies came up with it in private and decided > that it should be the plan for the whole project. I know that Bruce and > others have good reasons for starting the FDW project, but there hasn't been > much of an attempt to obtain community consensus around it. If Bruce and > others want contributors to work on FDWs instead of other sharding > approaches, then they need to win over those people as to why they should do > that. It's how this community works. There hasn't been much of an attempt to obtain community consensus about it because there isn't actually some grand plan, private or otherwise, much as Bruce's emails might make you think otherwise. EnterpriseDB *does* have a plan to try to continue enhancing foreign data wrappers so that you can run queries against foreign tables and get reasonable plans, something that currently isn't true. I haven't heard anybody objecting to that, and I don't expect to hear anybody objecting to that, because it's hard to imagine why you wouldn't want queries against foreign data wrappers to produce better plans than they do today. At worst, you might think it doesn't matter either way, but actually, I think there are a substantial number of people who are pretty happy about join pushdown and I expect that when and if we get aggregate pushdown working there will be even more people who are happy about that. The only other ongoing work that EnterpriseDB has that at all touches on this area is Ashutosh Bapat's work on 2PC for FDWs. I'm not convinced that's fully baked, and it conflicts with the XTM stuff the Postgres Pro guys are doing, which I *also* don't think is fully baked, so I'm not real keen on pressing forward aggressively with either approach right now. I think we (eventually) need a solution to the problem of consistent cross-node consistency, but I am deeply unconvinced that anything currently on the table is going to get us there. I did recommend the 2PC for FDW project, but I'm not amazingly happy with how it came out, and I think we need to think harder about other approaches before adopting something. > Alternately, you can just work on the individual FDW features, which > *everyone* thinks are a good idea, and when most of them are done, FDW-based > scaleout will be such an obvious solution that nobody will argue with it. That's exactly what the people at EnterpriseDB who are actually doing work in this area are attempting to do. Meanwhile, there's also Bruce, who is neither doing nor planning to do any work in this area, nor advising either EnterpriseDB or the PostgreSQL community to undertake any particular project, but who *is* making it sound like there is a super sekret plan that nobody else gets to see. However, as the guy who actually wrote the plan that EnterpriseDB is following, I happen to know that there's nothing more to it than what I wrote above. Argh! -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
