Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Oct 30, 2017 at 5:48 PM, Ashutosh Bapat wrote: > On Thu, Oct 26, 2017 at 7:41 PM, Masahiko Sawada > wrote: >> >> Because I don't want to break the current user semantics. that is, >> currently it's guaranteed that the subsequent

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 26, 2017 at 7:41 PM, Masahiko Sawada wrote: > > Because I don't want to break the current user semantics. that is, > currently it's guaranteed that the subsequent reads can see the > committed result of previous writes even if the previous transactions > were

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 26, 2017 at 4:11 PM, Masahiko Sawada wrote: >> Why do we want the the backend to linger behind, once it has added its >> foreign transaction entries in the shared memory and informed resolver >> about it? The foreign connections may take their own time and even

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 26, 2017 at 2:36 PM, Ashutosh Bapat wrote: > On Wed, Oct 25, 2017 at 3:15 AM, Masahiko Sawada > wrote: >> >> Foreign Transaction Resolver >> == >> I introduced a new background worker called "foreign

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Oct 25, 2017 at 3:15 AM, Masahiko Sawada wrote: > > Foreign Transaction Resolver > == > I introduced a new background worker called "foreign transaction > resolver" which is responsible for resolving the transaction prepared > on foreign servers.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Sep 29, 2017 at 9:12 PM, Robert Haas wrote: > > It's possible that we might find that neither of the above approaches > are practical and that the performance benefits of resolving the > transaction from the original connection are large enough that we want > to try

Re: [HACKERS] Transactions involving multiple postgres foreign servers

> On 02 Oct 2017, at 08:31, Masahiko Sawada wrote: > > On Sat, Sep 30, 2017 at 12:42 AM, Robert Haas wrote: >> On Wed, Sep 27, 2017 at 11:15 PM, Masahiko Sawada >> wrote: >>> I think that making a resolver process have

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Sat, Sep 30, 2017 at 12:42 AM, Robert Haas wrote: > On Wed, Sep 27, 2017 at 11:15 PM, Masahiko Sawada > wrote: >> I think that making a resolver process have connection caches to each >> foreign server for a while can reduce the overhead of

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 27, 2017 at 11:15 PM, Masahiko Sawada wrote: > I think that making a resolver process have connection caches to each > foreign server for a while can reduce the overhead of connection to > foreign servers. These connections will be invalidated by DDLs. Also, >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 27, 2017 at 4:05 PM, Ashutosh Bapat wrote: > On Wed, Sep 27, 2017 at 12:11 PM, Masahiko Sawada > wrote: >> On Tue, Sep 26, 2017 at 9:50 PM, Robert Haas wrote: >>> On Tue, Sep 26, 2017 at 5:06 AM,

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 27, 2017 at 12:11 PM, Masahiko Sawada wrote: > On Tue, Sep 26, 2017 at 9:50 PM, Robert Haas wrote: >> On Tue, Sep 26, 2017 at 5:06 AM, Masahiko Sawada >> wrote: >>> Based on the review comment from Robert, I'm

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 26, 2017 at 9:50 PM, Robert Haas wrote: > On Tue, Sep 26, 2017 at 5:06 AM, Masahiko Sawada > wrote: >> Based on the review comment from Robert, I'm planning to do the big >> change to the architecture of this patch so that a backend

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 26, 2017 at 5:06 AM, Masahiko Sawada wrote: > Based on the review comment from Robert, I'm planning to do the big > change to the architecture of this patch so that a backend process > work together with a dedicated background worker that is responsible > for

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Aug 1, 2017 at 1:40 AM, Robert Haas wrote: > On Thu, Jul 27, 2017 at 8:25 AM, Ashutosh Bapat > wrote: >> The remote transaction can be committed/aborted only after the fate of >> the local transaction is decided. If we commit remote

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Jul 31, 2017 at 7:27 PM, Alvaro Herrera wrote: > Robert Haas wrote: > >> An alternative approach is to have some kind of other identifier, >> let's call it a distributed transaction ID (DXID) which is mapped by >> each node onto a local XID. > > Postgres-XL seems

Re: [HACKERS] Transactions involving multiple postgres foreign servers

> On 31 Jul 2017, at 20:03, Robert Haas wrote: > > Regardless of whether we share XIDs or DXIDs, we need a more complex > concept of transaction state than we have now. Seems that discussion shifted from 2PC itself to the general issues with distributed transactions. So

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Aug 1, 2017 at 1:40 AM, Robert Haas wrote: > On Thu, Jul 27, 2017 at 8:25 AM, Ashutosh Bapat > wrote: >> The remote transaction can be committed/aborted only after the fate of >> the local transaction is decided. If we commit remote

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Aug 1, 2017 at 3:43 AM, Robert Haas wrote: > On Mon, Jul 31, 2017 at 1:27 PM, Alvaro Herrera > wrote: >> Postgres-XL seems to manage this problem by using a transaction manager >> node, which is in charge of assigning snapshots. I don't

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Jul 31, 2017 at 1:27 PM, Alvaro Herrera wrote: > Postgres-XL seems to manage this problem by using a transaction manager > node, which is in charge of assigning snapshots. I don't know how that > works, but perhaps adding that concept here could be useful too.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Robert Haas wrote: > An alternative approach is to have some kind of other identifier, > let's call it a distributed transaction ID (DXID) which is mapped by > each node onto a local XID. Postgres-XL seems to manage this problem by using a transaction manager node, which is in charge of

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Jul 28, 2017 at 10:14 AM, Michael Paquier wrote: > On Fri, Jul 28, 2017 at 7:28 AM, Masahiko Sawada > wrote: >> That also requires to share the same XID space with all remote nodes. > > You are putting your finger on the main bottleneck

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jul 27, 2017 at 8:25 AM, Ashutosh Bapat wrote: > The remote transaction can be committed/aborted only after the fate of > the local transaction is decided. If we commit remote transaction and > abort local transaction, that's not good. AtEOXact* functions

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Jul 28, 2017 at 7:28 AM, Masahiko Sawada wrote: > That also requires to share the same XID space with all remote nodes. You are putting your finger on the main bottleneck with global consistency that XC and XL has because of that. And the source feeding the XIDs is

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jul 27, 2017 at 8:02 PM, Robert Haas wrote: > On Thu, Jul 27, 2017 at 5:08 AM, Stas Kelvich > wrote: >> As far as I understand any solution that provides proper isolation for >> distributed >> transactions in postgres will require

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jul 27, 2017 at 6:58 AM, Robert Haas wrote: > > On a technical level, I am pretty sure that it is not OK to call > AtEOXact_FDWXacts() from the sections of CommitTransaction, > AbortTransaction, and PrepareTransaction that are described as > "non-critical resource

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jul 27, 2017 at 5:08 AM, Stas Kelvich wrote: > As far as I understand any solution that provides proper isolation for > distributed > transactions in postgres will require distributed 2PC somewhere under the > hood. > That is just consequence of parallelism

Re: [HACKERS] Transactions involving multiple postgres foreign servers

> On 27 Jul 2017, at 04:28, Robert Haas wrote: > > However, you would not > be guaranteed that all of those commits or rollbacks happen at > anything like the same time. So, you would have a sort of eventual > consistency. As far as I understand any solution that

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jul 27, 2017 at 10:28 AM, Robert Haas wrote: > On Fri, Apr 7, 2017 at 10:56 AM, Masahiko Sawada > wrote: >> Vinayak, why did you marked this patch as "Move to next CF"? AFAIU >> there is not discussion yet. > > I'd like to discuss this

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Apr 7, 2017 at 10:56 AM, Masahiko Sawada wrote: > Vinayak, why did you marked this patch as "Move to next CF"? AFAIU > there is not discussion yet. I'd like to discuss this patch. Clearly, a lot of work has been done here, but I am not sure about the approach. If

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Mar 29, 2017 at 11:14 PM, Masahiko Sawada wrote: > On Wed, Mar 22, 2017 at 2:49 AM, Masahiko Sawada > wrote: >> On Thu, Mar 16, 2017 at 2:37 PM, Vinayak Pokale >> wrote: >>> The following review has been

Re: [HACKERS] Transactions involving multiple postgres foreign servers

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed I have tested the latest patch and it looks good to me, so I

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Mar 2, 2017 at 11:56 AM, vinayak wrote: > > On 2017/02/28 16:54, Masahiko Sawada wrote: > > I've created a wiki page[1] describing about the design and > functionality of this feature. Also it has some examples of use case, > so this page would be helpful

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2017/02/28 16:54, Masahiko Sawada wrote: I've created a wiki page[1] describing about the design and functionality of this feature. Also it has some examples of use case, so this page would be helpful for even testing. Please refer it if you're interested in testing this feature. [1] 2PC

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Feb 15, 2017 at 3:11 PM, Masahiko Sawada wrote: > On Mon, Feb 6, 2017 at 10:48 PM, Masahiko Sawada > wrote: >> On Wed, Feb 1, 2017 at 8:25 PM, Robert Haas wrote: >>> On Mon, Jan 30, 2017 at 2:30 AM, Masahiko Sawada

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Feb 1, 2017 at 8:25 PM, Robert Haas wrote: > On Mon, Jan 30, 2017 at 2:30 AM, Masahiko Sawada > wrote: >> "txn" can be used for abbreviation of "Transaction", so for example >> pg_fdw_txn_resolver? >> I'm also fine to change the module and

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Jan 30, 2017 at 2:30 AM, Masahiko Sawada wrote: > "txn" can be used for abbreviation of "Transaction", so for example > pg_fdw_txn_resolver? > I'm also fine to change the module and function name. If we're judging the relative clarity of various ways of

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jan 26, 2017 at 6:49 PM, Masahiko Sawada wrote: > Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please > use attached patch. This patch has been moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Jan 30, 2017 at 12:50 PM, Ashutosh Bapat wrote: > On Sat, Jan 28, 2017 at 8:41 PM, Peter Eisentraut > wrote: >> On 1/26/17 4:49 AM, Masahiko Sawada wrote: >>> Sorry, I attached wrong version patch of pg_fdw_xact_resovler.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Sat, Jan 28, 2017 at 8:41 PM, Peter Eisentraut wrote: > On 1/26/17 4:49 AM, Masahiko Sawada wrote: >> Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please >> use attached patch. > > So in some other thread we are talking about renaming "xlog",

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2017/01/29 0:11, Peter Eisentraut wrote: On 1/26/17 4:49 AM, Masahiko Sawada wrote: Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please use attached patch. So in some other thread we are talking about renaming "xlog", because nobody knows what the "x" means. In the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 1/26/17 4:49 AM, Masahiko Sawada wrote: > Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please > use attached patch. So in some other thread we are talking about renaming "xlog", because nobody knows what the "x" means. In the spirit of that, let's find better names for new

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jan 26, 2017 at 6:04 PM, vinayak wrote: > Hi Sawada-san, > > On 2017/01/26 16:51, Masahiko Sawada wrote: > > Thank you for reviewing! > > I think this is a bug of pg_fdw_resolver contrib module. I had > forgotten to change the SQL executed by

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Hi Sawada-san, On 2017/01/26 16:51, Masahiko Sawada wrote: Thank you for reviewing! I think this is a bug of pg_fdw_resolver contrib module. I had forgotten to change the SQL executed by pg_fdw_resolver process. Attached latest version 002 patch. As previous version patch conflicts to

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Jan 19, 2017 at 4:04 PM, vinayak wrote: > > On 2017/01/16 17:35, Masahiko Sawada wrote: >> >> On Fri, Jan 13, 2017 at 3:48 PM, Masahiko Sawada >> wrote: >>> >>> On Fri, Jan 13, 2017 at 3:20 PM, Ashutosh Bapat >>>

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2017/01/16 17:35, Masahiko Sawada wrote: On Fri, Jan 13, 2017 at 3:48 PM, Masahiko Sawada wrote: On Fri, Jan 13, 2017 at 3:20 PM, Ashutosh Bapat wrote: Long time passed since original patch proposed by Ashutosh, so I explain again

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Jan 13, 2017 at 3:20 PM, Ashutosh Bapat wrote: >>> >> >> Long time passed since original patch proposed by Ashutosh, so I >> explain again about current design and functionality of this feature. >> If you have any question, please feel free to ask. > >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

>> > > Long time passed since original patch proposed by Ashutosh, so I > explain again about current design and functionality of this feature. > If you have any question, please feel free to ask. Thanks for the summary. > > Parameters > == [ snip ] > > Cluster-wide atomic commit >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Dec 23, 2016 at 1:49 AM, Masahiko Sawada wrote: > On Fri, Dec 9, 2016 at 4:02 PM, Masahiko Sawada wrote: >> On Fri, Dec 9, 2016 at 3:02 PM, vinayak >> wrote: >>> On 2016/12/05 14:42, Ashutosh Bapat wrote:

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Dec 9, 2016 at 3:02 PM, vinayak wrote: > On 2016/12/05 14:42, Ashutosh Bapat wrote: >> >> On Mon, Dec 5, 2016 at 11:04 AM, Haribabu Kommi >> wrote: >> >> >> On Fri, Nov 11, 2016 at 5:38 PM, Masahiko Sawada

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/12/05 14:42, Ashutosh Bapat wrote: On Mon, Dec 5, 2016 at 11:04 AM, Haribabu Kommi wrote: On Fri, Nov 11, 2016 at 5:38 PM, Masahiko Sawada wrote: 2PC is a basic building block to support the atomic commit and there are some

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Dec 5, 2016 at 4:42 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Mon, Dec 5, 2016 at 11:04 AM, Haribabu Kommi > wrote: > > > > > > On Fri, Nov 11, 2016 at 5:38 PM, Masahiko Sawada > > wrote: > >> > >> > >> 2PC is a

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Dec 5, 2016 at 11:04 AM, Haribabu Kommi wrote: > > > On Fri, Nov 11, 2016 at 5:38 PM, Masahiko Sawada > wrote: >> >> >> 2PC is a basic building block to support the atomic commit and there >> are some optimizations way in order to reduce

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Nov 11, 2016 at 5:38 PM, Masahiko Sawada wrote: > > 2PC is a basic building block to support the atomic commit and there > are some optimizations way in order to reduce disadvantage of 2PC. As > you mentioned, it's hard to support a single model that would suit >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Nov 2, 2016 at 9:22 PM, Ashutosh Bapat wrote: > On Mon, Oct 31, 2016 at 6:17 AM, Masahiko Sawada > wrote: >> On Fri, Oct 28, 2016 at 3:19 AM, Robert Haas wrote: >>> On Wed, Oct 26, 2016 at 2:00 AM, Masahiko

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Oct 31, 2016 at 6:17 AM, Masahiko Sawada wrote: > On Fri, Oct 28, 2016 at 3:19 AM, Robert Haas wrote: >> On Wed, Oct 26, 2016 at 2:00 AM, Masahiko Sawada >> wrote: >>> I think we can consider the atomic commit and the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Oct 26, 2016 at 2:00 AM, Masahiko Sawada wrote: > I think we can consider the atomic commit and the atomic visibility > separately, and the atomic visibility can build on the top of the > atomic commit. It is true that we can do that, but I'm not sure whether it's

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Oct 21, 2016 at 1:38 AM, Ashutosh Bapat wrote: > Once we have that information, the foreign server can actively poll > the local server to get the status of transaction xid and resolves the > prepared transaction itself. It can go a step further and inform

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Oct 21, 2016 at 2:38 PM, Ashutosh Bapat wrote: > On Wed, Oct 19, 2016 at 9:17 PM, Robert Haas wrote: >> On Thu, Oct 13, 2016 at 7:27 AM, Amit Langote >> wrote: >>> However, when I briefly read the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Oct 19, 2016 at 9:17 PM, Robert Haas wrote: > On Thu, Oct 13, 2016 at 7:27 AM, Amit Langote > wrote: >> However, when I briefly read the description in "Transaction Management in >> the R* Distributed Database Management System (C.

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Oct 19, 2016 at 11:47:25AM -0400, Robert Haas wrote: > It seems to me that the only way to really make this feature robust is > to have a background worker as part of the equation. The background > worker launches at startup and looks around for local state that tells > it whether there

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 13, 2016 at 7:27 AM, Amit Langote wrote: > However, when I briefly read the description in "Transaction Management in > the R* Distributed Database Management System (C. Mohan et al)" [2], it > seems that what Ashutosh is saying might be a correct way to

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 13, 2016 at 7:37 PM, Ashutosh Bapat wrote: >>> >>> If we are successful in COMMITTING foreign transactions during >>> post-commit phase, COMMIT message will be returned after we have >>> committed all foreign transactions. But in case we can not reach

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/10/13 19:37, Ashutosh Bapat wrote: >> In case we can not reach a foreign server during post-commit phase, >> basically the transaction and following transaction should stop until >> the crashed server revived. > > I have repeatedly given reasons why this is not correct. You and Amit >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

>> >> If we are successful in COMMITTING foreign transactions during >> post-commit phase, COMMIT message will be returned after we have >> committed all foreign transactions. But in case we can not reach a >> foreign server, and request times out, we can not revert back our >> decision that we

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Oct 7, 2016 at 4:25 PM, Ashutosh Bapat wrote: > On Thu, Oct 6, 2016 at 2:52 PM, Amit Langote > wrote: >> On 2016/10/06 17:45, Ashutosh Bapat wrote: >>> On Thu, Oct 6, 2016 at 1:34 PM, Masahiko Sawada

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 6, 2016 at 2:52 PM, Amit Langote wrote: > On 2016/10/06 17:45, Ashutosh Bapat wrote: >> On Thu, Oct 6, 2016 at 1:34 PM, Masahiko Sawada >> wrote: >>> On Thu, Oct 6, 2016 at 1:41 PM, Ashutosh Bapat >>>

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/10/06 17:45, Ashutosh Bapat wrote: > On Thu, Oct 6, 2016 at 1:34 PM, Masahiko Sawada wrote: >> On Thu, Oct 6, 2016 at 1:41 PM, Ashutosh Bapat >> wrote: My understanding is that basically the local server can not return

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 6, 2016 at 1:34 PM, Masahiko Sawada wrote: > On Thu, Oct 6, 2016 at 1:41 PM, Ashutosh Bapat > wrote: No, the COMMIT returns after the first phase. It can not wait for all the foreign servers to complete their

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Oct 6, 2016 at 1:41 PM, Ashutosh Bapat wrote: >>> >>> No, the COMMIT returns after the first phase. It can not wait for all >>> the foreign servers to complete their second phase >> >> Hm, it sounds like it's same as normal commit (not 2PC). >> What's the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

>> >> No, the COMMIT returns after the first phase. It can not wait for all >> the foreign servers to complete their second phase > > Hm, it sounds like it's same as normal commit (not 2PC). > What's the difference? > > My understanding is that basically the local server can not return > COMMIT to

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Oct 4, 2016 at 8:29 PM, Ashutosh Bapat wrote: > On Tue, Oct 4, 2016 at 1:11 PM, Amit Langote > wrote: >> On 2016/10/04 16:10, Ashutosh Bapat wrote: > Heuristics can not become the default behavior. A user should be given

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Oct 4, 2016 at 1:11 PM, Amit Langote wrote: > On 2016/10/04 16:10, Ashutosh Bapat wrote: Heuristics can not become the default behavior. A user should be given an option to choose a heuristic, and he should be aware of the pitfalls when using

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/10/04 16:10, Ashutosh Bapat wrote: >>> Heuristics can not become the default behavior. A user should be given >>> an option to choose a heuristic, and he should be aware of the >>> pitfalls when using this heuristic. I guess, first, we need to get a >>> solution which ensures that the

Re: [HACKERS] Transactions involving multiple postgres foreign servers

>> >> Heuristics can not become the default behavior. A user should be given >> an option to choose a heuristic, and he should be aware of the >> pitfalls when using this heuristic. I guess, first, we need to get a >> solution which ensures that the transaction gets committed on all the >> servers

[HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Oct 4, 2016 at 1:26 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com > wrote: >>> >>> Why always rollback any dangling transaction? There can be a case that >>> a foreign server has a dangling transaction which needs to be >>> committed because the portions of that

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Hi, On 2016/10/04 13:26, Ashutosh Bapat wrote: >>> >>> Why always rollback any dangling transaction? There can be a case that >>> a foreign server has a dangling transaction which needs to be >>> committed because the portions of that transaction on the other shards >>> are committed. >> >>

Re: [HACKERS] Transactions involving multiple postgres foreign servers

>> >> Why always rollback any dangling transaction? There can be a case that >> a foreign server has a dangling transaction which needs to be >> committed because the portions of that transaction on the other shards >> are committed. > > Right, we can heuristically make a decision whether we do

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 28, 2016 at 3:30 PM, Ashutosh Bapat wrote: > On Wed, Sep 28, 2016 at 10:43 AM, Masahiko Sawada > wrote: >> On Tue, Sep 27, 2016 at 9:06 PM, Ashutosh Bapat >> wrote: >>> On Tue, Sep 27, 2016 at

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 28, 2016 at 3:30 PM, Ashutosh Bapat wrote: > I agree, but we need to cope with above two problems. I have marked the patch as returned with feedback per the last output Ashutosh has provided. -- Michael -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Wed, Sep 28, 2016 at 10:43 AM, Masahiko Sawada wrote: > On Tue, Sep 27, 2016 at 9:06 PM, Ashutosh Bapat > wrote: >> On Tue, Sep 27, 2016 at 2:54 PM, Masahiko Sawada >> wrote: >>> On Mon, Sep 26, 2016 at 9:07 PM,

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 27, 2016 at 9:06 PM, Ashutosh Bapat wrote: > On Tue, Sep 27, 2016 at 2:54 PM, Masahiko Sawada > wrote: >> On Mon, Sep 26, 2016 at 9:07 PM, Ashutosh Bapat >> wrote: >>> On Mon, Sep 26, 2016 at

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 27, 2016 at 6:24 PM, Masahiko Sawada wrote: > * Providing the prepare id of 2PC. > Current patch adds new API prepare_id_provider() but we can use the > prepare id of 2PC that is used on parent server. And we assume that when this is used across many servers

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Tue, Sep 27, 2016 at 2:54 PM, Masahiko Sawada wrote: > On Mon, Sep 26, 2016 at 9:07 PM, Ashutosh Bapat > wrote: >> On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada >> wrote: >>> On Mon, Sep 26, 2016 at 7:28 PM,

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Sep 26, 2016 at 9:07 PM, Ashutosh Bapat wrote: > On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada > wrote: >> On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat >> wrote: >>> My original patch added

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Sep 26, 2016 at 5:25 PM, Masahiko Sawada wrote: > On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat > wrote: >> My original patch added code to manage the files for 2 phase >> transactions opened by the local server on the remote

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Sep 26, 2016 at 7:28 PM, Ashutosh Bapat wrote: > My original patch added code to manage the files for 2 phase > transactions opened by the local server on the remote servers. This > code was mostly inspired from the code in twophase.c which manages the >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

My original patch added code to manage the files for 2 phase transactions opened by the local server on the remote servers. This code was mostly inspired from the code in twophase.c which manages the file for prepared transactions. The logic to manage 2PC files has changed since [1] and has been

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/09/07 10:54, vinayak wrote: Thanks for the clarification. I had added pg_fdw_xact_resolver() to resolve any transactions which can not be resolved immediately after they were prepared. There was a comment from Kevin (IIRC) that leaving transactions unresolved on the foreign server

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On 2016/08/26 15:13, Ashutosh Bapat wrote: On Fri, Aug 26, 2016 at 11:37 AM, Masahiko Sawada > wrote: On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat >

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Aug 26, 2016 at 3:13 PM, Ashutosh Bapat wrote: > > > On Fri, Aug 26, 2016 at 11:37 AM, Masahiko Sawada > wrote: >> >> On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat >> wrote: >> > >> > >> > On Fri,

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Aug 26, 2016 at 11:37 AM, Masahiko Sawada wrote: > On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat > wrote: > > > > > > On Fri, Aug 26, 2016 at 11:22 AM, Masahiko Sawada > > > wrote: > >> > >> On Fri, Aug

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Aug 26, 2016 at 3:03 PM, Ashutosh Bapat wrote: > > > On Fri, Aug 26, 2016 at 11:22 AM, Masahiko Sawada > wrote: >> >> On Fri, Aug 26, 2016 at 1:32 PM, Vinayak Pokale >> wrote: >> > Hi All, >> > >> > Ashutosh

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Fri, Aug 26, 2016 at 11:22 AM, Masahiko Sawada wrote: > On Fri, Aug 26, 2016 at 1:32 PM, Vinayak Pokale > wrote: > > Hi All, > > > > Ashutosh proposed the feature 2PC for FDW for achieving atomic commits > > across multiple foreign servers. > > If

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Hi All, Ashutosh proposed the feature 2PC for FDW for achieving atomic commits across multiple foreign servers. If a transaction make changes to more than two foreign servers the current implementation in postgres_fdw doesn't make sure that either all of them commit or all of them rollback their

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Alvaro Herrera wrote: > Ashutosh Bapat wrote: > > > Here's updated patch. I didn't use version numbers in file names in my > > previous patches. I am starting from this onwards. > > Um, I tried this patch and it doesn't apply at all. There's a large > number of conflicts. Please update it and

Re: [HACKERS] Transactions involving multiple postgres foreign servers

Ashutosh Bapat wrote: > Here's updated patch. I didn't use version numbers in file names in my > previous patches. I am starting from this onwards. Um, I tried this patch and it doesn't apply at all. There's a large number of conflicts. Please update it and resubmit to the next commitfest. --

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Dec 24, 2015 at 8:32 AM, Michael Paquier wrote: > On Mon, Nov 9, 2015 at 8:55 PM, Ashutosh Bapat > wrote: > > > > > > On Sat, Nov 7, 2015 at 12:07 AM, Robert Haas > wrote: > >> > >> On Wed, Aug 12, 2015

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Thu, Dec 24, 2015 at 7:03 PM, Ashutosh Bapat wrote: > On Thu, Dec 24, 2015 at 8:32 AM, Michael Paquier >> Ashutosh, others, this thread has been stalling for more than 1 month >> and a half. There is a new patch that still applies

Re: [HACKERS] Transactions involving multiple postgres foreign servers

On Mon, Nov 9, 2015 at 8:55 PM, Ashutosh Bapat wrote: > > > On Sat, Nov 7, 2015 at 12:07 AM, Robert Haas wrote: >> >> On Wed, Aug 12, 2015 at 6:25 AM, Ashutosh Bapat >> wrote: >> > The previous patch would

Re: [HACKERS] Transactions involving multiple postgres foreign servers

> Any distributed transaction management requires 2PC in some or other form. > So, we should implement 2PC for FDW keeping in mind various forms of 2PC > used practically. Use that infrastructure to build XTM like capabilities > for restricted postgres_fdw uses. Previously, I have requested the

  1   2   >