Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-26 Thread Alvaro Herrera
Eric Ridge wrote:

> > Again, you'll probably need to put this low level requirement into
> > context if you want sound advice from this list.
> 
> I'm just thinking out lout here, but the context is likely something
> along the lines of externally storing all transaction ids, and
> periodically asking Postgres if they're
> known-to-be-aborted-by-all-transactions -- one at a time.

I think if you just check the global xmin, then you're going to maintain
a very long list of transactions "potentially running" whenever there
are long-lived transactions (pg_dump, for example).  You could try to
add a TransactionIdIsInProgress() somewhere and discard the xact
downright (by DidCommit and DidAbort) if it returns false.  A single
long-running transaction can keep the global xmin down by hundreds of
millions of Xids.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-26 Thread Robert Haas
On Sun, Oct 22, 2017 at 9:23 PM, Eric Ridge  wrote:
> When sitting inside an extension, and given an arbitrary TransactionId, how 
> can you determine that it aborted/crashed *and* that no other active 
> transaction thinks it is still running?
>
> I've tried to answer this question myself (against the 9.3 sources), and it 
> seems like it's just:
>
> {
>TransactionId oldestXmin = GetOldestXmin (false, false);
>TransactionId xid = 42;
>
>if (TransactionIdPrecedes(xid, oldestXmin) &&
>   !TransactionIdDidCommit(xid) &&
>   !TransactionIdIsInProgress(xid)) /* not even sure this is necessary? */
>{
>   /* xid is aborted/crashed and no active transaction cares */
>}
> }
>
> Can anyone confirm or deny that this is correct?  I feel like it is correct, 
> but I'm no expert.

I think TransactionIdPrecedes(xid, oldestXmin) &&
!TransactionIdDidCommit(xid) is sufficient.  If the transaction ID
precedes oldestXmin, then it's either committed or aborted; no other
state is possible.  If it didn't commit, it aborted, and it is right
to test that using !TransactionIdDidCommit(xid) since commits are
always recorded but aborts are only usually recorded.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-23 Thread Craig Ringer
On 23 October 2017 at 05:44, Eric Ridge  wrote:
>> On Oct 22, 2017, at 3:24 PM, Peter Geoghegan  wrote:

>> Again, you'll probably need to put this low level requirement into
>> context if you want sound advice from this list.
>
> I'm just thinking out lout here, but the context is likely something along 
> the lines of externally storing all transaction ids, and periodically asking 
> Postgres if they're known-to-be-aborted-by-all-transactions -- one at a time.

I think Peter is asking "why?".

-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Eric Ridge
> On Oct 22, 2017, at 3:24 PM, Peter Geoghegan  wrote:
> 
> On Sun, Oct 22, 2017 at 2:19 PM, Eric Ridge  wrote:
>> I'm looking for the status as any concurrent open transaction might see it.  
>> For example, if any concurrent transaction might see it as "in progress", 
>> that's what I'd want returned.  Does that make sense?
> 
> Maybe, but note that that's fundamentally something that can become
> stale immediately. And, just because an MVCC snapshot cannot see a row
> does not mean that it cannot affect its transaction/statement in some
> other way (e.g. unique index enforcement).

Sure, but I don't think I care if it becomes stale immediately.  If I ask "now" 
and PG says "in progress" but it then aborts a few cycles later, I'll just ask 
again sometime in the future.  It's not like a transaction is going to go from 
"aborted" back to "in progress" -- geez, at least I hope not!

I think question I want to answer is "will all active and future transactions 
see this TransationId as aborted at the time they started"?

> Again, you'll probably need to put this low level requirement into
> context if you want sound advice from this list.

I'm just thinking out lout here, but the context is likely something along the 
lines of externally storing all transaction ids, and periodically asking 
Postgres if they're known-to-be-aborted-by-all-transactions -- one at a time.

eric

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Peter Geoghegan
On Sun, Oct 22, 2017 at 2:19 PM, Eric Ridge  wrote:
> I'm looking for the status as any concurrent open transaction might see it.  
> For example, if any concurrent transaction might see it as "in progress", 
> that's what I'd want returned.  Does that make sense?

Maybe, but note that that's fundamentally something that can become
stale immediately. And, just because an MVCC snapshot cannot see a row
does not mean that it cannot affect its transaction/statement in some
other way (e.g. unique index enforcement).

Again, you'll probably need to put this low level requirement into
context if you want sound advice from this list.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Eric Ridge
> On Oct 22, 2017, at 2:50 PM, Jaime Casanova  
> wrote:
> 
> so, what you want is txid_status() [1]... while this is new in v10 you
> can use the code as guide or just migrate to v10 ;)

Oh neat, thanks.  



Doesn't that tell you the status relative to the transaction calling 
txid_status()?  

I'm looking for the status as any concurrent open transaction might see it.  
For example, if any concurrent transaction might see it as "in progress", 
that's what I'd want returned.  Does that make sense?  

That's why I was thinking GetOldestXmin() was the right thing to use rather 
than GetActiveSnapshot()->xmin and also why I thought to check 
TransactionIdPrecedes() first.

I am curious about the lock on ClogTruncationLock... could any of the 
TransactionIdDidXXX calls lie without that lock?  I haven't seen such a thing 
used in the 9.3 sources.  Maybe it's necessary for 10 or maybe I just missed it 
in 9.3?

Thanks for your time!

eric

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Jaime Casanova
On 22 October 2017 at 15:00, Eric Ridge  wrote:
>> On Oct 22, 2017, at 1:50 PM, Peter Geoghegan  wrote:
>>
>> On Sun, Oct 22, 2017 at 12:23 PM, Eric Ridge  wrote:
>>> Can anyone confirm or deny that this is correct?  I feel like it is 
>>> correct, but I'm no expert.
>>
>> What are you going to use the code for? I think that that context is
>> likely to matter here.
>
> I'm not exactly sure yet, but I'm thinking about storing transaction ids 
> externally and then regularly poking Postgres to see which ones are 
> aborted-and-no-longer-considered-visible so I can clean up my external list.
>

so, what you want is txid_status() [1]... while this is new in v10 you
can use the code as guide or just migrate to v10 ;)

[1] 
https://www.postgresql.org/docs/10/static/functions-info.html#functions-txid-snapshot

-- 
Jaime Casanova  www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Eric Ridge
> On Oct 22, 2017, at 1:50 PM, Peter Geoghegan  wrote:
> 
> On Sun, Oct 22, 2017 at 12:23 PM, Eric Ridge  wrote:
>> Can anyone confirm or deny that this is correct?  I feel like it is correct, 
>> but I'm no expert.
> 
> What are you going to use the code for? I think that that context is
> likely to matter here.

I'm not exactly sure yet, but I'm thinking about storing transaction ids 
externally and then regularly poking Postgres to see which ones are 
aborted-and-no-longer-considered-visible so I can clean up my external list.

eric

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How to determine that a TransactionId is really aborted?

2017-10-22 Thread Peter Geoghegan
On Sun, Oct 22, 2017 at 12:23 PM, Eric Ridge  wrote:
> Can anyone confirm or deny that this is correct?  I feel like it is correct, 
> but I'm no expert.

What are you going to use the code for? I think that that context is
likely to matter here.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers