Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Gaddam Sai Ram]

Thanks for your responses Michael and Amit Kapila,



Yes, we have included your suggestions in our code and started 
testing to reproduce the same issue. In case we encounter this issue again we 
will get back here.



Regards

G. Sai Ram

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Amit Kapila]

On Wed, Oct 25, 2017 at 8:04 PM, Gaddam Sai Ram
 wrote:
> Thanks for the response,
>
> Can you check if CurTransactionContext is valid at that point?
>
>
> Any Postgres function to check if CurTransactionContext is valid or not?
>
> To see, if this problem is related to CurTransactionContext, can you try to
> populate the list in TopMemoryContext and see if that works.
>
>
> Did you mean TopTransactionContext?
>

No, I mean what I have written.  I suspect in your case
TopTransactionContext will be same as CurTransactionContext because
you don't have any subtransaction.

> As of now, we don't free our dlist. We solely depend on Postgres to free our
> dlist while it frees the TopTransactionContext. But if we do allocate in
> TopMemoryContext, we need to take care of freeing our allocations.
>

Can't we do it temporarily to test?  I am not suggesting to make this
a permanent change rather a way to see the reason of the problem.

> And one more issue is, we found this bug once in all the testing we did. So
> trying to replay this bug seems very difficult.
>

Oh, then it is tricky.  I think there is a good chance that this is
some of your application issues where you probably haven't used memory
context as required, so probably you need to figure out a way to
reproduce this issue, otherwise, it might be difficult to track down
the actual cause.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Michael Paquier]

On Wed, Oct 25, 2017 at 7:34 AM, Gaddam Sai Ram
 wrote:
> Thanks for the response,
>
> Can you check if CurTransactionContext is valid at that point?
>
>
> Any Postgres function to check if CurTransactionContext is valid or not?

Assert(MemoryContextIsValid(CurTransactionContext));
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Gaddam Sai Ram]

Thanks for the response,


Can you check if CurTransactionContext is valid at that point? 








Any Postgres function to check if CurTransactionContext is valid or not?


To see, if this problem is related to CurTransactionContext, can you try to 
populate the list in TopMemoryContext and see if that works.








Did you mean TopTransactionContext? 
As of now, we don't free our dlist. We solely depend on Postgres to free our 
dlist while it frees the TopTransactionContext. But if we do allocate in 
TopMemoryContext, we need to take care of freeing our allocations.


And one more issue is, we found this bug once in all the testing we did. So 
trying to replay this bug seems very difficult.



Regards,

G. Sai Ram

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Amit Kapila]

On Tue, Oct 24, 2017 at 7:42 PM, Gaddam Sai Ram  wrote:

>
> Hello people,
>
> We are implementing in-memory index. As a part of that, during
> index callbacks, under CurTransactionContext, we cache all the DMLs of a
> transaction in dlist(postgres's doubly linked list).
> We registered transaction callback, and in transaction
> pre-commit event(XACT_EVENT_PRE_COMMIT), we iterate through all cached
> DMLs(dlist) and populate in my in-memory data structure.
>
>--> For detailed explanation of our implementation, please look
> into below thread.
>https://www.postgresql.org/message-id/15f4ea99b34.
> e69a4e1b633.8937474039794097334%40zohocorp.com
>
>--> It was working fine until I was greeted with a segmentation
> fault while accessing dlist!
>
>--> On further examining I found that dlist head is not NULL
> and it is pointing to some address, but the container I extracted is
> pointing to 0x7f7f7f7f7f7f7f5f, and I was not able to access any members in
> my container.
>
>--> https://wiki.postgresql.org/wiki/Developer_FAQ#Why_
> are_my_variables_full_of_0x7f_bytes.3F
> From the above wiki, we came to a conclusion that the memory
> context in which that dlist was present was freed.
> And yes CLOBBER_FREED_MEMORY is enabled by passing
> --enable-cassert to enable asserts in my code.
>
>--> Normally the memory context inside XACT_EVENT_PRE_COMMIT is
> TopTransactionContext but in this particular case, the memory context was
> 'MessageContext'. Little unusual! Not sure why!
>
>--> So basically CurTransactionContext shouldn't get freed
> before transaction COMMIT.
>--> So what has actually happened here??? Kindly help us with
> your insights!
>
>
Can you check if CurTransactionContext is valid at that point?  To see, if
this problem is related to CurTransactionContext, can you try to populate
the list in TopMemoryContext and see if that works.


-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Gaddam Sai Ram]

This sounds broken on its face --- if you want stuff to survive to 

top-level commit, you need to keep it in TopTransactionContext. 

CurTransactionContext might be a subtransaction's context that will 

go away at subtransaction commit/abort. 







https://github.com/postgres/postgres/blob/master/src/backend/utils/mmgr/README



>From the above README:



CurTransactionContext --- this holds data that has to survive until the end

of the current transaction, and in particular will be needed at top-level

transaction commit.  When we are in a top-level transaction this is the same

as TopTransactionContext, but in subtransactions it points to a child context.

It is important to understand that if a subtransaction aborts, its

CurTransactionContext is thrown away after finishing the abort processing;

but a committed subtransaction's CurTransactionContext is kept until top-level

commit (unless of course one of the intermediate levels of subtransaction

aborts).  This ensures that we do not keep data from a failed subtransaction

longer than necessary.  Because of this behavior, you must be careful to clean

up properly during subtransaction abort --- the subtransaction's state must be

delinked from any pointers or lists kept in upper transactions, or you will

have dangling pointers leading to a crash at top-level commit.  An example of

data kept here is pending NOTIFY messages, which are sent at top-level commit,

but only if the generating subtransaction did not abort.



-- So even if sub-transaction is committed, subtransaction's 
CurTransactionContext is kept until top-level commit.

-- If sub-transaction is aborted, we handled(clearing the data) it via 
RegisterSubXactCallback().



And in our particular use case(which gave segmentation fault), we didn't issue 
any sub-transaction. It was a single insert transaction.

Even then it resulted in some kind of context free.




Regards

G. Sai Ram

Re: [HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Tom Lane]

Gaddam Sai Ram  writes:
> We are implementing in-memory index. As a part of that, during 
> index callbacks, under CurTransactionContext, we cache all the DMLs of a 
> transaction in dlist(postgres's doubly linked list).
> We registered transaction callback, and in transaction pre-commit 
> event(XACT_EVENT_PRE_COMMIT), we iterate through all cached DMLs(dlist) and 
> populate in my in-memory data structure.

This sounds broken on its face --- if you want stuff to survive to
top-level commit, you need to keep it in TopTransactionContext.
CurTransactionContext might be a subtransaction's context that will
go away at subtransaction commit/abort.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] CurTransactionContext freed before transaction COMMIT ???

[Gaddam Sai Ram]

Hello people,



We are implementing in-memory index. As a part of that, during 
index callbacks, under CurTransactionContext, we cache all the DMLs of a 
transaction in dlist(postgres's doubly linked list).

We registered transaction callback, and in transaction pre-commit 
event(XACT_EVENT_PRE_COMMIT), we iterate through all cached DMLs(dlist) and 
populate in my in-memory data structure.



   -- For detailed explanation of our implementation, please look 
into below thread.






   
https://www.postgresql.org/message-id/15f4ea99b34.e69a4e1b633.8937474039794097334%40zohocorp.com




   -- It was working fine until I was greeted with a segmentation 
fault while accessing dlist!

   

   -- On further examining I found that dlist head is not NULL and 
it is pointing to some address, but the container I extracted is pointing to 
0x7f7f7f7f7f7f7f5f, and I was not able to access any members in my container.



   -- 
https://wiki.postgresql.org/wiki/Developer_FAQ#Why_are_my_variables_full_of_0x7f_bytes.3F

From the above wiki, we came to a conclusion that the memory 
context in which that dlist was present was freed.


And yes CLOBBER_FREED_MEMORY is enabled by passing --enable-cassert 
to enable asserts in my code.



   -- Normally the memory context inside XACT_EVENT_PRE_COMMIT is 
TopTransactionContext but in this particular case, the memory context was 
'MessageContext'. Little unusual! Not sure why!



   -- So basically CurTransactionContext shouldn't get freed before 
transaction COMMIT.

   -- So what has actually happened here??? Kindly help us with 
your insights!



For your reference, a code snippet of our implementation is pasted below:



Sample code snippet:



/*** Code snippet starts ***/



dlist_head DMLInfo = {{NULL, NULL}};




Insert_callback()


{

  old_context = MemoryContextSwitchTo(CurTransactionContext);

  

  GraphIndex_InsertInfo *current;

  current = (GraphIndex_InsertInfo *)palloc(sizeof(GraphIndex_InsertInfo));

  current-tableOid = tableOid;

  current-subTransactionID = GetCurrentSubTransactionId();

  current-colValue = (long)values[0]; // Varies with column type

  current-ctid = ctid;



  dlist_push_head(DMLInfo, current-list_node);

  

  MemoryContextSwitchTo(old_context);

  return TRUE;

}



static void GraphIndex_xactCallback(XactEvent event, void *arg) 

{


  GraphIndex_Table *tableInfo;


  GraphIndex_InsertInfo *current;

  dlist_iter iter;



  switch (event) 

  {

  case XACT_EVENT_PRE_PREPARE:

  case XACT_EVENT_PRE_COMMIT:

  PG_TRY();

  {

if(DMLInfo.head.next)

{

  dlist_reverse_foreach(iter, DMLInfo)

  {

current = 
dlist_container(GraphIndex_InsertInfo, list_node, iter.cur);

DMLProcessingFunction(current); // This is 
giving me the ERROR (while accessing members of current)

  }

  DMLInfo.head.next = NULL;

}

 }

 PG_CATCH();

 {

   /* Do cleanup */

 }

 PG_END_TRY();



 break;



 case XACT_EVENT_ABORT:

 case XACT_EVENT_PARALLEL_ABORT:

  /* No cleanup Necessary(No structure created) */

  break;



 default:

  return;

  }

}





 


 
/*** Code snippet ends ***/



Regards

G. Sai Ram