[jira] [Commented] (TRAFODION-2879) Core dump due to reference to deallocated memory for EstLogProp object

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322643#comment-16322643
 ] 

ASF GitHub Bot commented on TRAFODION-2879:
---

Github user asfgit closed the pull request at:

https://github.com/apache/trafodion/pull/1384


> Core dump due to reference to deallocated memory for EstLogProp object
> --
>
> Key: TRAFODION-2879
> URL: https://issues.apache.org/jira/browse/TRAFODION-2879
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: sql-cmp
>Affects Versions: 2.0-incubating
> Environment: Any
>Reporter: Hans Zeller
>Assignee: Hans Zeller
> Fix For: 2.3
>
>
> This looks like an old bug that was recently found by Eric Owhadi, with a 
> stack trace that looks like this:
> {noformat}
> #5  0x7f23af4264fa in NAHeap::deallocateHeapMemory (this=0x7f237ddd66c0, 
> addr=0x7f236f70fba8) at ../common/NAMemory.cpp:3230
> #6  0x7f23af42207b in NAMemory::deallocateMemory (this=0x7f237ddd66c0, 
> addr=0x7f236f70fba8) at ../common/NAMemory.cpp:1265
> #7  0x7f23ada97cdd in NABasicObject::operator delete (p=0x7f236f70fba8)
> at ../export/NABasicObject.cpp:139
> #8  0x7f23ac37fa24 in EstLogProp::~EstLogProp (this=0x7f236f70fba8, 
> __in_chrg=) at ../optimizer/EstLogProp.cpp:130
> #9  0x7f23ac38117e in IntrusiveSharedRefCount::destroyObjects 
> (
> this=0x7f236f70fbb8) at ../common/SharedPtr.h:226
> #10 0x7f23ac1ce7de in SharedRefCountBase::decrUseCount (
> this=0x7f236f70fbb8) at ../common/SharedPtr.h:136
> #11 0x7f23ac1cb845 in SharedPtr::~SharedPtr (
> ---Type  to continue, or q  to quit---
> this=0x7f2382e8e2d0, __in_chrg=)
> at ../common/SharedPtr.h:280
> #12 0x7f23ac1c591e in IntrusiveSharedPtr::~IntrusiveSharedPtr 
> (
> this=0x7f2382e8e2d0, __in_chrg=)
> at ../common/SharedPtr.h:376
> #13 0x7f23ab92d85a in CmpContext::~CmpContext (this=0x7f2382e8e090, 
> __in_chrg=) at ../arkcmp/CmpContext.cpp:382
> {noformat}
> I suspect that we are trying to delete the CmpContext::emptyInLogProp_ data 
> member here.
>  
> I think this data member is initialized incorrectly:
>  
> - The CmpContext object lives longer than an individual statement
> - Initially, CmpContext::emptyInLogProp_ is a NULL pointer
> - For every statement (or at least for most statements), we set this data 
> member to an object allocated from the statement heap, see 
> CmpStatement::initQueryAnalysis() in file arkcmp/CmpStatement.cpp.
> - We don’t deallocate this data member explicitly, as far as I can see. It 
> gets automatically deleted when we call the destructor or when we assign a 
> different value to the IntrusiveSharedPtr.
>  
> So, what happens is that when we destroy the statement and its heap, we leave 
> a dangling reference to a deallocated object in the CmpContext. As we can see 
> here, when we later destroy the context, we point to an object in deallocated 
> memory.
> We need to do two things to fix this: First, with these shared pointers, we 
> cannot allocate an object from the statement heap without calling the 
> destructor for the object. Second, we cannot point from the CmpContext to an 
> object that is allocated from the short-lived statement heap - at least not 
> beyond the lifetime of the statement.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TRAFODION-2879) Core dump due to reference to deallocated memory for EstLogProp object

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TRAFODION-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319298#comment-16319298
 ] 

ASF GitHub Bot commented on TRAFODION-2879:
---

GitHub user zellerh opened a pull request:

https://github.com/apache/trafodion/pull/1384

[TRAFODION-2879] Core dump ...plus four more fixes, see below

This is a PR for four fixes, submitted as separate commits. I would 
recommend reviewing each commit separately, as the fixes are not related.

Summary (also in the commit text for each fix):

**[TRAFODION-2879] Core dump due to reference to deallocated memory for 
EstLogProp object**

In the optimizer, we often use a define to refer to an "empty input
logical properties" object. The problem was that this object was
allocated on the statement heap but it was owned by the CmpContext
object, which lives longer than the statement heap. During debugging,
we have seen cases where this led to references of deleted memory.

The fix is to move the empty logical properties to the CmpStatement
object.

Files: arkcmp/*
   RelExpr.h

**[TRAFODION-2896] Internal error in stored procedures when a warning is 
generated in SQL**

This problem was seen by Any on a large cluster and Selva found this issue
during debugging. At this point we don't have an easy testcase to reproduce,
but we agree that the code change is something we should do.

udrserv/*

**[TRAFODION-2897] Invalid name in MySQL log file when using 
install_local_hadoop**

Bug reported on the dev list.

install_local_hadoop


**Other changes:**

- Fixes in the OSIM tool related to external Hive tables. Issue an 
unregister
  command before creating a new Hive table in OSIM simulation mode.
  Also get the REGISTER commands from the OSIM files and send them to
  Trafodion, not to Hive, as before. Finally, make some minor changes
  to showddl (avoid extra commas, fix comments that showed REGISTER
  as a Hive command).

CmpDescribe.cpp
OptimizerSimulator.cpp

- Cosmetic changes to JDBC TMUDF

JDBCUDR.java


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zellerh/trafodion bug/R23

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/trafodion/pull/1384.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1384


commit bb4edd48c909e889297ec0b0940d9d60745fd86c
Author: Hans Zeller 
Date:   2018-01-09T22:07:09Z

[TRAFODION-2897] Invalid name in MySQL log file when using 
install_local_hadoop

Bug reported on the dev list. MySQL does not understand shell syntax.

commit 90f0b665d54f6b04486e290afd118335167b1d18
Author: Hans Zeller 
Date:   2018-01-09T22:09:29Z

[TRAFODION-2879] Core dump due to reference to deallocated memory for 
EstLogProp object

In the optimizer, we often use a define to refer to an "empty input
logical properties" object. The problem was that this object was
allocated on the statement heap but it was owned by the CmpContext
object, which lives longer than the statement heap. During debugging,
we have seen cases where this led to references of deleted memory.

The fix is to move the empty logical properties to the CmpStatement
object.

commit 4d521af7799983c44f93e9ed46ed48c8871f7993
Author: Hans Zeller 
Date:   2018-01-09T22:10:33Z

[TRAFODION-2896] Internal error in stored procedures when a warning is 
generated in SQL

This problem was seen by Any on a large cluster and Selva found this issue
during debugging. At this point we don't have an easy testcase to reproduce,
but we agree that the code change is something we should do.

commit 40b1773401ac7bb323114f3cf180ccbfec1e8caa
Author: Hans Zeller 
Date:   2018-01-09T22:11:37Z

OSIM changes and comment fix for JDBCUDR

Two small changes w/o JIRA.

Fixes in the OSIM tool related to external Hive tables. Issue an
unregister command before creating a new Hive table in OSIM simulation
mode.  Also get the REGISTER commands from the OSIM files and send
them to Trafodion, not to Hive, as before. Finally, make some minor
changes to showddl (avoid extra commas, fix comments that showed
REGISTER as a Hive command).

Cosmetic changes in the JDBC UDR (fix sandbox location in comment and
use a utility method to get to the sandbox location instead of
hard-coding it.).




> Core dump due to reference to deallocated memory for EstLogProp object
> --
>
>