Suresh Subbiah created TRAFODION-2226:
-----------------------------------------

             Summary: Memory leak from Repository thread
                 Key: TRAFODION-2226
                 URL: https://issues.apache.org/jira/browse/TRAFODION-2226
             Project: Apache Trafodion
          Issue Type: Bug
          Components: connectivity-mxosrvr
    Affects Versions: 2.0-incubating
            Reporter: Suresh Subbiah
            Assignee: Arvind Narain
             Fix For: 2.1-incubating


An mxosrvr process which has been connected to and seen at least one SQL 
statement given by a user executed will leak memory from the repository thread, 
The rate of observed leak was about 3 GB during one month. The rate does depend 
on the amount of activity in the repository thread which in turn depends on the 
activity in the main thread. In this case the repository thread was 
periodically executing 2 statements. There was another observed instance of 
about a 2GB leak after one month.

This is top output
55682 trafodio  20   0 4033m 2.9g  33m S  0.3  2.4 157:16.88 mxosrvr   

With gdb we see that the the statement heap of the repository context is leaking
*cli_globals->executorMemory_->memoryList_->nextEntry_->nextEntry_->memoryList_
$104 = {
  <NABasicObject> = {
    _vptr.NABasicObject = 0x7f53493fca30, 
    h_ = 0x7f53276c1750
  }, 
  members of NAMemory: 
  name_ = "Statement Heap\000\000\001\000\000\000\002", 
  type_ = NAMemory::DERIVED_MEMORY, 
  initialSize_ = 8096, 
  maximumSize_ = 18446744073709551615, 
  incrementSize_ = 4194304, 
  parent_ = 0x7f533a92c1f0, 
  firstBlk_ = 0x7f525ede1060, 
  allocSize_ = 2393256448, 
  upperLimit_ = 0, 
  highWaterMark_ = 2393526936, 
  intervalWaterMark_ = 2393526936, 
  allocCnt_ = 458873, 
  totalSize_ = 2393389344, 
  blockCnt_ = 648, 
  thBlockCnt_ = 110, 


Memory is allocated with every execution (once a minute), but rarely 
deallocated.

NAHeap::allocateBlock (this=0x7f3c7f3dbe70, size=32848, failureIsFatal=1) at 
../common/NAMemory.cpp:2535
2535      p->segmentId_ = segmentId;
(gdb) bt
#0  NAHeap::allocateBlock (this=0x7f3c7f3dbe70, size=32848, failureIsFatal=1) 
at ../common/NAMemory.cpp:2535
#1  0x00007f3ca6b2105f in NAHeap::allocateHeapMemory (this=0x7f3c7f3dbe70, 
userSize=32767, failureIsFatal=1)
    at ../common/NAMemory.cpp:3360
#2  0x00007f3ca6b1d025 in NAMemory::allocateMemory (this=0x7f3c7f3dbe70, 
size=32767, failureIsFatal=1)
    at ../common/NAMemory.cpp:1380
#3  0x00007f3ca39cc9e1 in operator new[] (t=32767, h=0x7f3c7f3dbe70) at 
../export/NABasicObject.cpp:375
#4  0x00007f3ca575e327 in 
ExHbaseUMDtrafUniqueTaskTcb::ExHbaseUMDtrafUniqueTaskTcb (this=0x7f3c7f4009b8, 
tcb=0x7f3c7f3dea68)
    at ../executor/ExHbaseIUD.cpp:1758
#5  0x00007f3ca5763c68 in ExHbaseAccessUMDTcb::ExHbaseAccessUMDTcb 
(this=0x7f3c7f3dea68, hbaseAccessTdb=..., glob=
    0x7f3c7f3dc9a0) at ../executor/ExHbaseIUD.cpp:3520
#6  0x00007f3ca573beef in ExHbaseAccessTdb::build (this=0x7f3c7f3ec928, 
glob=0x7f3c7f3dc9a0)
    at ../executor/ExHbaseAccess.cpp:82
#7  0x00007f3ca563c193 in ex_root_tdb::build (this=0x7f3c7f3e81e8, 
cliGlobals=0x1378240, glob=0x7f3c7f3dc9a0)
    at ../executor/ex_root.cpp:209
#8  0x00007f3ca6fb27ac in CliStatement::fixup (this=0x7f3c7f3dbdb0, 
cliGlobals=0x1378240, input_desc=0x0, diagsArea=..., 
    doSimCheck=@0x7f3c7fe3973c, partitionUnavailable=@0x7f3c7fe3975c, 
donePrepare=0) at ../cli/Statement.cpp:2929
#9  0x00007f3ca6fb518a in CliStatement::execute (this=0x7f3c7f3dbdb0, 
cliGlobals=0x1378240, input_desc=0x0, diagsArea=..., 
    execute_state=CliStatement::INITIAL_STATE_, fixupOnly=0, cliflags=0) at 
../cli/Statement.cpp:3966
#10 0x00007f3ca6f30324 in SQLCLI_PerformTasks(CliGlobals *, ULng32, SQLSTMT_ID 
*, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, typedef __va_list_tag 
__va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) (cliGlobals=0x1378240, 
tasks=606, 
    statement_id=0x2589398, input_descriptor=0x0, output_descriptor=0x0, 
num_input_ptr_pairs=0, num_output_ptr_pairs=0, 
    ap=0x7f3c7fe39da0, input_ptr_pairs=0x0, output_ptr_pairs=0x0) at 
../cli/Cli.cpp:3301
#11 0x00007f3ca6f31426 in SQLCLI_ExecDirect2(CliGlobals *, SQLSTMT_ID *, 
SQLDESC_ID *, Int32, SQLDESC_ID *, Lng32, typedef __va_list_tag __va_list_tag 
*, SQLCLI_PTR_PAIRS *) (cliGlobals=0x1378240, statement_id=0x2589398, 
sql_source=0x7f3c7fe39ff0, 
    prepFlags=0, input_descriptor=0x0, num_ptr_pairs=0, ap=0x7f3c7fe39da0, 
ptr_pairs=0x0) at ../cli/Cli.cpp:3735
#12 0x00007f3ca6fcd8eb in SQL_EXEC_ExecDirect2 (statement_id=0x2589398, 
sql_source=0x7f3c7fe39ff0, prep_flags=0, 
    input_descriptor=0x0, num_ptr_pairs=0) at ../cli/CliExtern.cpp:2327
#13 0x00007f3ca9ad5445 in SRVR::WSQL_EXEC_ExecDirect (statement_id=0x2589398, 
sql_source=0x7f3c7fe39ff0, 
    input_descriptor=0x0, num_ptr_pairs=0) at SQLWrapper.cpp:363
#14 0x00007f3ca9abc1d0 in SRVR::EXECDIRECT (pSrvrStmt=0x2588d80) at 
sqlinterface.cpp:4671
#15 0x00007f3ca9a47ef7 in SRVR::ControlProc (pParam=0x2588d80) at 
csrvrstmt.cpp:768
#16 0x00007f3ca9a47535 in SRVR_STMT_HDL::ExecDirect (this=0x2588d80, 
inCursorName=0x0, 
    inSqlString=0x268d078 "update Trafodion.\"_REPOS_\".metric_query_aggr_table 
set AGGREGATION_LAST_UPDATE_UTC_TS = 
CONVERTTIMESTAMP(212340712811423977),AGGREGATION_LAST_ELAPSED_TIME = 
60000,TOTAL_EST_ROWS_ACCESSED = 0,TOTAL_EST"..., inStmtType=1, 
    inSqlStmtType=0, inSqlAsyncEnable=0, inQueryTimeout=0) at csrvrstmt.cpp:450
#17 0x0000000000578eaa in SessionWatchDog (arg=0x0) at SrvrConnect.cpp:974





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to