Dong Li created HAWQ-347:
----------------------------
Summary: Dead Lock Issue when concurrent use catalog and
persistent table.
Key: HAWQ-347
URL: https://issues.apache.org/jira/browse/HAWQ-347
Project: Apache HAWQ
Issue Type: Bug
Components: Transaction
Reporter: Dong Li
Assignee: Lei Chang
------
prepare
------
{code}
create tablespace myts filespace dfs_system;
{code}
----------
Process 1 : use ctrl_c to abort the create transaction
-----------
{code}
create FILESPACE fsinvalid ON hdfs ('localhost:10086/hawq/fsinvalid');
ctrl_C
{code}
---------
Process 2 : during the ctrl_c period, drop a tablespace
---------
{code}
drop TABLESPACE myts;
{code}
************
The lock state:
****************
1.abort (create Filespace)
get lock : Persistent table lock
need lock: pg_filespace lock
2.commit(drop tablespace)
get lock: pg_filespace lock
need lock: Persistent table lock
***********
Debug information
***********
1.debug information for abort (create Filespace) process
(lldb) bt
* thread #1: tid = 0x9781a3, 0x99364f7a libsystem_kernel.dylib`semop + 10,
queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x99364f7a libsystem_kernel.dylib`semop + 10
frame #1: 0x003eb14a postgres`PGSemaphoreLock(sema=0x18548cf8,
interruptOK='\0') + 301 at pg_sema.c:437
frame #2: 0x0045972c postgres`LWLockAcquire(lockid=PersistentObjLock,
mode=LW_EXCLUSIVE) + 799 at lwlock.c:557
frame #3: 0x006cd3a9
postgres`PersistentFilespace_Dropped(fsObjName=0x048f6598,
persistentTid=0x048f65bc, persistentSerialNum=3) + 411 at
cdbpersistentfilespace.c:1079
frame #4: 0x006e03b4
postgres`PersistentFileSysObj_DropObject(fsObjName=0x048f6598,
relStorageMgr=PersistentFileSysRelStorageMgr_None, relationName=0x00000000,
persistentTid=0x048f65bc, persistentSerialNum=3, ignoreNonExistence='\x01',
debugPrint='\0', debugPrintLevel=14) + 2170 at cdbpersistentfilesysobj.c:2090
frame #5: 0x006e04b8
postgres`PersistentFileSysObj_EndXactDrop(fsObjName=0x048f6598,
relStorageMgr=PersistentFileSysRelStorageMgr_None, relationName=0x00000000,
persistentTid=0x048f65bc, persistentSerialNum=3, ignoreNonExistence='\x01') +
113 at cdbpersistentfilesysobj.c:2123
frame #6: 0x004603c2 postgres`smgrDoDeleteActions(list=0x00b40844,
listCount=0x00b40848, forCommit='\0') + 1444 at smgr.c:1527
frame #7: 0x00461258 postgres`AtEOXact_smgr(forCommit='\0') + 226 at
smgr.c:2111
frame #8: 0x00072abf postgres`AbortTransaction + 774 at xact.c:2867
frame #9: 0x0007343b postgres`AbortCurrentTransaction + 183 at xact.c:3289
frame #10: 0x0046ade6 postgres`PostgresMain(argc=4, argv=0x0484b480,
username=0x0484baa8) + 6294 at postgres.c:4487
frame #11: 0x0040c1e9 postgres`BackendRun(port=0x03835c30) + 1008 at
postmaster.c:5875
frame #12: 0x0040b583 postgres`BackendStartup(port=0x03835c30) + 381 at
postmaster.c:5468
frame #13: 0x0040463d postgres`ServerLoop + 1111 at postmaster.c:2147
frame #14: 0x004033ca postgres`PostmasterMain(argc=9, argv=0x03834620) +
5127 at postmaster.c:1439
frame #15: 0x0031aa6b postgres`main(argc=9, argv=0x03834620) + 1007 at
main.c:226
frame #16: 0x0000201d postgres`_start + 212
frame #17: 0x00001f48 postgres`start + 40
--------------------------------------------
2.debug information for commit(drop tablespace) process
* thread #1: tid = 0x97829a, 0x99364f7a libsystem_kernel.dylib`semop + 10,
queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: 0x99364f7a libsystem_kernel.dylib`semop + 10
frame #1: 0x003eb14a postgres`PGSemaphoreLock(sema=0x185492e0,
interruptOK='\x01') + 301 at pg_sema.c:437
frame #2: 0x00456615 postgres`ProcSleep(locallock=0x04847cc8,
lockMethodTable=0x0097c804) + 736 at proc.c:983
frame #3: 0x00451f4b postgres`WaitOnLock(locallock=0x04847cc8,
owner=0x0484d15c) + 428 at lock.c:1351
frame #4: 0x004515a5 postgres`LockAcquire(locktag=0xbfffe788, lockmode=1,
sessionLock='\0', dontWait='\0') + 6297 at lock.c:995
frame #5: 0x0044e98f postgres`LockRelationOid(relid=5009, lockmode=1) + 58
at lmgr.c:102
frame #6: 0x00039686 postgres`relation_open(relationId=5009, lockmode=1) +
105 at heapam.c:880
frame #7: 0x0003a01f postgres`heap_open(relationId=5009, lockmode=1) + 31
at heapam.c:1286
frame #8: 0x0020444c postgres`is_filespace_shared(fsoid=16384) + 29 at
filespace.c:737
frame #9: 0x0009b637 postgres`emit_mmxlog_fs_record(type=MM_OBJ_TABLESPACE,
filespace=16384, tablespace=204983, database=0, relfilenode=0,
persistentTid=0x0509564c, persistentSerialNum=2, segnum=0, flags=' ',
beginLoc=0xbfffed90) + 475 at xlog_mm.c:498
frame #10: 0x0009ba23
postgres`mmxlog_log_remove_tablespace(tablespace=204983,
persistentTid=0x0509564c, persistentSerialNum=2) + 189 at xlog_mm.c:584
frame #11: 0x006dc13a
postgres`PersistentTablespace_DroppedVerifiedActionCallback(fsObjName=0x05095628,
persistentTid=0x0509564c, persistentSerialNum=2,
verifyExpectedResult=PersistentFileSysObjVerifyExpectedResult_StateChangeNeeded)
+ 97 at cdbpersistenttablespace.c:1079
frame #12: 0x006def9a
postgres`PersistentFileSysObj_StateChange(fsObjName=0x05095628,
persistentTid=0x0509564c, persistentSerialNum=2,
nextState=PersistentFileSysState_Free, retryPossible='\0', flushToXLog='\0',
oldState=0xbfffef0c, verifiedActionCallback=0x006dc0d9) + 920 at
cdbpersistentfilesysobj.c:1432
frame #13: 0x006dc485
postgres`PersistentTablespace_Dropped(fsObjName=0x05095628,
persistentTid=0x0509564c, persistentSerialNum=2) + 771 at
cdbpersistenttablespace.c:1142
frame #14: 0x006e02da
postgres`PersistentFileSysObj_DropObject(fsObjName=0x05095628,
relStorageMgr=PersistentFileSysRelStorageMgr_None, relationName=0x00000000,
persistentTid=0x0509564c, persistentSerialNum=2, ignoreNonExistence='\0',
debugPrint='\0', debugPrintLevel=14) + 1952 at cdbpersistentfilesysobj.c:2064
frame #15: 0x006e04b8
postgres`PersistentFileSysObj_EndXactDrop(fsObjName=0x05095628,
relStorageMgr=PersistentFileSysRelStorageMgr_None, relationName=0x00000000,
persistentTid=0x0509564c, persistentSerialNum=2, ignoreNonExistence='\0') + 113
at cdbpersistentfilesysobj.c:2123
frame #16: 0x004603c2 postgres`smgrDoDeleteActions(list=0x00b40844,
listCount=0x00b40848, forCommit='\x01') + 1444 at smgr.c:1527
frame #17: 0x00461258 postgres`AtEOXact_smgr(forCommit='\x01') + 226 at
smgr.c:2111
frame #18: 0x000720f6 postgres`CommitTransaction + 749 at xact.c:2427
frame #19: 0x00072fb6 postgres`CommitTransactionCommand + 310 at xact.c:3056
frame #20: 0x004686b8 postgres`finish_xact_command + 116 at postgres.c:3123
frame #21: 0x00465b6a postgres`exec_simple_query(query_string=0x0507c42c,
seqServerHost=0x00000000, seqServerPort=-1) + 2137 at postgres.c:1780
frame #22: 0x0046b46c postgres`PostgresMain(argc=4, argv=0x0484b480,
username=0x0484baa8) + 7964 at postgres.c:4711
frame #23: 0x0040c1e9 postgres`BackendRun(port=0x03911ea0) + 1008 at
postmaster.c:5875
frame #24: 0x0040b583 postgres`BackendStartup(port=0x03911ea0) + 381 at
postmaster.c:5468
frame #25: 0x0040463d postgres`ServerLoop + 1111 at postmaster.c:2147
frame #26: 0x004033ca postgres`PostmasterMain(argc=9, argv=0x03834620) +
5127 at postmaster.c:1439
frame #27: 0x0031aa6b postgres`main(argc=9, argv=0x03834620) + 1007 at
main.c:226
frame #28: 0x0000201d postgres`_start + 212
frame #29: 0x00001f48 postgres`start + 40
-----------------------
Other information
-----------------------
intern=# select * from pg_stat_activity ;
datid | datname | procpid | sess_id | usesysid | usename |
current_query | waiting |
query_start | backend_start | client_addr |
client_port | application_name | xact_start |
waiting_resource
--------+------------+---------+---------+----------+---------+------------------------------------------------------------------------+---------+-------------------------------+-------------------------------+-------------+-------------+------------------+-------------------------------+------------------
16387 | postgres | 83974 | 65 | 10 | intern | drop TABLESPACE
myts; | t | 2016-01-15
18:48:18.910256+08 | 2016-01-15 18:47:57.794331+08 | | -1
| psql | 2016-01-15 18:48:18.910256+08 | f
205198 | dbindifffs | 83863 | 64 | 10 | intern | create
FILESPACE fsinvalid ON hdfs ('localhost:10086/hawq/fsinvalid'); | f |
2016-01-15 18:47:26.849302+08 | 2016-01-15 18:47:26.790389+08 | |
-1 | psql | 2016-01-15 18:47:26.849302+08 | f
16534 | intern | 89599 | 66 | 10 | intern | select * from
pg_stat_activity ; | f | 2016-01-15
19:25:14.432401+08 | 2016-01-15 19:13:54.205517+08 | | -1
| psql | 2016-01-15 19:25:14.432401+08 | f
(3 rows)
intern=# select * from pg_locks ;
locktype | database | relation | page | tuple | transactionid | classid |
objid | objsubid | transaction | pid | mode | granted |
mppsessionid | mppiswriter | gp_segment_id
---------------+----------+----------+------+-------+---------------+---------+-------+----------+-------------+-------+---------------------+---------+--------------+-------------+---------------
relation | 0 | 5009 | | | | |
| | 0 | 83863 | AccessExclusiveLock | t |
64 | f | -10000
transactionid | | | | | 2133 | |
| | 2133 | 89599 | ExclusiveLock | t |
66 | f | -10000
relation | 0 | 1213 | | | | |
| | 0 | 83974 | RowExclusiveLock | t |
65 | f | -10000
transactionid | | | | | 2118 | |
| | 0 | 83863 | ExclusiveLock | t |
64 | f | -10000
relation | 0 | 5009 | | | | |
| | 0 | 83974 | AccessShareLock | f |
65 | f | -10000
relation | 16534 | 10336 | | | | |
| | 2133 | 89599 | AccessShareLock | t |
66 | f | -10000
transactionid | | | | | 2124 | |
| | 0 | 83974 | ExclusiveLock | t |
65 | f | -10000
(7 rows)
intern=# select * from pg_class where oid=5009;
relname | relnamespace | reltype | relowner | relam | relfilenode |
reltablespace | relpages | reltuples | reltoastrelid | reltoastidxid |
relaosegrelid | relaosegidxid | relhasindex | relisshared | relkind |
relstorage | relnatts | relchecks | reltriggers | relukeys | relfkeys | relrefs
| relhasoids | relhaspkey | relhasrules | relhassubclass | relfrozenxid |
relacl | reloptions
--------------+--------------+---------+----------+-------+-------------+---------------+----------+-----------+---------------+---------------+---------------+---------------+-------------+-------------+---------+------------+----------+-----------+-------------+----------+----------+---------+------------+------------+-------------+----------------+--------------+-------------+------------
pg_filespace | 11 | 6438 | 10 | 0 | 5009 |
1664 | 1 | 1 | 0 | 0 | 0
| 0 | t | t | r | h | 4 |
0 | 0 | 0 | 0 | 0 | t | f
| f | f | 903 | {=r/intern} |
(1 row)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
