Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/12112 )
Change subject: [util] use lighter locking scheme for ThreadMgr ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/12112/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12112/5//COMMIT_MSG@24 PS5, Line 24: might be related : to the fact that process and thread management exercised by python : multiprocessing seems a bit weird > Those as well might be some implications related to the way how the child m Yep, indeed -- the issue was related to the way how workers are spawned in Python multiprocessing. I have a repro scenario which works on other OS as well. Since those workers are spawned by fork(), the thread registry of already opened Kudu client copied over into the spawned processes. Later on, the identifiers of threads coincides with the identifiers used in the parent process -- the records for the latter threads are in the ThreadMgr registry which was copied over by fork() while creating every worker in the multiprocessing pool, so an attempt to insert a duplicate record into the registry triggers an assert. In Python 3 that has been resolved by using context and additional modes (e.g., 'forkserver') that can be used to resolve the problem. -- To view, visit http://gerrit.cloudera.org:8080/12112 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d49c1c39392e01c45019844430a4fe3d116c277 Gerrit-Change-Number: 12112 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Will Berkeley <wdberke...@gmail.com> Gerrit-Comment-Date: Tue, 25 Dec 2018 23:42:51 +0000 Gerrit-HasComments: Yes
