Am 10.06.19 um 23:00 schrieb Julia Lawall: > > > On Sun, 9 Jun 2019, Markus Elfring wrote: > >> Hello, >> >> The Coccinelle software supports also a variant of parallel data processing >> after the parameter “--jobs” was passed. >> https://github.com/coccinelle/coccinelle/blob/7ec31ed1fadf738bc487ccefdc63bfe0598f44cc/docs/manual/spatch_options.tex#L745 >> >> It is nice when it works to distribute analysis on source files to some >> processors. >> Unfortunately, undesirable software behaviour can be observed if a database >> like “PostgreSQL 11.3-7.1” would be used in such a system configuration. >> Thus I imagine that it can be occasionally appropriate to perform desired >> parallel data processing by the usage of a detailed pipeline instead. >> https://ocaml.github.io/ocamlunix/pipes.html >> >> How do you think about another application for this kind of >> inter-process communication? > > When you use -j,
with a value bigger than one > you fork processes that thus the child processes write to > a different region of memory than the parent process. This can be. But such forked processes have got access also to copied data from the the parent process. > I don't have any interest in connecting Coccinelle to a database, I am curious if any other users of your software can adjust the corresponding development interests. > so I'm not going to spend any time on debugging your issue. I hope that collateral software evolution can happen if more advices will be taken better into account from existing software documentation. https://docs.sqlalchemy.org/en/13/core/pooling.html#using-connection-pools-with-multiprocessing The data storage is working already if known database connections would be performed only by a single process (instead of background processes). > Maybe you can use a hashtable, I would find an other data structure more helpful for the storage of analysis results from parallel source code processing. > then use merge in a finalize, Thanks for such feedback. > and the write the information to the database in the parent thread. This is also my software development idea for working with a detailed process pipeline. > The use of merge is illustrated in eg tests/merge_vars.cocci I would about the detail that a test source file is empty here. https://github.com/coccinelle/coccinelle/commit/0207001b5241092e7b22cf4f99b22db13c01a24d#diff-27862a1fe13b63e871b358e9fc54b3b7 I find the usage of Coccinelle's merge functionality unclear at the moment. Can such an area become better documented? Regards, Markus _______________________________________________ Cocci mailing list [email protected] https://systeme.lip6.fr/mailman/listinfo/cocci
