Re: [Rdkit-discuss] Modified Mol objects with concurrent.futures
Hi Michael, The problem occurs because child processes return their results using pickle, and the ordinary rdkit molecule object when is being pickled it looses information. A solution that I use is to convert the molecule objects to PropertyMol objects, which retain their properties. Best, Christos Christos Kannas Researcher Ph.D Student Mob (UK): +44 (0) 7447700937 Mob (Cyprus): +357 99530608 [image: View Christos Kannas's profile on LinkedIn] http://cy.linkedin.com/in/christoskannas On 2 February 2015 at 09:03, Reutlinger, Michael michael.reutlin...@roche.com wrote: Hi all, I am currently trying to parallelize part of a script using RDKIT and concurrent.futures. The function that is executed in parallel returns processed molecules as RDKIT Mol objects. Without parallelization everything is fine and the Mol objects keep all the properties that they had before the processing. When using concurrent.futures, the returned molecules lose all properties and seem to be created from scratch maybe with unknown side-effects. I am wondering if anyone experienced the same issue and knows how to circumvent this. I attached a ipython notebook with a small script demonstrating the issue. Best, Michael Example Code: from concurrent import futures from rdkit import Chem from rdkit.Chem import AllChem from rdkit.Chem.Draw import IPythonConsole def process(mol): if not Name in mol.GetPropNames(): print Processing: Name missing mol.SetProp(Processed,True) return mol mol = Chem.MolFromSmiles(N[C@@H](C)C(=O)O) mol.SetProp(Name,Alanine) with futures.ProcessPoolExecutor(max_workers=1) as pool: future = pool.submit(process, mol) molOut = future.result() if Name not in molOut.GetPropNames(): print Result: Name missing if Processed not in molOut.GetPropNames(): print Result: Processed missing -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Rdkit-discuss mailing list Rdkit-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rdkit-discuss -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Rdkit-discuss mailing list Rdkit-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rdkit-discuss
[Rdkit-discuss] Modified Mol objects with concurrent.futures
Hi all, I am currently trying to parallelize part of a script using RDKIT and concurrent.futures. The function that is executed in parallel returns processed molecules as RDKIT Mol objects. Without parallelization everything is fine and the Mol objects keep all the properties that they had before the processing. When using concurrent.futures, the returned molecules lose all properties and seem to be created from scratch maybe with unknown side-effects. I am wondering if anyone experienced the same issue and knows how to circumvent this. I attached a ipython notebook with a small script demonstrating the issue. Best, Michael Example Code: from concurrent import futures from rdkit import Chem from rdkit.Chem import AllChem from rdkit.Chem.Draw import IPythonConsole def process(mol): if not Name in mol.GetPropNames(): print Processing: Name missing mol.SetProp(Processed,True) return mol mol = Chem.MolFromSmiles(N[C@@H](C)C(=O)O) mol.SetProp(Name,Alanine) with futures.ProcessPoolExecutor(max_workers=1) as pool: future = pool.submit(process, mol) molOut = future.result() if Name not in molOut.GetPropNames(): print Result: Name missing if Processed not in molOut.GetPropNames(): print Result: Processed missing RDKIT_ParallelProblem (1).ipynb Description: Binary data -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Rdkit-discuss mailing list Rdkit-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rdkit-discuss
Re: [Rdkit-discuss] Modified Mol objects with concurrent.futures
Hi Christos, thanks for pointing out the pickle issue and the solution using PropertyMol. After reading the documentation this should definitely solve the problem. Best, Michael Michael Reutlinger, PhD Scientist, Molecular Design and Chemical Biology Roche Pharma Research and Early Development Roche Innovation Center Basel F. Hoffmann-La Roche Ltd Grenzacherstrasse 124 4070 Basel Switzerland Phone +41 61 688 87 95 Fax +41 61 688 64 59 *Confidentiality Note:* This message is intended only for the use of the named recipient(s) and may contain confidential and/or proprietary information. If you are not the intended recipient, please contact the sender and delete this message. Any unauthorized use of the information contained in this message is prohibited. On Mon, Feb 2, 2015 at 11:17 AM, Christos Kannas chriskan...@gmail.com wrote: Hi Michael, The problem occurs because child processes return their results using pickle, and the ordinary rdkit molecule object when is being pickled it looses information. A solution that I use is to convert the molecule objects to PropertyMol objects, which retain their properties. Best, Christos Christos Kannas Researcher Ph.D Student Mob (UK): +44 (0) 7447700937 Mob (Cyprus): +357 99530608 [image: View Christos Kannas's profile on LinkedIn] http://cy.linkedin.com/in/christoskannas On 2 February 2015 at 09:03, Reutlinger, Michael michael.reutlin...@roche.com wrote: Hi all, I am currently trying to parallelize part of a script using RDKIT and concurrent.futures. The function that is executed in parallel returns processed molecules as RDKIT Mol objects. Without parallelization everything is fine and the Mol objects keep all the properties that they had before the processing. When using concurrent.futures, the returned molecules lose all properties and seem to be created from scratch maybe with unknown side-effects. I am wondering if anyone experienced the same issue and knows how to circumvent this. I attached a ipython notebook with a small script demonstrating the issue. Best, Michael Example Code: from concurrent import futures from rdkit import Chem from rdkit.Chem import AllChem from rdkit.Chem.Draw import IPythonConsole def process(mol): if not Name in mol.GetPropNames(): print Processing: Name missing mol.SetProp(Processed,True) return mol mol = Chem.MolFromSmiles(N[C@@H](C)C(=O)O) mol.SetProp(Name,Alanine) with futures.ProcessPoolExecutor(max_workers=1) as pool: future = pool.submit(process, mol) molOut = future.result() if Name not in molOut.GetPropNames(): print Result: Name missing if Processed not in molOut.GetPropNames(): print Result: Processed missing -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ ___ Rdkit-discuss mailing list Rdkit-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rdkit-discuss -- Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/___ Rdkit-discuss mailing list Rdkit-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rdkit-discuss
