JP mentioned that he would like to use multiple CPUs for his conformation
generation code, which he posted during his talk today. Here's a
parallelization of his code. On my laptop, with 4 processes allocated,
I get a 2.5x speedup. I have an Intel Core i7 with 1 CPU and 4 cores.
It uses the concurrent.futures module which is standard in Python 3.2 and later.
Since RDKit doesn't support Python 3.x, I'm actually using the back port for
Python 2.x, which is available from the Python package index. I use it often. I
even include it as part of the chemfp distribution. I presented some examples of
how to use concurrent.futures at the EuroPython 2012 conference. You can see the
presentation and video at
During the "developing the community" round-table we talked about posting more
code examples to the list, so here it is.
from rdkit import Chem
from rdkit.Chem import AllChem
# Download this from http://pypi.python.org/pypi/futures
from concurrent import futures
## On my machine, it takes 39 seconds with 1 worker and 10 seconds with 4.
## 29.055u 0.102s 0:28.68 101.6% 0+0k 0+3io 0pf+0w
## With 4 threads it takes 11 seconds.
## 34.933u 0.188s 0:10.89 322.4% 0+0k 125+1io 0pf+0w
# (The "u"ser time includes time spend in the children processes.
# The wall-clock time is 28.68 and 10.89 seconds, respectively.)
# This function is called in the subprocess.
# The parameters (molecule and number of conformers) are passed via a Python
def generateconformations(m, n):
m = Chem.AddHs(m)
for id in ids:
# EmbedMultipleConfs returns a Boost-wrapped type which
# cannot be pickled. Convert it to a Python list, which can.
return m, list(ids)
smi_input_file, sdf_output_file = sys.argv[1:3]
n = int(sys.argv)
writer = Chem.SDWriter(sdf_output_file)
suppl = Chem.SmilesMolSupplier(smi_input_file, titleLine=False)
with futures.ProcessPoolExecutor(max_workers=max_workers) as executor:
# Submit a set of asynchronous jobs
jobs = 
for mol in suppl:
job = executor.submit(generateconformations, mol, n)
# Process the job results (in submission order) and save the conformers.
for job in jobs:
mol, ids = job.result()
for id in ids:
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
Rdkit-discuss mailing list