2h4dl commented on issue #80: multiprocessing pool hangs on join URL: https://github.com/apache/rocketmq-client-python/issues/80#issuecomment-569236738 Here is an example: ``` import multiprocessing as mp from rocketmq.client import Producer from rocketmq.client import PushConsumer from rocketmq.client import Message def message_parse(message_body): ''' Deserialize message content and return a list from this message ''' ... ... return a_list def worker(task): """ Every task is cpu bound. """ try: ''' process ''' except Exception as e: pass return process_result def process(message_body): task_list = message_parse(message_body) running_pool = [] pool = mp.Pool(processes=5) for task in task_list: _result = pool.apply_async(worker, args=(task)) running_pool.append(_result) pool.close() pool.join() result_list = [] for p in running_pool: try: result = p.get(timeout=5) result_list.append(result) except: pass return str(result) def callback(rocketmq_msg): msg_body = process(rocketmq_msg.body) msg = Message(rocketmq_msg.topic) msg.set_keys(rocketmq_msg.keys) msg.set_body(msg_body) producer.send_sync(msg) ``` May this can help.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
