Thanks. Sounds reasonable, perhaps blocking on local fifo or a socket would do. Cheers,
[http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] Nastooh Avessta ENGINEER.SOFTWARE ENGINEERING nave...@cisco.com Phone: +1 604 647 1527 Cisco Systems Limited 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121 VANCOUVER BRITISH COLUMBIA V7X 1J1 CA Cisco.com<http://www.cisco.com/> [Think before you print.]Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON, Canada, M5J 2T3. Phone: 416-306-7000; Fax: 416-306-7099. Preferences<http://www.cisco.com/offer/subscribe/?sid=000478326> - Unsubscribe<http://www.cisco.com/offer/unsubscribe/?sid=000478327> – Privacy<http://www.cisco.com/web/siteassets/legal/privacy.html> From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Thursday, April 30, 2015 1:16 AM To: Nastooh Avessta (navesta) Cc: user@spark.apache.org Subject: Re: External Application Run Status One way you could try would be, Inside the map, you can have a synchronized thread and you can block the map till the thread finishes up processing. Thanks Best Regards On Wed, Apr 29, 2015 at 9:38 AM, Nastooh Avessta (navesta) <nave...@cisco.com<mailto:nave...@cisco.com>> wrote: Hi In a multi-node setup, I am invoking a number external apps, through Runtime.getRuntime.exec from rdd.map function, and would like to track their completion status. Evidently, such calls spawn a separate thread, which is not tracked by the standalone scheduler, i.e., reduce or collect are called prior to completion of these tasks. I am wondering if : 1- there is way to delay calls to reduce/collect, till their respective external app threads are completed 2- and if not, is it possible to query the node id and pid of such threads Cheers, [http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] Nastooh Avessta ENGINEER.SOFTWARE ENGINEERING nave...@cisco.com<mailto:nave...@cisco.com> Phone: +1 604 647 1527 Cisco Systems Limited 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121 VANCOUVER BRITISH COLUMBIA V7X 1J1 CA Cisco.com<http://www.cisco.com/> [Think before you print.]Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON, Canada, M5J 2T3. Phone: 416-306-7000; Fax: 416-306-7099. Preferences<http://www.cisco.com/offer/subscribe/?sid=000478326> - Unsubscribe<http://www.cisco.com/offer/unsubscribe/?sid=000478327> – Privacy<http://www.cisco.com/web/siteassets/legal/privacy.html>