Hi Matt – Thanks for the prompt reply. This processor is perfect for our needs – great job! Olav
From: Matt Burgess [mailto:[email protected]] Sent: Tuesday, 8 November 2016 6:08 a.m. To: [email protected] Subject: Re: PutHiveStreaming has only one concurrent task Olav, I contributed that processor, and for the life of me I can't remember why it is forcing serial execution. My guess is that it was a limitation of the version of the Hive Streaming libraries. I will try it with multiple concurrent tasks to see if it works, and if so, I will write a Jira to support concurrency for this processor. In either case I will reply here with my findings. Regards, Matt On Sun, Nov 6, 2016 at 7:18 PM, Joe Witt <[email protected]<mailto:[email protected]>> wrote: Olav, I'll let someone more familiar with that processor comment on the best usage pattern but will say that for any processor which does not offer controls on the number of concurrent tasks then it means that processor is meant to run serially by design. Thanks Joe On Sun, Nov 6, 2016 at 7:06 PM, Olav Jordens <[email protected]<mailto:[email protected]>> wrote: Hi, I have noticed that it is not possible to change the number of concurrent tasks on the Scheduling tab of the PutHiveStreaming processor from the default value of 1. Is there a reason for this, or is it a bug? What would be the consequence if I try to get around this by using a DistributeLoad processor followed by a sequence of PutHiveStreaming processors, which would all be streaming into the same Hive table? Thanks, Olav Olav Jordens Senior ETL Developer Two Degrees Mobile Limited =========================== (M) 022 620 2429 (P) 09 919 7000 www.2degreesmobile.co.nz<http://www.2degreesmobile.co.nz> [cid:[email protected]] Two Degrees Mobile Limited | 47-49 George Street | Newmarket | Auckland | New Zealand | PO Box 8355 | Symonds Street | Auckland 1150 | New Zealand | Fax +64 9 919 7001<tel:%2B64%209%20919%207001> [cid:[email protected]] [cid:[email protected]] [cid:[email protected]] [cid:[email protected]] ________________________________ Disclaimer The e-mail and any files transmitted with it are confidential and may contain privileged or copyright information. If you are not the intended recipient you must not copy, distribute, or use this e-mail or the information contained in it for any purpose other than to notify us of the error. If you have received this message in error, please notify the sender immediately, by email or phone (+64 9 919 7000<tel:%28%2B64%209%20919%207000>) and delete this email from your system. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Two Degrees Mobile Limited. We do not guarantee that this material is free from viruses or any other defects although due care has been taken to minimize the risk
