Olav, I contributed that processor, and for the life of me I can't remember why it is forcing serial execution. My guess is that it was a limitation of the version of the Hive Streaming libraries. I will try it with multiple concurrent tasks to see if it works, and if so, I will write a Jira to support concurrency for this processor. In either case I will reply here with my findings.
Regards, Matt On Sun, Nov 6, 2016 at 7:18 PM, Joe Witt <[email protected]> wrote: > Olav, > > I'll let someone more familiar with that processor comment on the best > usage pattern but will say that for any processor which does not offer > controls on the number of concurrent tasks then it means that processor is > meant to run serially by design. > > Thanks > Joe > > On Sun, Nov 6, 2016 at 7:06 PM, Olav Jordens <Olav.Jordens@2degreesmobile. > co.nz> wrote: > >> Hi, >> >> >> >> I have noticed that it is not possible to change the number of concurrent >> tasks on the Scheduling tab of the PutHiveStreaming processor from the >> default value of 1. Is there a reason for this, or is it a bug? What would >> be the consequence if I try to get around this by using a DistributeLoad >> processor followed by a sequence of PutHiveStreaming processors, which >> would all be streaming into the same Hive table? >> >> >> >> Thanks, >> >> Olav >> >> >> >> >> Olav Jordens >> Senior ETL Developer >> Two Degrees Mobile Limited >> =========================== >> (M) 022 620 2429 >> (P) 09 919 7000 >> www.2degreesmobile.co.nz >> >> Two Degrees Mobile Limited | 47-49 George Street | Newmarket | Auckland | >> New Zealand | >> PO Box 8355 | Symonds Street | Auckland 1150 | New Zealand | Fax +64 9 >> 919 7001 >> >> >> >> >> ------------------------------ >> >> Disclaimer >> The e-mail and any files transmitted with it are confidential and may >> contain privileged or copyright information. If you are not the intended >> recipient you must not copy, distribute, or use this e-mail or the >> information contained in it for any purpose other than to notify us of the >> error. If you have received this message in error, please notify the sender >> immediately, by email or phone (+64 9 919 7000) and delete this email >> from your system. Any views expressed in this message are those of the >> individual sender, except where the sender specifically states them to be >> the views of Two Degrees Mobile Limited. We do not guarantee that this >> material is free from viruses or any other defects although due care has >> been taken to minimize the risk >> >> >> >> >
