Thanks for the reply Ruslan. I was guessing it wouldn't be possible, but thought I would ask incase there was an easy way to do it.I hadn't read about Oozie - looks useful!
Regards, John. -----Original Message----- From: Ruslan Al-Fakikh [mailto:[email protected]] Sent: 17 April 2013 18:09 To: [email protected] Subject: Re: Stopping after load with no tuples? Hi I haven't heard of such functionality in Pig, but it seems that you are looking for a functionality that is not what Pig is for. It lacks some control things like IF ELSE statements, because it is not a general-purpose language. There are a couple of ways you can do such things: 1) Launch Pig with a bash wrapper. 2) Embed Pig into a Java or Python, etc (just like you would embed SQL into a regular language). Pig supports it out of the box 3) Use Oozie or something similar for the jobs orchestration. Hope that helps Ruslan Al-Fakikh On Tue, Apr 9, 2013 at 5:28 PM, John Farrelly < [email protected]> wrote: > Hi there, > > I have a pig script similar to the one below. When testing this on a > cluster with an empty file, I see it taking ages to complete - It goes > through all the commands and runs the jobs across the tracker. In our > test environment, we will (for various reasons) have an input > directory with a file which is empty. Is there any way to tell pig to > exit the script if there is no data available (or a command like EXIT > IF data EMPTY?). My UDF loader does return null as the tuple when it > reads an empty file, so ideally I'd like it to stop processing. Is there any > way to do this? > > Thanks, > John. > > define MyDataLoader com.example.test.MyDataLoader('config1'); > raw = LOAD '$inputdir' USING MyDataLoader AS (date:chararray, > values:bag{t:(policy:chararray)}); > > data1 = FOREACH raw GENERATE date, ..... > data2 = GROUP data1 BY (date, ..... > data3 = FOREACH data3 GENERATE group.date, ..... > > STORE data3 INTO 'MyDataStore' > </pre>**************************************************************** > ************************<br>This email and any files transmitted with > are confidential and intended solely for the<br>use of the individual > or entity to whom they are addressed. If you have received > this<br>email in error then please delete it and notify the sender. Do > not make a copy or forward<br>it to anyone. This footnote also > confirms that this email message has been swept for the<br>presence of > computer viruses.<br><br>Adaptive Mobile Security Ltd, Ferry House, 48 > Lower Mount Street, Dublin 2, Ireland<br>Directors: B. Collins, G. > Maclachlan (UK), N. Grierson (UK), J. Ennis (UK), D. Summers > (UK).<br>Registered in Ireland, Company No. 370343, VAT > Reg.No.IE6390343O<br>************************************************* > ***************************************</pre> > > </pre>****************************************************************************************<br>This email and any files transmitted with are confidential and intended solely for the<br>use of the individual or entity to whom they are addressed. If you have received this<br>email in error then please delete it and notify the sender. Do not make a copy or forward<br>it to anyone. This footnote also confirms that this email message has been swept for the<br>presence of computer viruses.<br><br>Adaptive Mobile Security Ltd, Ferry House, 48 Lower Mount Street, Dublin 2, Ireland<br>Directors: B. Collins, G. Maclachlan (UK), N. Grierson (UK), J. Ennis (UK), D. Summers (UK).<br>Registered in Ireland, Company No. 370343, VAT Reg.No.IE6390343O<br>****************************************************************************************</pre>
