You could use a shell script to generate the periods and call then using backticks:
%declare startperiod `stringdatetoepoch 2012/05/17/02 ...` %declare endperiod `stringdatetoepoch ...` scott. On Sat, May 19, 2012 at 9:56 AM, Mustafi, Priyo <[email protected]> wrote: > Hi All, > I have some data which has epoch_time field (System.currentmillis). I need > to filter out the data to make sure the records are within a time range. So > I wrote a UDF to convert yyyy/MM/dd/HH to time in millis. The filter would > go like below > > > filteredData = Filter data by StringDateToEpoch('2012/05/17/02') <= > epoch_time && epoch_time < StringDateToEpoch('2012/05/17/03'); > > > Obviously the StringDateToEpoch UDF is getting called twice for each row in > the data which is very inefficient since they are both constants. Is there a > way to define constants which are evaluated at the time of definition or > something like below? > > define startperiod StringDateToEpoch('2012/05/17/02'); > define endperiod StringDateToEpoch('2012/05/17/03'); > filteredData = Filter data by startperiod <= epoch_time && by epoch_time < > endperiod; > > > I can ask the users to run a java program to generate the start/end period > first and then use that to call by pig script but that is painful. > > Any ideas? > > Thanks > Priyo >
