Thanks On Thu, Mar 22, 2012 at 5:09 PM, Thejas Nair <[email protected]> wrote:
> On 3/22/12 4:18 PM, Mohit Anchlia wrote: > >> Thanks! Is there any time function that I can use in pig to get the >> current >> time? >> > > There might be a date udf you can use. But as you probably want same date > for all data (if your query happens to runs across date boundary, or a node > has a problem) the better option is to use parameter substitution in pig. > > Change the line to - > B = foreach A generate *, '$DATE' as date; > > Then specify the date param your pig command, using the *nix date cmd - > > pig -p DATE=`date +%Y%m%d` query.pig > > -Thejas > > > > > >> On Thu, Mar 22, 2012 at 3:58 PM, Jonathan Coveney<[email protected]>** >> wrote: >> >> There is not going to be an additional cost to adding the date as Thejas >>> posted. If that's what you want, do it. >>> >>> Optionally, you could extend your StoreFunc to prepend the date to every >>> record it writes. I assure you, doing what Thejas said is simpler (or >>> just >>> having the database you write to populate the date for you!) >>> >>> 2012/3/22 Mohit Anchlia<[email protected]**> >>> >>> On Thu, Mar 22, 2012 at 2:34 PM, Thejas Nair<[email protected]> >>>> wrote: >>>> >>>> Is this what you are looking for ? - >>>>> >>>>> >>>>> A = LOAD '$in' USING PigStorage('\t') AS (... >>>>> >>>>> B = foreach A generate *, '20120322' as date; >>>>> >>>>> STORE B into ... >>>>> >>>>> Thanks, >>>>> Thejas >>>>> >>>> >>>> >>>> this I am assuming will run another map task? I was trying to avoid >>>> that. >>>> >>>> >>>>> >>>>> >>>>> On 3/22/12 1:13 PM, Mohit Anchlia wrote: >>>>> >>>>> Yes that's exactly what I am asking. Reading from flat file and then >>>>>> inserting it into the database. And I want to insert date before >>>>>> >>>>> storing. >>>> >>>>> >>>>>> for eg I want to add date before A gets stored: >>>>>> >>>>>> >>>>>> A = LOAD '$in' USING PigStorage('\t') AS (... >>>>>> >>>>>> STORE A into ... >>>>>> >>>>>> On Thu, Mar 22, 2012 at 12:54 PM, Jonathan Coveney<[email protected] >>>>>> >>>>> ** >>>>> >>>>>> wrote: >>>>>> >>>>>> Do you mean you're reading a relation from Hadoop, and want to append >>>>>> >>>>> the >>>> >>>>> date to the row before inserting it? I'm not quite sure what you're >>>>>>> asking >>>>>>> for. >>>>>>> >>>>>>> > >>> 2012/3/22 Mohit Anchlia<[email protected]****> >>> >>>> >>>>>>> Sorry I mean to ask if there is any way to insert date into the ALIAS >>>>>>> >>>>>> so >>>> >>>>> that I can use it before storing it into DB. >>>>>>>> >>>>>>>> On Thu, Mar 22, 2012 at 12:47 PM, Mohit Anchlia< >>>>>>>> >>>>>>> [email protected] >>>> >>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>> I am reading bunch of columns from a flat file and inserting it into >>>>>>>> >>>>>>>>> >>>>>>>>> the >>>>>>>> >>>>>>> >>>>>>> database. Is there a way to also insert date? >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
