Ah! Thanks a lot! I am trying to learn pig. :) So I was looking into all the different sources & types. Kind of aimless drifting.
Sumit ________________________________ From: Dmitriy Ryaboy <[email protected]> To: [email protected] Sent: Mon, 25 April, 2011 9:54:25 PM Subject: Re: Getting the total Mapred time This is not available from inside Pig Latin -- PigStats are provided to allow programs that embed pig to monitor and report on it. What are you trying to do? D On Mon, Apr 25, 2011 at 2:03 AM, sumit ghosh <[email protected]> wrote: > Hi, > > I guess this is java Syntax. I am looking to use this function in Pig > Latin. > What will be the syntax in Pig Latin. > > > Thanks, > Sumit > > > ________________________________ > From: Dmitriy Ryaboy <[email protected]> > To: [email protected]; [email protected] > Sent: Sun, 24 April, 2011 5:04:20 AM > Subject: Re: Getting the total Mapred time > > I am still unclear on what isn't clear :) > > What are you trying to do? Did you read the javadocs for PigStats? > > To call get duration it's just > > PigStats stats = PigRunner.run(myscript, mylistener); > long duration = stats.getDuration(); > > D > > > > On Sat, Apr 23, 2011 at 10:09 AM, Chaitanya Sharma <[email protected] > >wrote: > > > Are there any example's around this which could prove useful for > > understanding this? > > > > Thanks, > > Chaitanya > > > > On Fri, Apr 22, 2011 at 1:52 PM, Dmitriy Ryaboy <[email protected]> > > wrote: > > > > > I may be misunderstanding what you are asking. The tricky part is > > measuring > > > MR time *without* wait time, which one cannot control (it depends > mostly > > on > > > the size and utilization level of your cluster). This tricky bit is > what > > > PigStats helps you with. > > > If you just want to measure the full time, including wait time, you can > > > just > > > time the process like you would time anything else. Use the "time" unix > > > command, or if you are in Java, use System.currentTimeMillis() before > you > > > start executing and again when the procedure returns. > > > > > > On Thu, Apr 21, 2011 at 10:18 PM, sumit ghosh <[email protected]> > wrote: > > > > > > > Hi, > > > > > > > > How to get the actual time spent in doing all the map-reduce > operations > > > > while > > > > executing a pig script. It should exclude the time wait for the > > scheduler > > > - > > > > and > > > > any other waiting time. Please help. > > > > > > > > Another question is - how to use the function getDuration() in: > > > > > > > > > > > > > >http://pig.apache.org/docs/r0.8.0/api/org/apache/pig/tools/pigstats/PigStats.html >l > >l > > > > > > > > Can you please help me with the Pig Statements. > > > > > > > > Thanks, > > > > Sumit > > > > > > > > > >
