I just wanna to spilt the big job into smaller ones but still write into one script file.
I still does not know why the first one of the compiled job have only 1 reducer. Thanks. Haitao Yao [email protected] weibo: @haitao_yao Skype: haitao.yao.final On 2012-9-17, at 上午12:35, Alan Gates wrote: > 'exec' will force your job to start. However, I strongly doubt this will > solve your OOME problem, as some one part of your job is running out of > memory. Whichever part that is will still fail I suspect. Pig jobs don't > generally accrue memory as they go since most memory intensive operations are > done in their own task. If you can isolate the part of your script that is > causing an OOME (which exec should help with) and send that portion to the > list we may be able to help figure out what's causing the issue. > > Alan. > > On Sep 15, 2012, at 10:52 PM, Haitao Yao wrote: > >> Hi, all >> I forgot the keyword which force Pig to finish the job and then >> continue the following script. >> My job failed because of OOME, so I want to split the jobs into smaller >> ones but still written in a single pig script(because the script is >> generated) . >> Is there any keywords that can achieve this? >> thanks. >> >> >> >> Haitao Yao >> [email protected] >> weibo: @haitao_yao >> Skype: haitao.yao.final >> >
