Still would like to see the script or the explain plan.. D
On Sat, Sep 15, 2012 at 7:50 PM, Haitao Yao <[email protected]> wrote: > No, I also thought it is a mapper , but It surely is a reducer. all the > mappers succeeded and the reducer failed. > > > > Haitao Yao > [email protected] > weibo: @haitao_yao > Skype: haitao.yao.final > > On 2012-9-16, at 上午10:08, Haitao Yao wrote: > >> Hi, >> I 'v encountered a problem: the job failed because of POSplit retained >> too much memory in the reducer. How can I specify more reducers for the >> spill? >> >> Here's the screen snapshot of the Heap dump. >> <aa.jpg> >> >> >> And here's the snippet of my split script: >> >> split RawData into AURawData if type == 2, NURawData if type == 1, >> InRawData if type == 9, GCData if type == 61, HCData if type == 71, >> TutorialRawData if type == 3 or t ype == 15; >> >> There's 3 similar split clause in my script. The reducer count is always 1. >> How can I increase it? >> >> Thanks. >> >> >> >> Haitao Yao >> [email protected] >> weibo: @haitao_yao >> Skype: haitao.yao.final >> >
