Search for 'nested foreach' statements in the link I sent. You can use ORDER BY and LIMIT within these statements and I think that's what you're looking for.
Thanks, Gunther. On Mon, Oct 22, 2012 at 10:20 AM, jamal sasha <[email protected]> wrote: > Hi > Thanks for reply . > But how do I sort this for each user group instead of the entire list by > score. > And then for each user group I want to have top 20 rather than selecting > top 20 from the whole list > Any ideas :( > Thanks > > On Monday, October 22, 2012, Gunther Hagleitner < > [email protected]> > wrote: > > That's fairly straightforward. Take a look at: > > http://pig.apache.org/docs/r0.10.0/basic.html (order by, limit). > > > > Thanks, > > Gunther. > > > > On Mon, Oct 22, 2012 at 7:12 AM, jamal sasha <[email protected]> > wrote: > > > >> Hi > >> Great . Thanks alot. > >> How do I sort the result by score and select top 20 (say)? > >> > >> On Monday, October 22, 2012, Gunther Hagleitner < > >> [email protected]> > >> wrote: > >> > This should work: > >> > > >> > matrix = load 'data1.txt' using PigStorage(',') as (row:chararray, > >> > column:chararray, value:float); > >> > vectors = load 'data2.txt' using PigStorage(',') as (user:chararray, > >> > column:chararray); > >> > > >> > joined = join vectors by column, matrix by column; > >> > groups = group joined by (user, row); > >> > result = foreach groups generate group.user, group.row, (float) > >> > SUM(joined.value); > >> > > >> > store result into 'result'; > >> > > >> > Thanks, > >> > Gunther. > >> > > >> > On Sun, Oct 21, 2012 at 7:40 PM, jamal sasha <[email protected]> > >> wrote: > >> > > >> >> Hi, > >> >> I am trying to do matrix multiplication using pig. > >> >> > >> >> Basically I have data in the form: > >> >> data1.txt > >> >> item1,item2,0.3 > >> >> item1, item3, 0.4 > >> >> item1, item5, 0.6 > >> >> > >> >> And then I another data in the form > >> >> data2.txt > >> >> user1,item1 > >> >> user1,item2 > >> >> user1,item5 > >> >> ... > >> >> user2,item2 > >> >> etc > >> >> > >> >> Just to give some context.. I am trying to build a top n > recommendation > >> >> system.. which is as follows. > >> >> Matrix formed by data2.txt > >> >> item1 item2 item3 item4 item5 > >> >> user1 1 1 0 0 1 > >> >> > >> >> > >> >> Matrix formed by data1.txt > >> >> > >> >> item1 item2 item 3 item4 item5 > >> >> item1 1 0.3 0.4 0 > 0.6 > >> >> item2 1 > >> >> item3 1 > >> >> item4 1 > >> >> item5 > >> 1 > >> >> > >> >> > >> >> So recommendations for user1 would be whether user1 is the score > >> >> computation as followed > >> >> Score for user 1 for item 1 = (ignore item1, item1 score) u12* > item_12 + > >> >> u13*item_13 + u14*item14 + u15*item15 > >> >> > >> >> = > >> >> 1 *0.3 + 0*0.4 + 0*0 + 1 * 0.6 = 0.9 > >> >> > >> >> And then i find this score for user1 and item2 > >> >> > >> >> And then for user 2 .. item 1 and so on. > >> >> > >> >> I understand this is more of an implementation challenge.. and not > sure > >> >> whether this is the right place to ask this.. But any suggestions > will > >> be > >> >> greatly appreciated. > >> >> Thanks > >> >> Jamal > >> >> > >> > > >> > > >
