pig-user  

Large tuples

Mridul Muralidharan
Tue, 10 Jun 2008 12:57:24 -0700

Hi,

  How does pig handle really large tuples.
Assuming after a group, the resulting alias has small subset of tuples (out of the many which were generated) which are really large in size. In excess of a gig as a ballpark figure (so that the tuple is spread across many dfs blocks).

Does pig handle this case ? If yes how (refs/rtfm would be great too) ?

Thanks,
Mridul