This looks like a bug. Can you please file a jira with steps to reproduce?
On Fri, Apr 18, 2014 at 2:45 PM, Alex Rasmussen alex...@trifacta.comwrote:
I'm using PigStorage(',') for all stores.
I agree about the expensiveness of CROSS, but I'm still kind of confused as
to why it would lose
I'm noticing some really strange behavior with a CROSS operation in one of
my scripts.
I'm CROSSing a table T1 with another table T2 to produce T3. T1 has one
row, and T2 has 2,982,035 rows.
If I STORE both T1 and T2 before CROSSing them together to get T3, like so:
-- ... Long script that,
What is the storage func you're using? My guess is that there is some
shared state in the Storage func. Take a look at this SO that is dealing
with shared state in Stores.
http://stackoverflow.com/questions/20225842/apache-pig-append-one-dataset-to-another-one/20235592#20235592.
The reason why
I'm using PigStorage(',') for all stores.
I agree about the expensiveness of CROSS, but I'm still kind of confused as
to why it would lose records in this case.
--Alex
On Fri, Apr 18, 2014 at 2:28 PM, Pradeep Gollakota pradeep...@gmail.comwrote:
What is the storage func you're using? My
STORing and LOADing relations often is a workaround for these kinds of bugs.
On Friday, April 18, 2014, Alex Rasmussen alex...@trifacta.com wrote:
I'm using PigStorage(',') for all stores.
I agree about the expensiveness of CROSS, but I'm still kind of confused as
to why it would lose
