pig creates many small files when it spills
-------------------------------------------
Key: PIG-176
URL: https://issues.apache.org/jira/browse/PIG-176
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich
Assignee: Alan Gates
Currently, on spill pig can generate millions of small (under 128K) files.
Partially this is due to PIG-170 but even with that patch, you can still try
and spill small bags.
The proposal is to not spill small files. Alan told me that the logic is
already there but we just need to bump the size limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
