[
https://issues.apache.org/jira/browse/PIG-176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590074#action_12590074
]
Alan Gates commented on PIG-176:
--------------------------------
Pi,
Did you want to rework this patch now since PIG-111 is in and you can read the
properties from pig's Properties object rather than System.getProperties()?
> pig creates many small files when it spills
> -------------------------------------------
>
> Key: PIG-176
> URL: https://issues.apache.org/jira/browse/PIG-176
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Alan Gates
> Attachments: pig_176_smallbags_v1.patch
>
>
> Currently, on spill pig can generate millions of small (under 128K) files.
> Partially this is due to PIG-170 but even with that patch, you can still try
> and spill small bags.
> The proposal is to not spill small files. Alan told me that the logic is
> already there but we just need to bump the size limit.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
