[
https://issues.apache.org/jira/browse/CRUNCH-463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Wills resolved CRUNCH-463.
-------------------------------
Resolution: Fixed
Fix Version/s: 0.11.0
0.8.4
Fixed on master and 0.8. Thanks for the heads up, [~hekoru]!
> Copying the Configuration object in every CrunchInputSplit causes OOM errors
> for jobs with lots of splits
> ---------------------------------------------------------------------------------------------------------
>
> Key: CRUNCH-463
> URL: https://issues.apache.org/jira/browse/CRUNCH-463
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.10.0
> Reporter: Hector Izquierdo Seliva
> Assignee: Josh Wills
> Fix For: 0.8.4, 0.11.0
>
> Attachments: CRUNCH-463.patch
>
>
> Trying to run a job with 11k input files and that yields about 25k splits
> results in OOM errors due to too many copies of the Configuration object
> being created when the CrunchInputSplit is initialised. I know that that's
> the result of CRUNCH-313, but perhaps a better way to deal with that problem
> should be found.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
