[ https://issues.apache.org/jira/browse/CRUNCH-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Wills updated CRUNCH-669: ------------------------------ Attachment: CRUNCH-669.patch > Add an option to preserve Crunch temp directories > ------------------------------------------------- > > Key: CRUNCH-669 > URL: https://issues.apache.org/jira/browse/CRUNCH-669 > Project: Crunch > Issue Type: Improvement > Reporter: Josh Wills > Priority: Major > Attachments: CRUNCH-669.patch > > > I have a problem where a Crunch client can potentially get killed through no > fault of its own (e.g., an Airflow task failing a heartbeat check), which > will kill the client, but leave the MR job itself still running. The problem > is that the finalize() operation can delete the temp directory that contains > the config info for Crunch, which will make any subsequent tasks fail to run > to completion b/c their config info has been removed, or because some temp > output in the directory that would have been useful can be deleted. I'd like > to add an option to override the automatic delete of the temp directory > during the finalize() operation for these situations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)