[
https://issues.apache.org/jira/browse/MAPREDUCE-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated MAPREDUCE-5194:
-------------------------------------
Attachment: M5194-2.patch
bq. Should we make openConnection synchronized and add an analogous
(synchronized) closeConnection method?
Sure, that makes sense as internal factoring, though the guard exists for the
assignment of {{connection}} not the logic of {{openConnection}}. I added a
comment to point out the {{HttpURLConnection::disconnect()}} issue, so its
semantics are clearer.
bq. In copyMapsFromHost should we check for stopped before we call
openConnection?
Creating an instance of {{HttpURLConnection}} should just wrap the {{URI}}, so
I think it's sufficient to check after it's assigned.
bq. 3.Also, should we call disconnect in the case stopped is true after the
call to Fetcher.connect in copyMapsFromHost?
Good point. Added an explicit disconnect to avoid the resource leak.
> Heed interrupts during Fetcher shutdown
> ---------------------------------------
>
> Key: MAPREDUCE-5194
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5194
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: task
> Reporter: Chris Douglas
> Assignee: Chris Douglas
> Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5194-1.patch, M5194-2.patch, M5194.patch
>
>
> In the current implementation, {{Fetcher}} instances usually exit gracefully
> when the shuffle succeeds. When it fails, threads are interrupted, but may
> continue running harmlessly until the JVM shuts down.
> However, to generate consistent checkpoints, these threads should exit
> cleanly to quiesce the state of the shuffle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
