[jira] [Updated] (GIRAPH-128) RPC port from BasicRPCCommunications should be only a starting port, and retried
[ https://issues.apache.org/jira/browse/GIRAPH-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching updated GIRAPH-128: --- Attachment: GIRAPH-128.4.patch Sorry, I missed the mocking question. Fixed it here. RPC port from BasicRPCCommunications should be only a starting port, and retried Key: GIRAPH-128 URL: https://issues.apache.org/jira/browse/GIRAPH-128 Project: Giraph Issue Type: Improvement Affects Versions: 0.1.0 Reporter: Avery Ching Assignee: Avery Ching Attachments: GIRAPH-128.2.patch, GIRAPH-128.3.patch, GIRAPH-128.4.patch Currently Giraph uses a basic port + the task partition to get the RPC port. This doesn't work well for when there are multiple Giraph jobs running simultaneously in the same Hadoop cluster (port conflict). At the same time, it is nice to use this simple algorithm because it makes it very easy to debug problems (you can find the troublesome mapper from the RPC port name). I will be proposing a simple scheme to retry with another port. I will round the total number of mappers up to the nearest power of 10 (let's that that number Z). Then I will increment the port number by Z, retrying up to 20 tries. If you have enough ports, this scheme would guarantee that up to 20 mappers / node would be supported. It should be sufficient for most clusters. At the same time, we still maintain the easy debugging method since you it's still easy to figure out the mapper partition from the port (port % Z = map partition). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-128) RPC port from BasicRPCCommunications should be only a starting port, and retried
[ https://issues.apache.org/jira/browse/GIRAPH-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Ching updated GIRAPH-128: --- Attachment: GIRAPH-128.2.patch Updated after GIRAPH-124 was committed. RPC port from BasicRPCCommunications should be only a starting port, and retried Key: GIRAPH-128 URL: https://issues.apache.org/jira/browse/GIRAPH-128 Project: Giraph Issue Type: Improvement Affects Versions: 0.1.0 Reporter: Avery Ching Assignee: Avery Ching Attachments: GIRAPH-128.2.patch Currently Giraph uses a basic port + the task partition to get the RPC port. This doesn't work well for when there are multiple Giraph jobs running simultaneously in the same Hadoop cluster (port conflict). At the same time, it is nice to use this simple algorithm because it makes it very easy to debug problems (you can find the troublesome mapper from the RPC port name). I will be proposing a simple scheme to retry with another port. I will round the total number of mappers up to the nearest power of 10 (let's that that number Z). Then I will increment the port number by Z, retrying up to 20 tries. If you have enough ports, this scheme would guarantee that up to 20 mappers / node would be supported. It should be sufficient for most clusters. At the same time, we still maintain the easy debugging method since you it's still easy to figure out the mapper partition from the port (port % Z = map partition). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
