[
https://issues.apache.org/jira/browse/GIRAPH-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617852#comment-13617852
]
Eli Reisman commented on GIRAPH-362:
------------------------------------
This is interesting because I had trouble in the YARN patch with the taskid
stuff too. I noticed in a recent patch Maja removed a hardcoded setting of the
master task id and set it with getTaskPartitionId type calls.
Does anyone know exactly where the task id dependencies in Giraph are, what
they are, etc? Are there any Giraph tasks that need a certain task id for a job
to run? How about Hadoop or MR dependencies in the IO formats needing this?
Thanks!
> Address master task id for communication for master (known issue from
> GIRAPH-211)
> ---------------------------------------------------------------------------------
>
> Key: GIRAPH-362
> URL: https://issues.apache.org/jira/browse/GIRAPH-362
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
>
> There is a workaround from GIRAPH-211 to handle requests a little differently
> due to issues communicating to the master. We should fix this to be a
> regular request in the future.
> {code}
> public void sendWritableRequest(Integer destWorkerId,
> InetSocketAddress remoteServer,
> WritableRequest request) {
> if (clientRequestIdRequestInfoMap.isEmpty()) {
> byteCounter.resetAll();
> }
> boolean registerRequest = true;
> /*if[HADOOP_NON_SECURE]
> else[HADOOP_NON_SECURE]*/
> if (request.getType() == RequestType.SASL_TOKEN_MESSAGE_REQUEST) {
> registerRequest = false;
> }
> /*end[HADOOP_NON_SECURE]*/
> Channel channel = getNextChannel(remoteServer);
> RequestInfo newRequestInfo = new RequestInfo(remoteServer, request);
> if (registerRequest) {
> request.setClientId(clientId);
> request.setRequestId(
> addressRequestIdGenerator.getNextRequestId(remoteServer));
> ClientRequestId clientRequestId =
> new ClientRequestId(destWorkerId, request.getRequestId());
> RequestInfo oldRequestInfo = clientRequestIdRequestInfoMap.putIfAbsent(
> clientRequestId, newRequestInfo);
> if (oldRequestInfo != null) {
> throw new IllegalStateException("sendWritableRequest: Impossible to "
> +
> "have a previous request id = " + request.getRequestId() + ", " +
> "request info of " + oldRequestInfo);
> }
> }
> ChannelFuture writeFuture = channel.write(request);
> newRequestInfo.setWriteFuture(writeFuture);
> if (limitNumberOfOpenRequests &&
> clientRequestIdRequestInfoMap.size() > maxNumberOfOpenRequests) {
> waitSomeRequests(maxNumberOfOpenRequests);
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
