I think you are right. The executor gets the driver port from "RpcEnv.address".
Best Regards, Shixiong Zhu 2015-11-26 11:45 GMT-08:00 Rad Gruchalski <ra...@gruchalski.com>: > Dear all, > > I am currently looking at modifying NettyRpcEnv for this PR: > https://github.com/apache/spark/pull/9608 > The functionality which I’m trying to achieve is the following: if there > is a configuration property spark.driver.advertisedPort, make executors > reply to advertisedPort instead of spark.driver.port. This would enable > NettyRpcEnv work correctly on Mesos with Docker bridge networking > (something what is working spot on for AkkaRcpEnv). > > I’ve spent some time looking at the new NettyRpcEnv and I think I know > what is happening but seeking for confirmation. > > The master RpcEndpointAddress appears to be shipped to the executor as > part of a serialized message, when the executors are requested, inside of > the NettyRpcEndpointRef. In order to make my PR work, I need to change the > RcpEndpointAddress shipped to the executors on the master. > I think the best candidate is: > > > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala#L125 > > Am I correct that this value becomes the RcpEndpointAddress? If so, my > naive implementation would be like this: > > if (server != null) RpcAddress(host, > conf.getInt(“spark.driver.advertisedPort”, server.getPort())) else null > > however, I am not sure what impact such change would have, about to test > my changes as soon as the code from master branch builds. > > If that is not the correct place to make such change, what would be the > best place to investigate? Is there an overview of NettyRcpEnv available > anywhere? > > Kind regards, > Radek Gruchalski > ra...@gruchalski.com <ra...@gruchalski.com> > de.linkedin.com/in/radgruchalski/ > > > *Confidentiality:*This communication is intended for the above-named > person and may be confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. >
