David, This is a problem, that many people encounter. And I think, it's about time to deal with it.
We have the following method, that collects all local addresses, that will be sent with node attributes: *IgniteUtils#resolveLocalAddresses()*. And it iterates over all local interfaces, that the machine has. Maybe we don't need to do it? How about including only those interfaces, that are specified in configuration of *CommunicationSpi* and *DiscoverySpi*, and the ones, that are returned from a configured *AddressResolver*. To keep compatibility we could make a default address resolver return addresses from all local interfaces. What do you think? Denis вт, 20 нояб. 2018 г. в 22:45, David Harvey <syssoft...@gmail.com>: > What we prototyped was configuring via spring the list of IPs to ignore, > because a given installation seemed to have a constant address for the > bridge network, and this approach was reliable, once you know the bridge > IPs. It is also a more general solution. > > When the container starts, you get a list of IP addresses from the kernel. > At that point it is impossible to know from inside the container which of > those addresses can be used by other ignite nodes, at least without > external information. For exampe, ifI have ignite running on an AWS > instance that has an internal and external address, it is impossible to > know which address will be able to reach the other nodes, unless you are > told. So perhaps we should have used a list of ranges rather than a list > in our prototype. > > For the docker sub-case where all the nodes seem to get the same useless > address, I would think we can ignore IP address/port pairs that are the > current node is also advertising. That does not generalize to other > cases were the kernel provides unusable addresses. I didn't quite > understand why if we try to connect to port we are advertising, this would > need to timeout, rather than getting immediately rejected, unless Ignite > has explicit code to do detected and ignore a self message. But if there > is a IP:port pair that the current node is claiming as an endpoint, it > should not try to use that IP:port to connect to other nodes..... > > On Tue, Nov 20, 2018 at 2:27 PM David Harvey <syssoft...@gmail.com> wrote: > > > What we prototyped was configuring via spring the list of IPs to ignore, > > because a given installation seemed to have a constant address for the > > bridge network, and this approach was reliable, once you know the bridge > > IPs. > > > > When the container starts, you get a list of IP addresses from the > > kernel. At that point it is impossible to know from inside the > container > > which of those addresses can be used by other ignite nodes, at least > > without external information. Similarly, if I have an AWS instance > > > > I am wondering > > > > > > > > On Tue, Nov 20, 2018 at 2:08 PM Alexey Goncharuk < > > alexey.goncha...@gmail.com> wrote: > > > >> Hi David, > >> > >> This is something we have also encountered recently and I was wondering > >> how > >> this can be mitigated in a general case. Do you know if an application > can > >> detect that it is being run in a docker container and add the > >> corresponding > >> list of bridge IPs automatically on start? If so, I this we can add this > >> to > >> the Ignite so that it works out of the box. > >> > >> --AG > >> > >> > >> вт, 20 нояб. 2018 г. в 19:58, David Harvey <syssoft...@gmail.com>: > >> > >> > We see some annoying behavior with S3 discovery because Ignite will > >> push to > >> > the discovery S3 bucket the IP address of the local docker bridge > >> network > >> > (172.17.0.1) in our case. Basically, each node when coming online > >> tries > >> > that address first, and has to go through a network timeout to > recover. > >> > > >> > To address this, have prototyped a simple extension to > >> TcpCommunicationSpi > >> > to allow configuration of a list of IP addresses that should be > >> completely > >> > ignored, and will create a ticket and generate a pull request for it. > >> > > >> > If there is a better approach, please let us know. > >> > > >> > Thanks > >> > Dave Harvey > >> > > >> > > >
