Hi All, As I still see problems with IPv6 name resolutions on a large population (62M+ instances in my case), I would like this subject not to be forgotten. My patch (provided at the beginning of this thread) prohibiting using IPv6 name resolutions for IPv4-only resolve mode in dual-stack enabled libcurl, proved to be helpful to mitigate bad IPv6 side effects for the IPv4 mode.
It has been used in production environment for several months, and A/B tests confirmed that it made IPv4-only resolve mode in dual-stack libcurl to be the same as IPv6-disabled single-stack libcurl as far as connection establishment performance is concerned. So, I recommend to include my patch into upcoming libcurl releases (either "as is" or with some modifications) or to provide any other solution allowing to avoid IPv6 side effects for the IPv4-only mode. Based on IPv6 name resolution problems seen in the field, I just don't see how my applications (or any application using dual-stack in user-critical scenarios on a very large population) can go without fixing IPv6 bad side effects for IPv4-only resolve mode. Thanks, Dmitry Karpov -----Original Message----- From: curl-library <curl-library-boun...@lists.haxx.se> On Behalf Of Dmitry Karpov via curl-library Sent: Tuesday, July 12, 2022 12:26 PM To: libcurl development <curl-library@lists.haxx.se> Cc: Dmitry Karpov <dkar...@roku.com> Subject: RE: IPv6 resolution problems for IPv4 resolve mode > Yes, the name could be better but I don't we need to argue about that detail. > The question is really what libcurl should do when the different values are > set. Very well said. It precisely describes the problem. The resolution is the part of the transfer and the IPRESOLVE_* "kind of" implies that the specified mode is applied both for resolution and connection. I was also thinking about providing additional option like CURLOPT_DNS_IPRESOLVE (name can be better), which will decouple IP modes used for name resolution and connection, but it will be essentially the same as providing additional bit to IPRESOLVE_. > >Complicating this is the DNS cache, which is shared across all curl > >handles. Evicting cache entries to control connections is > >counter-intuitive and inefficient. Consider the case of one handle > >wanting IPv4 for SSH to a host, but IPv6 for http... > That's exactly the case we had when we recently changed this code, which then > introduced the problems Dmitry is seeing... I should notice that I didn't suggest evicting anything from the DNS cache. My suggestion was not to perform "dual" queries when IPv4 only mode was specified in the IPRESOLVE to avoid any dual-stack related regressions. The "wanting IPv4 for SSH to a host, but IPv6 for http" case is understandable, but it looks more like a very specialized use case, where the "IPv4 for SSH to a host" transfer is willing to perform "dual" query for the next IPv6 HTTP transfer regardless any potential "dual resolution" overhead and potential timeouts. Even in good cases "dual" queries take more time than single IPv4 queries, so the use case when one transfer uses "IPv4 only" mode is willing to sacrifice its start performance and face resolution timeouts to provide a potentially faster start for the next transfer, doesn't seem to me like a general use case which should be necessarily covered by default. I totally agree that such use cases are valid and can be desired, but I don't think that they should be covered by default creating problems for "IPv4 only" transfers, which don't want to use dual-stack and step on any dual-stack related issues. > > The fact that some issues still occur with IPv6 in some environments > > just means that those environments are broken. Yes, IPv6 is growing, but problems will still be there in the near future at least. The main support focus is still on IPv4, and ISPs quite often neglect IPv6 or do something that break IPv6 on older routers. I also hope that we will come to the point when problems with IPv6 disappear, but we are not there yet. That's why it is important to make sure that IPv6-enabled libcurl does provide "IPv4 only" mode which doesn't have any IPv6-related side effects. In my case, turning on IPv6 in libcurl created severe regressions on embedded devices with certain ISPs/routers, where all critical operations were using "IPv4 only" mode but stepped on IPv6 resolution problems resulting into transfer timeouts. And it was very painful and costly to fix these issues when OTA updates couldn't get critical patches because of these issues. So, I hope we realize that we need to fix this problem and just trying to find the best solution. Thanks, Dmitry -----Original Message----- From: curl-library <curl-library-boun...@lists.haxx.se> On Behalf Of Daniel Stenberg via curl-library Sent: Tuesday, July 12, 2022 7:28 AM To: Timothe Litt via curl-library <curl-library@lists.haxx.se> Cc: Daniel Stenberg <dan...@haxx.se> Subject: Re: IPv6 resolution problems for IPv4 resolve mode On Mon, 11 Jul 2022, Timothe Litt via curl-library wrote: > The name IPRESOLVE_* implies that it controls resolution, but the > description says it controls "what kind of IP addresses to use when > establishing a (possibly cached) connection:. Yes, the name could be better but I don't we need to argue about that detail. The question is really what libcurl should do when the different values are set. > Complicating this is the DNS cache, which is shared across all curl > handles. Evicting cache entries to control connections is > counter-intuitive and inefficient. Consider the case of one handle > wanting IPv4 for SSH to a host, but IPv6 for http... That's exactly the case we had when we recently changed this code, which then introduced the problems Dmitry is seeing... > The cache should, subject to TTL, hold all resolutions ever performed > for a name - v4, v6, or both. This includes caching the case of a > resolution returning "Answers:0" (no address of that type). The TTL is not generally accessible so the cache is not using that. There is also no negative caching (ie a cache miss will cause another name resolve) - but it has been mentioned as a TODO since basically forever. > With respect to a default: it should be both. The fact that some > issues still occur with IPv6 in some environments just means that > those environments are broken. It's fine to have a work-around for > them. But defaulting to the work-around only discourages these from > being fixed. Like it or not, IPv6 is the going-forward plan for the > network and adoption is happening, however slowly. I think we all agree on this. Dmitry wants the/a work-around to avoid the IPv6 problem he's seeing. > Again: the handle should specify how to connect. The cache should > hold the result of lookups, which are triggered when needed. Don't conflate > the two. Since Dmitry's problem is in the resolve phase, the work-around for that issue probably must change the resolving part and then as a direct follow-up effect it affects what is cached for that name... I can't think of a away to avoid that! -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
