Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, Feb 6, 2013 at 4:57 AM, Nick Jones nick.fa.jo...@gmail.com wrote: On Tue, Jan 29, 2013 at 9:47 PM, Miloslav Trmač m...@volny.cz wrote: On Tue, Jan 29, 2013 at 4:58 AM, Nick Jones nick.fa.jo...@gmail.com wrote: As a quick summary: I would suggest, in addition to addressing the outstanding bugs and issues covered by the Fedora feature, a flag be added to the set of getaddrinfo parameters that instructs it NOT to do DNS lookups, only perform alternative resource lookups supported by nss. This flag would be something like: AI_NODNS This will allow application developers to make use of getaddrinfo for resolving names using non dns sources (hosts file is DNS, so this means ldap and others), then perform internet domain lookups using an alternative DNS library that is standardised in Fedora. That's unnecessarily difficult for the application developers. Application should have a single API to call; if they have to call two separate APIs, some of them won't. Admittedly there is a problem with adding the AI_NODNS flag, in addition to what you mentioned. That is, what happens when an application compiled to use AI_NODNS is run against an older libc. Precisely what should happen: if (hints-ai_flags ~(AI_PASSIVE|AI_CANONNAME|AI_NUMERICHOST|AI_ADDRCONFIG|AI_V4MAPPED #ifdef HAVE_LIBIDN |AI_IDN|AI_CANONIDN|AI_IDN_ALLOW_UNASSIGNED |AI_IDN_USE_STD3_ASCII_RULES #endif |AI_NUMERICSERV|AI_ALL)) return EAI_BADFLAGS; I agree that splitting name resolution into two parts is an added complication for application developers, but I feel getaddrinfo itself is split into two of parts: - name resolution using databases such as ldap via nss - name resolution using a dns A or record lookup Making getaddrinfo and nss fully asynchronous will not happen soon, but my suggestion is to split the behaviour to allow an application to be able to resolve names from non-dns sources (which is an important function) then make use of a more modern dns resolver. If I understand you correctly, you want to split the queries that return immediately vs. those that may take a long time. In that case the split should not be DNS/non-DNS - in particular NIS/LDAP belong to the asynchronous (DNS) case. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
- Original Message - From: Nick Jones nick.fa.jo...@gmail.com In the summary of this Fedora feature for fixing name resolution is this snippet: Fedora could be seen as the leader in linux networking Yep. My intention to improve the overall networking features is not limited to what is described in FixNetworkNameResolution. getaddrinfo is important for Linux systems, and the existing issues with it's implementation are well worth fixing, but when it comes to DNS, getaddrinfo is holding Fedora and Linux back. When it comes to DNS, it's absolutely necessary to run a local resolver like dnsmasq or unbound (the latter supporting DNSSEC) to achieve some of the advanced features like split lists of DNS name servers (e.g. for VPNs). But at least dnsmasq is already supported by NetworkManager. Even though DNS is not the only source of 'names' that getaddrinfo can return, it is the most important, and in the context of DNS, getaddrinfo will not let Fedora reach the above stated goal. I'd be happy to discuss the details, probably in a separate discussion from this feature. Maybe IRC some time? It's not just the lack of support for asynchronous usage, getaddrinfo is a very feature poor resolver: - No support for alternative DNS record types: mx, txt ((ab)used for a lot of things) True enough. I was thinking about implementing at least SRV records. - No support for DNSSEC True enough. And also no support for DNS lifetimes. There probably should be a more coprehensive API for that while getaddrinfo would work on top of it. - No TCP fallback for that matter, which is essential for the above This should be handled in the DNS daemon (or supplementary tools) anyway. - Probably lot more, I'm no expert on dns Possibly. What is needed for networking leadership is: - a feature rich resolver, supporting the above as well as query and response filtering, ala AI_ADDRCONF Agreed. - an implementation of an IPv6 migration strategy along the lines of Happy Eyeballs I'm afraid this is not limited to DNS. - can be used both asynchronously or synchronously That would be nice. Communication with a daemon over sockets works asynchronously for free. - ideally pulls in no dependencies outside libc, except perhaps libidn That would require a custom communication protocol for the async part. - ideally the DNS packet parsing routines use only ANSI C - can be used by both C and C++ applications, and is easily bound to other scripting languages, even in the async case That again suggests a daemon with IPC. - provides reverse lookup Sure. - more I can't think of right now but I hope others will add more It would be better to maintain such information on a wiki page or something. Nobody will come back to the e-mails now and then. The question is does this all get added into glibc? 1) Yes: - glibc getaddrinfo gets updated to support this through the addition of new hints flags and return values, or a new function interface is created instead of shoehorning into the old interface, in any case the old getaddrinfo interface will always be callable using traditional, existing semantics and run time linking to existing apps will not be broken - the nss layer gets rewritten to support asynchronous usage (and decent error reporting and additional lookup domains maybe samba?) - perhaps the resulting work gets packaged into a proposed extension to POSIX Even if this could be done, it looks like a very-long-term project and needs more than just dropping suggestions to a mailing list. 2) No: - simply say: forget it, lets do proper DNS outside of libc We already can do proper DNS outside libc. There's no problem with that. The more complicated part is getting all sorts of resolution methods together. I say 1) could be possible over the course of a few years but for now, 2) is more realistic. As a quick summary: I would suggest, in addition to addressing the outstanding bugs and issues covered by the Fedora feature, a flag be added to the set of getaddrinfo parameters that instructs it NOT to do DNS lookups, only perform alternative resource lookups supported by nss. This flag would be something like: AI_NODNS I'm not in favor of adding this flag, as DNS is not IMO as specific. This will allow application developers to make use of getaddrinfo for resolving names using non dns sources What for? (hosts file is DNS, This is a huge mistake. Many of the bugs are results of treating the /etc/hosts the same way as DNS resolution. While the /etc/hosts file often overrides the information that would otherwise be recieved through DNS, it is no DNS at all, just a way to specify static mapping between names and addresses. so this means ldap and others), then perform internet domain lookups using an alternative DNS library that is standardised in Fedora. I don't think this is a good solution. Given that in most systems, the nss
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Tue, Jan 29, 2013 at 9:47 PM, Miloslav Trmač m...@volny.cz wrote: Hello, just a minor point, not getting into the wider should getaddrinfo() be the primary interface debate... On Tue, Jan 29, 2013 at 4:58 AM, Nick Jones nick.fa.jo...@gmail.com wrote: As a quick summary: I would suggest, in addition to addressing the outstanding bugs and issues covered by the Fedora feature, a flag be added to the set of getaddrinfo parameters that instructs it NOT to do DNS lookups, only perform alternative resource lookups supported by nss. This flag would be something like: AI_NODNS This will allow application developers to make use of getaddrinfo for resolving names using non dns sources (hosts file is DNS, so this means ldap and others), then perform internet domain lookups using an alternative DNS library that is standardised in Fedora. That's unnecessarily difficult for the application developers. Application should have a single API to call; if they have to call two separate APIs, some of them won't. Mirek Admittedly there is a problem with adding the AI_NODNS flag, in addition to what you mentioned. That is, what happens when an application compiled to use AI_NODNS is run against an older libc. The binary interface is the same but not the behavioural interface, therefore the application will run but will perform two dns lookups. One way to get around this is by adding a versioned symbol for getaddrinfo, which is no different from the current non versioned symbol. It will be present for enforcement of runtime and rpm build time compatability. Another way is to add a new functional interface like: getnondnsaddrinfo, except with a better name, which performs non dns lookups only. Adding new interfaces to glibc is only done when necessary so a lot of justification for either would be needed, rightly so. I agree that splitting name resolution into two parts is an added complication for application developers, but I feel getaddrinfo itself is split into two of parts: - name resolution using databases such as ldap via nss - name resolution using a dns A or record lookup Making getaddrinfo and nss fully asynchronous will not happen soon, but my suggestion is to split the behaviour to allow an application to be able to resolve names from non-dns sources (which is an important function) then make use of a more modern dns resolver. The behaviour of existing binaries run against the a libc will not be changed in any way. They will continue to use getaddrinfo as they have always done. New applications will have the freedom to lookup a name using getaddrinfo first, then perform the specifically A or record lookup using a dedicated dns library that is fully asynchronous, and also supports richer dns functionality, thus can be re-used for other purposes. Nick -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
Hello, just a minor point, not getting into the wider should getaddrinfo() be the primary interface debate... On Tue, Jan 29, 2013 at 4:58 AM, Nick Jones nick.fa.jo...@gmail.com wrote: As a quick summary: I would suggest, in addition to addressing the outstanding bugs and issues covered by the Fedora feature, a flag be added to the set of getaddrinfo parameters that instructs it NOT to do DNS lookups, only perform alternative resource lookups supported by nss. This flag would be something like: AI_NODNS This will allow application developers to make use of getaddrinfo for resolving names using non dns sources (hosts file is DNS, so this means ldap and others), then perform internet domain lookups using an alternative DNS library that is standardised in Fedora. That's unnecessarily difficult for the application developers. Application should have a single API to call; if they have to call two separate APIs, some of them won't. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Mon, Jan 21, 2013 at 6:25 PM, Daniel P. Berrange berra...@redhat.com wrote: On Fri, Jan 18, 2013 at 09:37:37AM +0800, Nick Jones wrote: On Wednesday, January 16, 2013 10:11 PM, Jaroslav Reznik wrote: As the Feature was renamed and the content of Feature has been extended as requested by FESCo, re-announcing it to the community for re-review. See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution * Detailed description Currently the getaddrinfo() function doesn't work as it was desinged. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future. One feature I would like to see added to future glibc, is a fully asynchronous version of getaddrinfo, and also getnameinfo. Asynchronous for non filesystem fds at least. The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a In the summary of this Fedora feature for fixing name resolution is this snippet: Fedora could be seen as the leader in linux networking getaddrinfo is important for Linux systems, and the existing issues with it's implementation are well worth fixing, but when it comes to DNS, getaddrinfo is holding Fedora and Linux back. Even though DNS is not the only source of 'names' that getaddrinfo can return, it is the most important, and in the context of DNS, getaddrinfo will not let Fedora reach the above stated goal. It's not just the lack of support for asynchronous usage, getaddrinfo is a very feature poor resolver: - No support for alternative DNS record types: mx, txt ((ab)used for a lot of things) - No support for DNSSEC - No TCP fallback for that matter, which is essential for the above - Probably lot more, I'm no expert on dns What is needed for networking leadership is: - a feature rich resolver, supporting the above as well as query and response filtering, ala AI_ADDRCONF - an implementation of an IPv6 migration strategy along the lines of Happy Eyeballs - can be used both asynchronously or synchronously - ideally pulls in no dependencies outside libc, except perhaps libidn - ideally the DNS packet parsing routines use only ANSI C - can be used by both C and C++ applications, and is easily bound to other scripting languages, even in the async case - provides reverse lookup - more I can't think of right now but I hope others will add more The question is does this all get added into glibc? 1) Yes: - glibc getaddrinfo gets updated to support this through the addition of new hints flags and return values, or a new function interface is created instead of shoehorning into the old interface, in any case the old getaddrinfo interface will always be callable using traditional, existing semantics and run time linking to existing apps will not be broken - the nss layer gets rewritten to support asynchronous usage (and decent error reporting and additional lookup domains maybe samba?) - perhaps the resulting work gets packaged into a proposed extension to POSIX 2) No: - simply say: forget it, lets do proper DNS outside of libc I say 1) could be possible over the course of a few years but for now, 2) is more realistic. As a quick summary: I would suggest, in addition to addressing the outstanding bugs and issues covered by the Fedora feature, a flag be added to the set of getaddrinfo parameters that instructs it NOT to do DNS lookups, only perform alternative resource lookups supported by nss. This flag would be something like: AI_NODNS This will allow application developers to make use of getaddrinfo for resolving names using non dns sources (hosts file is DNS, so this means ldap and others), then perform internet domain lookups using an alternative DNS library that is standardised in Fedora. Given that in most systems, the nss alternatives are local file based, and given the option of using nsscache, these lookups could be considered reasonably constrained therefore NOT in need of asynchronous implementations. Over time, the alternative DNS library could be considered for inclusion in the core c library, and could replace the DNS lookup aspect of getaddrinfo, at least for A and lookups, and more if additional query flags are defined for getaddrinfo or if an alternative function interface is defined. In addition, work can be done to make the nss layer asynchronous, or at least constrain it to local file resources and standardise on the use of nsscache.
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On 01/23/2013 07:31 PM, John Reiser wrote: On 01/23/2013 10:35 AM, Dan Williams wrote: On Wed, 2013-01-23 at 09:48 -0800, John Reiser wrote: The signal handler can write a packet into a pipe from the process to itself, and that can be hooked up to an event loop API. Clearly. But then you have to deal with signal handling and all the things you're not able to do from a signal handler like allocate memory, figure out any locking that may be required, and deal with signal handler re-entrancy. (...) This is known as the self-pipe trick. Doing a web search for that term will find lots of advice on how to use it properly. This is exactly what the signal+pipe scheme facilitates; you get to write a couple dozen lines of straight-line code. The main loop gets notified that the pipe has a packet, then calls the NM callback. All the NM callback has to do is read the fixed-length packet from the pipe, unwrap the answer, and feed it to the client's callback. Right. The issue with signal based interfaces is that they're fundamentally bad for multi-threading and libraries -- signal dispositions/handlers are global per process, not per thread. If two different libraries loaded into a process want to use/reserve the same signal (which also steals a signal from the main program), you're in trouble. See this 4 blog series about exactly this issue around wait/SIGCHLD in Qt/glib for example: http://www.macieira.org/blog/2012/07/forkfd-part-4-proposed-solutions/ -- Pedro Alves -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, 2013-01-23 at 02:46 +0100, Lennart Poettering wrote: On Mon, 21.01.13 10:25, Daniel P. Berrange (berra...@redhat.com) wrote: The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Bh. It's designed around process signal delivery. I am shuddering. Ew. Signals are not an event loop API. Signals are not an event loop API. Signals are not an event loop API. But apparently that's hard for some people to understand... Dan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, Jan 23, 2013 at 11:24:25AM -0600, Dan Williams wrote: On Wed, 2013-01-23 at 02:46 +0100, Lennart Poettering wrote: On Mon, 21.01.13 10:25, Daniel P. Berrange (berra...@redhat.com) wrote: The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Bh. It's designed around process signal delivery. I am shuddering. Ew. Signals are not an event loop API. Signals are not an event loop API. Signals are not an event loop API. But apparently that's hard for some people to understand... I never claimed it was a nice API, just that it provides an async hostname lookup capability already :-) Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On 01/23/2013 09:27 AM, Daniel P. Berrange wrote: On Wed, Jan 23, 2013 at 11:24:25AM -0600, Dan Williams wrote: On Wed, 2013-01-23 at 02:46 +0100, Lennart Poettering wrote: On Mon, 21.01.13 10:25, Daniel P. Berrange (berra...@redhat.com) wrote: The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Bh. It's designed around process signal delivery. I am shuddering. Ew. Signals are not an event loop API. Signals are not an event loop API. Signals are not an event loop API. But apparently that's hard for some people to understand... I never claimed it was a nice API, just that it provides an async hostname lookup capability already :-) The signal handler can write a packet into a pipe from the process to itself, and that can be hooked up to an event loop API. -- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, 2013-01-23 at 09:48 -0800, John Reiser wrote: On 01/23/2013 09:27 AM, Daniel P. Berrange wrote: On Wed, Jan 23, 2013 at 11:24:25AM -0600, Dan Williams wrote: On Wed, 2013-01-23 at 02:46 +0100, Lennart Poettering wrote: On Mon, 21.01.13 10:25, Daniel P. Berrange (berra...@redhat.com) wrote: The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Bh. It's designed around process signal delivery. I am shuddering. Ew. Signals are not an event loop API. Signals are not an event loop API. Signals are not an event loop API. But apparently that's hard for some people to understand... I never claimed it was a nice API, just that it provides an async hostname lookup capability already :-) The signal handler can write a packet into a pipe from the process to itself, and that can be hooked up to an event loop API. Clearly. But then you have to deal with signal handling and all the things you're not able to do from a signal handler like allocate memory, figure out any locking that may be required, and deal with signal handler re-entrancy. It's much easier for consumers of your library to give them an API that can be easily integrated into their event loop using select(). But of course that requires that you design your library in an asynchronous manner too [1], which is just good practice anyway so neither you nor your API clients don't have to deal with this stuff. Dan [1] eg, client asks for a name lookup and hands your library a callback function, your library constructs and sends the DNS request, then gives control back to the caller and lets them select() on the UDP socket, and when data is available they call back into your library so you can handle the response, and when you have one you call the client's callback -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, 23.01.13 12:35, Dan Williams (d...@redhat.com) wrote: Bh. It's designed around process signal delivery. I am shuddering. Ew. Signals are not an event loop API. Signals are not an event loop API. Signals are not an event loop API. But apparently that's hard for some people to understand... I never claimed it was a nice API, just that it provides an async hostname lookup capability already :-) The signal handler can write a packet into a pipe from the process to itself, and that can be hooked up to an event loop API. Clearly. But then you have to deal with signal handling and all the things you're not able to do from a signal handler like allocate memory, figure out any locking that may be required, and deal with signal handler re-entrancy. It's much easier for consumers of your library to give them an API that can be easily integrated into their event loop using select(). But of course that requires that you design your library in an asynchronous manner too [1], which is just good practice anyway so neither you nor your API clients don't have to deal with this stuff. Yeah, signal handlers are really hard to get right. I am pretty sure that if somebody took the time to go through the Fedora packages and would just check how many signal handlers forget to save/restore errno and write patches for those he'd get himself way up in the Ohloh statistics of people with the most patches in various projects. He'd probably even be able to get his name into a number of CVEs as a side effect... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On 01/23/2013 10:35 AM, Dan Williams wrote: On Wed, 2013-01-23 at 09:48 -0800, John Reiser wrote: The signal handler can write a packet into a pipe from the process to itself, and that can be hooked up to an event loop API. Clearly. But then you have to deal with signal handling and all the things you're not able to do from a signal handler like allocate memory, figure out any locking that may be required, and deal with signal handler re-entrancy. In cases like this one, it is easy to avoid all those problems. The packet is fixed length [and in this case probably includes both the actual request and the result], declared locally in the handler, allocated on the stack. A plain write() of the packet requires no locking (small writes on a pipe are atomic), and no concerns about re-entrancy (order cannot matter). The most that could matter is filling the pipe, but on linux you get at least 64kB before that happens, which should be enough for hundreds of packets that have not yet been processed by the main loop. If it matters, then use one atomic counter keep track of how many packets the pipe contains; you get to decide what to do if it overflows (probably: log the occurrence, then drop the packet.) It's much easier for consumers of your library to give them an API that can be easily integrated into their event loop using select().snip [1] eg, client asks for a name lookup and hands your library a callback function, your library constructs and sends the DNS request, then gives control back to the caller and lets them select() on the UDP socket, and when data is available they call back into your library so you can handle the response, and when you have one you call the client's callback This is exactly what the signal+pipe scheme facilitates; you get to write a couple dozen lines of straight-line code. The main loop gets notified that the pipe has a packet, then calls the NM callback. All the NM callback has to do is read the fixed-length packet from the pipe, unwrap the answer, and feed it to the client's callback. -- -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Mon, 21.01.13 10:25, Daniel P. Berrange (berra...@redhat.com) wrote: The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Bh. It's designed around process signal delivery. I am shuddering. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Fri, Jan 18, 2013 at 09:37:37AM +0800, Nick Jones wrote: On Wednesday, January 16, 2013 10:11 PM, Jaroslav Reznik wrote: As the Feature was renamed and the content of Feature has been extended as requested by FESCo, re-announcing it to the community for re-review. See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution * Detailed description Currently the getaddrinfo() function doesn't work as it was desinged. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future. One feature I would like to see added to future glibc, is a fully asynchronous version of getaddrinfo, and also getnameinfo. Asynchronous for non filesystem fds at least. The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. Perhaps I'm misunderstanding what you're after, but doesn't getaddrinfo_a already provide an async version of getaddrinfo ? Asynchronous Hostname Lookup API http://www.akkadia.org/drepper/asynchnl.pdf That said I don't see the reverse - getnameinfo_a Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wed, Jan 16, 2013 at 6:38 PM, Sam Varshavchik mr...@courier-mta.com wrote: Jaroslav Reznik writes: See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution Why would this be a Fedora issue or a feature? Seems to me this falls squarely inside the scope of glibc, and it should be taken care of in glibc upstream, rather than in Fedora. The feature page it may be necessary to update applications, either to take advantage of the fixes, or to remove workarounds. This might need cross-package coordination. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
On Wednesday, January 16, 2013 10:11 PM, Jaroslav Reznik wrote: As the Feature was renamed and the content of Feature has been extended as requested by FESCo, re-announcing it to the community for re-review. See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution * Detailed description Currently the getaddrinfo() function doesn't work as it was desinged. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future. One feature I would like to see added to future glibc, is a fully asynchronous version of getaddrinfo, and also getnameinfo. Asynchronous for non filesystem fds at least. The glibc maintainers don't seem to be against this idea and I am willing to put time into design and implementation. I have a number of high level ideas for this project that I would like to have documented as starting points. I am not a fedora community member, so someone will need to help me raise bugzillas and paste my notes on the wiki. Nick -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
As the Feature was renamed and the content of Feature has been extended as requested by FESCo, re-announcing it to the community for re-review. See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution * Detailed description Currently the getaddrinfo() function doesn't work as it was desinged. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future. - Rationale We are submitting this bug fixing effort as a Feature because: It is a high-impact change that will (positively) affect allmost all networking software Developers will be able to use getaddrinfo() without ugly workarounds for new code We are going to publish guidelines for proper getaddrinfo() usage Documentation for getaddrinfo() bugs will be availabe Possible workarounds will be offered for backward compatibility Comments and errata will be sent to standards organizations We want to recieve critical response during the whole process It will be part of the next networking testweek - Problem statement The behavior of getaddrinfo() is often nonstandard, undocumented, surprising, or just plain wrong. We already indentified a number of problems. The most prominent examples are here. getaddrinfo() may return duplicate or even wrong addresses from /etc/hosts getaddrinfo() with NULL servname may return duplicate addresses getaddrinfo() with AI_PASSIVE may still return address list not suitable for bind() getaddrinfo() with AI_ADDRCONFIG may fail to translate literal addresses getaddrinfo() with AI_ADDRCONFIG may fail to resolve /etc/hosts addresses getaddrinfo() with AI_ADDRCONFIG may send unwanted queries getaddrinfo() has a bad choice of default flags/code Whether or not the problematic actually occurs depends on /etc/hosts configuration, /etc/resolv.conf configuration, getaddrinfo() input parameters, runtime kernel network interface configuration, and more. While testing the known bugs or reading the source code, more and more bugs are discovered. Bug reports related to getaddrinfo() can be found upstream: http://sourceware.org/bugzilla/buglist.cgi?quicksearch=getaddrinfo -Affected software The above problems affect software that wants to use getaddrinfo() to: Get parameters for connect() or sendto() to start communicating Get parameters for bind() to listen on specific addresses Build IP address based accesslists Perform name resolution for other purposes Although it would be nice to also test and fix all software in Fedora using getaddrinfo(), that is not feasible. Therefore we are going to concentrate on checking and fixing the GNU C library, checking and fixing the most important toolkits dealing with networking, and documenting a set of guidelines for daemons and application software. Fedora bugs related to dualstack networking including name resolution problems should be added to the following tracker bug: http://bugzilla.redhat.com/show_bug.cgi?id=883152 - Original Message - As decided by FESCo on 2012-12-05 meeting, all proposed Features are required to pass through the community review by announcing them on devel-announce list. FESCo votes on new features no sooner than a week from the announcement. = Features/DualstackNetworking = https://fedoraproject.org/wiki/Features/DualstackNetworking * Detailed description Fedora supports dualstack global networking. That means the computer with Fedora is connected to internet using both IPv4 and IPv6 protocols. But many important system services and applications either don't do IPv6, do it incorrectly, or don't cope with various network conditions. Unfortunately, while trying to improve IPv6 support, some IPv4 use cases became broken as well. That's why the goal of this feature is not only to support IPv4, but to support all possible real-world cases. Dualstack-ready software must cope with all possible scenarios including IPv4-only connectivity, IPv6-only connectivity and dual connectivity. The software must also cope with node-local (aka localhost) networking, which as been used by software for decades. Though it would be nice to have all applications in Fedora fixed to work in any of the scenarios, it is not feasible to test that. Therefore this feature is about major software used in servers, desktops and laptops. The list of such applications will be completed over the time. Bugs related to dualstack networking should be added to the following tracker bug: http://bugzilla.redhat.com/show_bug.cgi?id=883152 Also see: https://bugzilla.redhat.com/show_bug.cgi?id=ipv6blocker ___ devel-announce mailing list
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
Jaroslav Reznik writes: See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution Why would this be a Fedora issue or a feature? Seems to me this falls squarely inside the scope of glibc, and it should be taken care of in glibc upstream, rather than in Fedora. pgpy_jtvM_4_Q.pgp Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
= Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution Why would this be a Fedora issue Isn't glibc used in Fedora for name resolution? or a feature? See https://fedoraproject.org/wiki/Features/FixNetworkNameResolution#Rationale Seems to me this falls squarely inside the scope of glibc Not entirely. See https://fedoraproject.org/wiki/Features/FixNetworkNameResolution#Affected_software and it should be taken care of in glibc upstream As much as possible, yes, as is the case with many other fedora features. rather than in Fedora. For me, the reliability and robustness of fedora networking is fairly important. I'd say more important than most of the “update package X to version Y” features. Submitting patches upstream is part of what makes Fedora a good distribution. Cheers, Pavel -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Reproposed F19 Feature: Fix Network Name Resolution [Was: DualstackNetworking]
As the Feature was renamed and the content of Feature has been extended as requested by FESCo, re-announcing it to the community for re-review. See https://fedorahosted.org/fesco/ticket/986 = Features/Fix Network Name Resolution = https://fedoraproject.org/wiki/Features/FixNetworkNameResolution * Detailed description Currently the getaddrinfo() function doesn't work as it was desinged. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future. - Rationale We are submitting this bug fixing effort as a Feature because: It is a high-impact change that will (positively) affect allmost all networking software Developers will be able to use getaddrinfo() without ugly workarounds for new code We are going to publish guidelines for proper getaddrinfo() usage Documentation for getaddrinfo() bugs will be availabe Possible workarounds will be offered for backward compatibility Comments and errata will be sent to standards organizations We want to recieve critical response during the whole process It will be part of the next networking testweek - Problem statement The behavior of getaddrinfo() is often nonstandard, undocumented, surprising, or just plain wrong. We already indentified a number of problems. The most prominent examples are here. getaddrinfo() may return duplicate or even wrong addresses from /etc/hosts getaddrinfo() with NULL servname may return duplicate addresses getaddrinfo() with AI_PASSIVE may still return address list not suitable for bind() getaddrinfo() with AI_ADDRCONFIG may fail to translate literal addresses getaddrinfo() with AI_ADDRCONFIG may fail to resolve /etc/hosts addresses getaddrinfo() with AI_ADDRCONFIG may send unwanted queries getaddrinfo() has a bad choice of default flags/code Whether or not the problematic actually occurs depends on /etc/hosts configuration, /etc/resolv.conf configuration, getaddrinfo() input parameters, runtime kernel network interface configuration, and more. While testing the known bugs or reading the source code, more and more bugs are discovered. Bug reports related to getaddrinfo() can be found upstream: http://sourceware.org/bugzilla/buglist.cgi?quicksearch=getaddrinfo -Affected software The above problems affect software that wants to use getaddrinfo() to: Get parameters for connect() or sendto() to start communicating Get parameters for bind() to listen on specific addresses Build IP address based accesslists Perform name resolution for other purposes Although it would be nice to also test and fix all software in Fedora using getaddrinfo(), that is not feasible. Therefore we are going to concentrate on checking and fixing the GNU C library, checking and fixing the most important toolkits dealing with networking, and documenting a set of guidelines for daemons and application software. Fedora bugs related to dualstack networking including name resolution problems should be added to the following tracker bug: http://bugzilla.redhat.com/show_bug.cgi?id=883152 - Original Message - As decided by FESCo on 2012-12-05 meeting, all proposed Features are required to pass through the community review by announcing them on devel-announce list. FESCo votes on new features no sooner than a week from the announcement. = Features/DualstackNetworking = https://fedoraproject.org/wiki/Features/DualstackNetworking * Detailed description Fedora supports dualstack global networking. That means the computer with Fedora is connected to internet using both IPv4 and IPv6 protocols. But many important system services and applications either don't do IPv6, do it incorrectly, or don't cope with various network conditions. Unfortunately, while trying to improve IPv6 support, some IPv4 use cases became broken as well. That's why the goal of this feature is not only to support IPv4, but to support all possible real-world cases. Dualstack-ready software must cope with all possible scenarios including IPv4-only connectivity, IPv6-only connectivity and dual connectivity. The software must also cope with node-local (aka localhost) networking, which as been used by software for decades. Though it would be nice to have all applications in Fedora fixed to work in any of the scenarios, it is not feasible to test that. Therefore this feature is about major software used in servers, desktops and laptops. The list of such applications will be completed over the time. Bugs related to dualstack networking should be added to the following tracker bug: http://bugzilla.redhat.com/show_bug.cgi?id=883152 Also see: https://bugzilla.redhat.com/show_bug.cgi?id=ipv6blocker ___ devel-announce mailing list
