Re: Standard C/C++ and Mozilla
On Wed, Aug 7, 2013 at 6:47 PM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: (Sorry for the late reply, please blame it on Canadian statutory holidays, and my birthday date!) Happy birthday! On Fri, Aug 2, 2013 at 11:09 PM, Brian Smith br...@briansmith.org wrote: 1. It avoids a phase of mass rewrites s/mozilla:Whatever/std::whatever/. (See below). 2. It is reasonable to expect that std::whatever works as the C++ standard says it should. It isn't reasonable to expect mozilla::Whatever to work exactly like std::whatever. And, often, mozilla::Whatever isn't actually the same as std::whatever. As Jeff mentioned, I think it's more important that we expect developers to read and believe the documentation where it exists. The MFBT code is very well documented, and the documentation is usually in sync with the implementation. That is already a huge improvement over the newer std::* stuff. std::auto_ptr is perhaps the biggest example of people not reading documentation about std::* stuff. ;-) Still, all things being equal, it is better to help developers use std::* instead of inventing something new, because it is likely (and increasingly more likely) that they already know how std::* works. (I note that you also made this point below.) But more importantly, as others mentioned, the fact that something lives in the std namespace doesn't mean that it adheres to the C++ standard. So it seems to me like you're assuming that code living in the std namespace is bug free, but that's not true. And when something lives in the std namespace, fixing it is very difficult. I agree it is very difficult to deal with bugs in standard libraries. Finding a bug in the implementation of std::whatever in a compiler/stdlib we can't upgrade is a good reason to create mozilla::Whatever. Worrying, ahead of time, that std::whatever might maybe have a bug in some implementation, isn't a good reason to create mozilla::Whatever. YAGNI. But for whatever it's worth, I think that in general, for the std replacement code living in MFBT, it's best for us to try really hard to match the C++ standard where it makes sense. We sometimes go through a crazy amount of pain to do that (see my patch in bug 802806 as an example!). But if something doesn't make sense in the C++ standard or is not fit for our needs, we should do the right thing, depending on the case at hand. snip But then again I don't find this argument very convincing in this particular case. I hope that this discussion has provided some good reasons why implementing out mozilla::Whatever sometimes makes sense. And later on when we decide to switch to std::whatever, I'd consider such a rewrite a net win because it makes our code easier to approach by people familiar with std::whatever. My argument is not that we should avoid creating mozilla::Whatever at all costs. My argument is that we should prefer upgrading compilers and/or just add/backport something to STLPort to writing new code in MFBT that is the same-but-different than what the standard C++ library provides. Obviously, if there is a major benefit to having our own thing, then we should do so. (I already gave the example of mozilla/Atomics.h.) About the mozilla-build sed issue, that is really really surprising/disappointing -- I'd have expected that we ship GNU sed there? Even bsd sed on Mac supports -i. Please file a bug about this with more details. https://bugzilla.mozilla.org/show_bug.cgi?id=373784 (reported 2007-03-13). Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Wed, Aug 7, 2013 at 6:47 PM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: But for whatever it's worth, I think that in general, for the std replacement code living in MFBT, it's best for us to try really hard to match the C++ standard where it makes sense. We sometimes go through a crazy amount of pain to do that (see my patch in bug 802806 as an example!). But if something doesn't make sense in the C++ standard or is not fit for our needs, we should do the right thing, depending on the case at hand. Bug 802806 is about fixing a bug in part of mozilla::TypeTraits, which is designed to be line std::type_traits. The reason we can't use std::type_traits (from my reading of bug 900040) is that STLPort's implementation is incorrect. So, I think this is a good example of where we disagree. My position is that we should fix STLPort's implementation for GCC 4.4 ARM Linux (maybe just backport a fixed version) and use std::type_traits everywhere. Question: What, precisely, are the differences in semantics between the similarly-named functions in mozilla:TypeTraits and std::type_traits? I think it is hard to definitively answer this question and that's why I'd like us to consider creating things like mozilla::TypeTraits a last-resort option behind fixing STLPort for GCC 4.4 ARM Linux or dropping support for old versions of compilers. (This isn't to say that the implementation or documentation for mozilla::TypeTraits is bad; it is very impressive. Just, I wonder if we needed to spend as much effort on it compared to the alternatives.) Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 2013-08-08 11:34 AM, Brian Smith wrote: On Wed, Aug 7, 2013 at 6:47 PM, Ehsan Akhgari ehsan.akhg...@gmail.com mailto:ehsan.akhg...@gmail.com wrote: But for whatever it's worth, I think that in general, for the std replacement code living in MFBT, it's best for us to try really hard to match the C++ standard where it makes sense. We sometimes go through a crazy amount of pain to do that (see my patch in bug 802806 as an example!). But if something doesn't make sense in the C++ standard or is not fit for our needs, we should do the right thing, depending on the case at hand. Bug 802806 is about fixing a bug in part of mozilla::TypeTraits, which is designed to be line std::type_traits. The reason we can't use std::type_traits (from my reading of bug 900040) is that STLPort's implementation is incorrect. So, I think this is a good example of where we disagree. My position is that we should fix STLPort's implementation for GCC 4.4 ARM Linux (maybe just backport a fixed version) and use std::type_traits everywhere. Question: What, precisely, are the differences in semantics between the similarly-named functions in mozilla:TypeTraits and std::type_traits? I think it is hard to definitively answer this question and that's why I'd like us to consider creating things like mozilla::TypeTraits a last-resort option behind fixing STLPort for GCC 4.4 ARM Linux or dropping support for old versions of compilers. (This isn't to say that the implementation or documentation for mozilla::TypeTraits is bad; it is very impressive. Just, I wonder if we needed to spend as much effort on it compared to the alternatives.) Now that I know we can modify our STLport as needed, I do agree that it is preferred to fix that and use std::type_traits instead. At that time, I did not have that knowledge. And we want to move on from gcc 4.4. for other reasons anyway, so perhaps there isn't anything that we disagree on? Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Thu, Aug 08, 2013 at 08:15:29PM -0700, Brian Smith wrote: On Thu, Aug 8, 2013 at 3:57 PM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: On 2013-08-08 11:34 AM, Brian Smith wrote: My position is that we should fix STLPort's implementation for GCC 4.4 ARM Linux (maybe just backport a fixed version) and use std::type_traits everywhere. Now that I know we can modify our STLport as needed, I do agree that it is preferred to fix that and use std::type_traits instead. At that time, I did not have that knowledge. And we want to move on from gcc 4.4. for other reasons anyway, so perhaps there isn't anything that we disagree on. Great. I actually didn't know about the STLPort stuff either when this conversation started. One final thing: Will an upgrade from GCC 4.4 to GCC 4.7 reduce or eliminate the need for us to use STLPort? Is there any documentation on what causes us to depend on STLPort? My guess is that soon STLPort, and not GCC or clang or MSVC limitations, is what is going to be slowing us down, so perhaps we should have a plan for getting rid of the need for it. Building with libstdc++ on android simply doesn't work at the moment. It also used to produce significantly bigger packages when it used to work. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
(Sorry for the late reply, please blame it on Canadian statutory holidays, and my birthday date!) On Fri, Aug 2, 2013 at 11:09 PM, Brian Smith br...@briansmith.org wrote: On Sat, Aug 3, 2013 at 12:47 AM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: On 2013-08-02 5:21 PM, Brian Smith wrote: 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. Yes, but it's still not clear to me why you prefer this. 1. It avoids a phase of mass rewrites s/mozilla:Whatever/std::whatever/. (See below). 2. It is reasonable to expect that std::whatever works as the C++ standard says it should. It isn't reasonable to expect mozilla::Whatever to work exactly like std::whatever. And, often, mozilla::Whatever isn't actually the same as std::whatever. As Jeff mentioned, I think it's more important that we expect developers to read and believe the documentation where it exists. The MFBT code is very well documented, and the documentation is usually in sync with the implementation. That is already a huge improvement over the newer std::* stuff. std::auto_ptr is perhaps the biggest example of people not reading documentation about std::* stuff. ;-) But more importantly, as others mentioned, the fact that something lives in the std namespace doesn't mean that it adheres to the C++ standard. So it seems to me like you're assuming that code living in the std namespace is bug free, but that's not true. And when something lives in the std namespace, fixing it is very difficult. But for whatever it's worth, I think that in general, for the std replacement code living in MFBT, it's best for us to try really hard to match the C++ standard where it makes sense. We sometimes go through a crazy amount of pain to do that (see my patch in bug 802806 as an example!). But if something doesn't make sense in the C++ standard or is not fit for our needs, we should do the right thing, depending on the case at hand. This saves us from the massive rewrites later to s/mozilla::Whatever/std::**whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. Disruptive in what sense? I recently did two of these kinds of conversions and nobody complained. You have to rebase all your patches in your patch queue and/or run scripts on your patches (that, IIRC, don't run on windows because mozilla-build doesn't have sed -i). I'm not complaining about the conversions you've done, because they are net wins. But, it's still less disruptive to avoid unnecessary rounds of rewrites when possible, and s/mozilla::Whatever/std::whatever/ seems unnecessary to me when we could have just named mozilla::Whatever std::whatever to start with. I agree that huge refactorings can be a pain. I've been trying to do that in smaller chunks to alleviate some of the pain (see bug 784739 as an example.) If you see people making unjustified huge changes without prior discussion, you should definitely object! But then again I don't find this argument very convincing in this particular case. I hope that this discussion has provided some good reasons why implementing out mozilla::Whatever sometimes makes sense. And later on when we decide to switch to std::whatever, I'd consider such a rewrite a net win because it makes our code easier to approach by people familiar with std::whatever. About the mozilla-build sed issue, that is really really surprising/disappointing -- I'd have expected that we ship GNU sed there? Even bsd sed on Mac supports -i. Please file a bug about this with more details. Cheers, -- Ehsan http://ehsanakhgari.org/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 08/02/2013 08:09 PM, Brian Smith wrote: 2. It is reasonable to expect that std::whatever works as the C++ standard says it should. It isn't reasonable to expect mozilla::Whatever to work exactly like std::whatever. And, often, mozilla::Whatever isn't actually the same as std::whatever. Why is this a problem? People should never assume mozilla::* is the same as std::*, unless they verify in the documentation that it is. Sometimes our documentation makes promises of compatibility (or approximate compatibility, or common-case compatibility). Usually it doesn't. Caveat lector. Reading documentation, it seems to me, has to be a baseline requirement for all Mozilla contributors. If a contributor isn't reading documentation about the classes/structures/algorithms he uses in the patches he writes, I'm scared. (Which is not to say that doc-reading is always the solution, as sometimes our docs are incoherent, scattered, or non-existent. But for this new stuff, the docs come with the code, and are a prerequisite to landing. I think we've held that line pretty well. So that's a legacy problem not relevant here.) Jeff ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Wed, Jul 31, 2013 at 7:41 PM, Joshua Cranmer pidgeo...@gmail.comwrote: implementation, libc++, libstdc++, and stlport. Since most nice charts of C++11 compatibility focus on what the compiler needs to do, I've put together a high-level overview of the major additions to the standard library [3]: * std::function/std::bind -- Generalization of function pointers Note that Eric Rescorla implemented his own std::bind polyfill when he was working on WebRTC. I also have some new code I am working on where std::bind is extremely helpful. Now that you have the background for what is or will be in standard C++, let me discuss the real question I want to discuss: how much of this should we be using in Mozilla? For purposes of discussion, I think it's worth breaking down the C++ (and C) standard library into the following components: * Containers--vector, map, etc. * Strings * I/O * Platform support (threading, networking, filesystems, locales) * Other helpful utilities (std::random, std::tuple, etc.) The iostream library has some issues with using (particularly static constructors IIRC), and is not so usable for most of the things that Gecko needs to do. It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. Even if fully using the standard library is untenable from a performance perspective, usability may be enhanced if we align some of our APIs which mimic STL functionality with the actual STL APIs. For example, we could add begin()/end()/push_back()/etc. methods to nsTArray to make it a fairly drop-in replacement for std::vector, or at least close enough to one that it could be used in other STL APIs (like std::sort, std::find, etc.). However, this does create massive incongruities in our API, since the standard library prefers naming stuff with this_kind_of_convention whereas most Mozilla style guides prefer ThisKindOfConvention. Perhaps a more annoying issue--though not a showstoper--is that unique_ptr::release() means something quite different than nsXXXPtr::Release() means. With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We should definitely prefer using the standard C++ library over writing any new code for MFBT, *unless* there is consensus that the new thing we'd do in MFBT is substantially clearer. (For example, I think some people successfully argued that we should have our own atomic types because our interface is clearly better than std::atomic.) Even in the case where MFBT or XPCOM stuff is generally better, We should *allow* using the standard C++ library anywhere that has additional constraints that warrant a different tradeoff; e.g. needing to be built separately from Gecko and/or otherwise needing to minimize Gecko dependencies. 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. This saves us from the massive rewrites later to s/mozilla::Whatever/std::whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. In the case where it is just STLPort being behind, we should just add the thing to STLPort (and try to upstream it). in the case where the lack of support for a useful standard library feature is more widespread, we should still implement std::whatever if the language support we have enables us to do so. I am not sure where such implementations should live. 4. When should we prefer our own implementations to standard library implementations? It is a judgement call. The default should be to use standard library functions, but we shouldn't be shy about using our own stuff if it is clearly better. On the other side, we shouldn't be shy about replacing uses of same-thing-but-different Mozilla-specific libraries with uses of the standard libraries, all things being equal. 5. To what degree should our platform-bridging libraries (xpcom/mfbt/necko/nspr) use or align with the C++ standard library? I am not sure why you include Necko in that list. Did you mean NSS? For NSPR and NSS, I would like to include some very basic utilities like ScopedPRFileDesc that are included directly in NSPR/NSS, so that we can use them in GTest-based tests, even if NSPR and NSS otherwise stick with C. But, I don't know if the module owners of those modules will accept them. 6. Where support for an API
Re: Standard C/C++ and Mozilla
It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. I found this comment interesting. It wasn't that long ago I was instructed to get rid of all iostream-based logging from media/webrtc/signaling and media/mtransport if it we wanted the logging to appear in opt builds. https://bugzilla.mozilla.org/show_bug.cgi?id=795126 https://bugzilla.mozilla.org/show_bug.cgi?id=841899 I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 Can we now use iostreams throughout this code? -EH On Fri, Aug 2, 2013 at 2:21 PM, Brian Smith br...@briansmith.org wrote: On Wed, Jul 31, 2013 at 7:41 PM, Joshua Cranmer pidgeo...@gmail.com wrote: implementation, libc++, libstdc++, and stlport. Since most nice charts of C++11 compatibility focus on what the compiler needs to do, I've put together a high-level overview of the major additions to the standard library [3]: * std::function/std::bind -- Generalization of function pointers Note that Eric Rescorla implemented his own std::bind polyfill when he was working on WebRTC. I also have some new code I am working on where std::bind is extremely helpful. Now that you have the background for what is or will be in standard C++, let me discuss the real question I want to discuss: how much of this should we be using in Mozilla? For purposes of discussion, I think it's worth breaking down the C++ (and C) standard library into the following components: * Containers--vector, map, etc. * Strings * I/O * Platform support (threading, networking, filesystems, locales) * Other helpful utilities (std::random, std::tuple, etc.) The iostream library has some issues with using (particularly static constructors IIRC), and is not so usable for most of the things that Gecko needs to do. It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. Even if fully using the standard library is untenable from a performance perspective, usability may be enhanced if we align some of our APIs which mimic STL functionality with the actual STL APIs. For example, we could add begin()/end()/push_back()/etc. methods to nsTArray to make it a fairly drop-in replacement for std::vector, or at least close enough to one that it could be used in other STL APIs (like std::sort, std::find, etc.). However, this does create massive incongruities in our API, since the standard library prefers naming stuff with this_kind_of_convention whereas most Mozilla style guides prefer ThisKindOfConvention. Perhaps a more annoying issue--though not a showstoper--is that unique_ptr::release() means something quite different than nsXXXPtr::Release() means. With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We should definitely prefer using the standard C++ library over writing any new code for MFBT, *unless* there is consensus that the new thing we'd do in MFBT is substantially clearer. (For example, I think some people successfully argued that we should have our own atomic types because our interface is clearly better than std::atomic.) Even in the case where MFBT or XPCOM stuff is generally better, We should *allow* using the standard C++ library anywhere that has additional constraints that warrant a different tradeoff; e.g. needing to be built separately from Gecko and/or otherwise needing to minimize Gecko dependencies. 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. This saves us from the massive rewrites later to s/mozilla::Whatever/std::whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. In the case where it is just STLPort being behind, we should just add the thing to STLPort (and try to upstream it). in the case where the lack of support for a useful standard library feature is more
Re: Standard C/C++ and Mozilla
I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 I can't access that bug, but maybe you mean https://bugzilla.mozilla.org/show_bug.cgi?id=onelogger ? I feel like the goals there are orthogonal to NSPR vs iostream. I haven't had a chance to work on this lately, but I do intend to land something when I can. On Fri, Aug 2, 2013 at 2:41 PM, Ethan Hugg ethanh...@gmail.com wrote: It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. I found this comment interesting. It wasn't that long ago I was instructed to get rid of all iostream-based logging from media/webrtc/signaling and media/mtransport if it we wanted the logging to appear in opt builds. https://bugzilla.mozilla.org/show_bug.cgi?id=795126 https://bugzilla.mozilla.org/show_bug.cgi?id=841899 I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 Can we now use iostreams throughout this code? -EH On Fri, Aug 2, 2013 at 2:21 PM, Brian Smith br...@briansmith.org wrote: On Wed, Jul 31, 2013 at 7:41 PM, Joshua Cranmer pidgeo...@gmail.com wrote: implementation, libc++, libstdc++, and stlport. Since most nice charts of C++11 compatibility focus on what the compiler needs to do, I've put together a high-level overview of the major additions to the standard library [3]: * std::function/std::bind -- Generalization of function pointers Note that Eric Rescorla implemented his own std::bind polyfill when he was working on WebRTC. I also have some new code I am working on where std::bind is extremely helpful. Now that you have the background for what is or will be in standard C++, let me discuss the real question I want to discuss: how much of this should we be using in Mozilla? For purposes of discussion, I think it's worth breaking down the C++ (and C) standard library into the following components: * Containers--vector, map, etc. * Strings * I/O * Platform support (threading, networking, filesystems, locales) * Other helpful utilities (std::random, std::tuple, etc.) The iostream library has some issues with using (particularly static constructors IIRC), and is not so usable for most of the things that Gecko needs to do. It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. Even if fully using the standard library is untenable from a performance perspective, usability may be enhanced if we align some of our APIs which mimic STL functionality with the actual STL APIs. For example, we could add begin()/end()/push_back()/etc. methods to nsTArray to make it a fairly drop-in replacement for std::vector, or at least close enough to one that it could be used in other STL APIs (like std::sort, std::find, etc.). However, this does create massive incongruities in our API, since the standard library prefers naming stuff with this_kind_of_convention whereas most Mozilla style guides prefer ThisKindOfConvention. Perhaps a more annoying issue--though not a showstoper--is that unique_ptr::release() means something quite different than nsXXXPtr::Release() means. With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We should definitely prefer using the standard C++ library over writing any new code for MFBT, *unless* there is consensus that the new thing we'd do in MFBT is substantially clearer. (For example, I think some people successfully argued that we should have our own atomic types because our interface is clearly better than std::atomic.) Even in the case where MFBT or XPCOM stuff is generally better, We should *allow* using the standard C++ library anywhere that has additional constraints that warrant a different tradeoff; e.g. needing to be built separately from Gecko and/or otherwise needing to minimize Gecko dependencies. 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement
Re: Standard C/C++ and Mozilla
Sorry I should've noticed that 855335 is a sec-bug. It's title is Audit SIPCC printf-style format strings which means we went through every logging call and repaired a few which had incorrect printf-style args. -EH On Fri, Aug 2, 2013 at 2:44 PM, Justin Lebar justin.le...@gmail.com wrote: I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 I can't access that bug, but maybe you mean https://bugzilla.mozilla.org/show_bug.cgi?id=onelogger ? I feel like the goals there are orthogonal to NSPR vs iostream. I haven't had a chance to work on this lately, but I do intend to land something when I can. On Fri, Aug 2, 2013 at 2:41 PM, Ethan Hugg ethanh...@gmail.com wrote: It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. I found this comment interesting. It wasn't that long ago I was instructed to get rid of all iostream-based logging from media/webrtc/signaling and media/mtransport if it we wanted the logging to appear in opt builds. https://bugzilla.mozilla.org/show_bug.cgi?id=795126 https://bugzilla.mozilla.org/show_bug.cgi?id=841899 I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 Can we now use iostreams throughout this code? -EH On Fri, Aug 2, 2013 at 2:21 PM, Brian Smith br...@briansmith.org wrote: On Wed, Jul 31, 2013 at 7:41 PM, Joshua Cranmer pidgeo...@gmail.com wrote: implementation, libc++, libstdc++, and stlport. Since most nice charts of C++11 compatibility focus on what the compiler needs to do, I've put together a high-level overview of the major additions to the standard library [3]: * std::function/std::bind -- Generalization of function pointers Note that Eric Rescorla implemented his own std::bind polyfill when he was working on WebRTC. I also have some new code I am working on where std::bind is extremely helpful. Now that you have the background for what is or will be in standard C++, let me discuss the real question I want to discuss: how much of this should we be using in Mozilla? For purposes of discussion, I think it's worth breaking down the C++ (and C) standard library into the following components: * Containers--vector, map, etc. * Strings * I/O * Platform support (threading, networking, filesystems, locales) * Other helpful utilities (std::random, std::tuple, etc.) The iostream library has some issues with using (particularly static constructors IIRC), and is not so usable for most of the things that Gecko needs to do. It is very useful for building a logging interface that is safer and more convenient than NSPR's printf-style logging. Note that, again, Eric Rescorla already built an (partial) iostream-based wrapper around NSPR for WebRTC. I would say that, if there is no additional overhead, then we should consider making iostream-based logging the default way of doing things in Gecko because it is so much less error-prone. Even if fully using the standard library is untenable from a performance perspective, usability may be enhanced if we align some of our APIs which mimic STL functionality with the actual STL APIs. For example, we could add begin()/end()/push_back()/etc. methods to nsTArray to make it a fairly drop-in replacement for std::vector, or at least close enough to one that it could be used in other STL APIs (like std::sort, std::find, etc.). However, this does create massive incongruities in our API, since the standard library prefers naming stuff with this_kind_of_convention whereas most Mozilla style guides prefer ThisKindOfConvention. Perhaps a more annoying issue--though not a showstoper--is that unique_ptr::release() means something quite different than nsXXXPtr::Release() means. With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We should definitely prefer using the standard C++ library over writing any new code for MFBT, *unless* there is consensus that the new thing we'd do in MFBT is substantially clearer. (For example, I think some people successfully argued that we should have our own atomic types because our interface is clearly better than std::atomic.) Even in the case where
Re: Standard C/C++ and Mozilla
On 2013-08-02 5:21 PM, Brian Smith wrote: 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. Yes, but it's still not clear to me why you prefer this. This saves us from the massive rewrites later to s/mozilla::Whatever/std::whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. Disruptive in what sense? I recently did two of these kinds of conversions and nobody complained. In the case where it is just STLPort being behind, we should just add the thing to STLPort (and try to upstream it). in the case where the lack of support for a useful standard library feature is more widespread, we should still implement std::whatever if the language support we have enables us to do so. I am not sure where such implementations should live. Yes, upstreaming fixes is clearly ideal, but sometimes pragmatism wins. For example, I personally wouldn't have the first clue what I need to do in order to modify STLport (how to make b2g/Android builds use my modified library, how to upstream the fix, what to do when we pick up the changes, how long that would take, what to do if my changes are not accepted upstream, etc.) 4. When should we prefer our own implementations to standard library implementations? It is a judgement call. The default should be to use standard library functions, but we shouldn't be shy about using our own stuff if it is clearly better. On the other side, we shouldn't be shy about replacing uses of same-thing-but-different Mozilla-specific libraries with uses of the standard libraries, all things being equal. If you agree that it's a judgement call, then prescribing what the default should be is, well, also a judgement call! 6. Where support for an API we wish to use is not universal, what is the preferred way to mock that support? [Note: similar questions also apply to NSPR and NSS with respect to newer C99 and C11 functionality.] There is no tolerance for mass changes like s/PRInt32/int32_t/ in NSPR or NSS, AFAICT. We mostly treat those libraries as read-only anyway, for the better or worse. C99 and C11 are basically off the table too, because Microsoft refuses to support them in MSVC. Yes, focusing on improving C code like this is a lost cause. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
It was brought to my attention that it's unclear what I'm looking for in this conversation, so let me try to summarize. I am not convinced that there is something wrong with the way that things currently work (polyfill where the feature is not available everywhere, else use it if appropriate). I'm trying to understand what the shortcomings of the current behavior is. I think it's best if we knew which problems we're trying to solve first. Cheers, -- Ehsan http://ehsanakhgari.org/ On Fri, Aug 2, 2013 at 6:47 PM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: On 2013-08-02 5:21 PM, Brian Smith wrote: 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. Yes, but it's still not clear to me why you prefer this. This saves us from the massive rewrites later to s/mozilla::Whatever/std::**whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. Disruptive in what sense? I recently did two of these kinds of conversions and nobody complained. In the case where it is just STLPort being behind, we should just add the thing to STLPort (and try to upstream it). in the case where the lack of support for a useful standard library feature is more widespread, we should still implement std::whatever if the language support we have enables us to do so. I am not sure where such implementations should live. Yes, upstreaming fixes is clearly ideal, but sometimes pragmatism wins. For example, I personally wouldn't have the first clue what I need to do in order to modify STLport (how to make b2g/Android builds use my modified library, how to upstream the fix, what to do when we pick up the changes, how long that would take, what to do if my changes are not accepted upstream, etc.) 4. When should we prefer our own implementations to standard library implementations? It is a judgement call. The default should be to use standard library functions, but we shouldn't be shy about using our own stuff if it is clearly better. On the other side, we shouldn't be shy about replacing uses of same-thing-but-different Mozilla-specific libraries with uses of the standard libraries, all things being equal. If you agree that it's a judgement call, then prescribing what the default should be is, well, also a judgement call! 6. Where support for an API we wish to use is not universal, what is the preferred way to mock that support? [Note: similar questions also apply to NSPR and NSS with respect to newer C99 and C11 functionality.] There is no tolerance for mass changes like s/PRInt32/int32_t/ in NSPR or NSS, AFAICT. We mostly treat those libraries as read-only anyway, for the better or worse. C99 and C11 are basically off the table too, because Microsoft refuses to support them in MSVC. Yes, focusing on improving C code like this is a lost cause. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Fri, Aug 02, 2013 at 06:47:08PM -0400, Ehsan Akhgari wrote: On 2013-08-02 5:21 PM, Brian Smith wrote: 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. Yes, but it's still not clear to me why you prefer this. This saves us from the massive rewrites later to s/mozilla::Whatever/std::whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. Disruptive in what sense? I recently did two of these kinds of conversions and nobody complained. In the case where it is just STLPort being behind, we should just add the thing to STLPort (and try to upstream it). in the case where the lack of support for a useful standard library feature is more widespread, we should still implement std::whatever if the language support we have enables us to do so. I am not sure where such implementations should live. Yes, upstreaming fixes is clearly ideal, but sometimes pragmatism wins. For example, I personally wouldn't have the first clue what I need to do in order to modify STLport (how to make b2g/Android builds use my modified library, how to upstream the fix, what to do when we pick up the changes, how long that would take, what to do if my changes are not accepted upstream, etc.) Android and b2g are now using an in-tree STLport copy. We can patch it if necessary, and it will be used everywhere. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Fri, Aug 02, 2013 at 02:44:57PM -0700, Justin Lebar wrote: I agree that iostream-based logging would be safer. If we had it I wouldn't have had to work on this one: https://bugzilla.mozilla.org/show_bug.cgi?id=855335 I can't access that bug, but maybe you mean https://bugzilla.mozilla.org/show_bug.cgi?id=onelogger ? I feel like the goals there are orthogonal to NSPR vs iostream. I haven't had a chance to work on this lately, but I do intend to land something when I can. That sounds similar to what i wanted to do in bug 602467, but i never got to get anywhere with it, besides ideas in some parts of my brain. And yes, being free of static initializers is a must-have feature imho. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Sat, Aug 3, 2013 at 12:47 AM, Ehsan Akhgari ehsan.akhg...@gmail.comwrote: On 2013-08-02 5:21 PM, Brian Smith wrote: 3. How should we handle bridge support for standardized features not yet universally-implemented? Generally, I would much rather we implement std::whatever ourselves than implement mozilla::Whatever, all other things being equal. Yes, but it's still not clear to me why you prefer this. 1. It avoids a phase of mass rewrites s/mozilla:Whatever/std::whatever/. (See below). 2. It is reasonable to expect that std::whatever works as the C++ standard says it should. It isn't reasonable to expect mozilla::Whatever to work exactly like std::whatever. And, often, mozilla::Whatever isn't actually the same as std::whatever. This saves us from the massive rewrites later to s/mozilla::Whatever/std::**whatever/; while such rewrites are generally a net win, they are still disruptive enough to warrant trying to avoid them when possible. Disruptive in what sense? I recently did two of these kinds of conversions and nobody complained. You have to rebase all your patches in your patch queue and/or run scripts on your patches (that, IIRC, don't run on windows because mozilla-build doesn't have sed -i). I'm not complaining about the conversions you've done, because they are net wins. But, it's still less disruptive to avoid unnecessary rounds of rewrites when possible, and s/mozilla::Whatever/std::whatever/ seems unnecessary to me when we could have just named mozilla::Whatever std::whatever to start with. Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 8/2/2013 10:09 PM, Brian Smith wrote: 2. It is reasonable to expect that std::whatever works as the C++ standard says it should. It isn't reasonable to expect mozilla::Whatever to work exactly like std::whatever. And, often, mozilla::Whatever isn't actually the same as std::whatever Judging by the recent record of std::atomic and the investigations into std::is_pod, this is not necessarily the case. STLport's std::is_pod does not return the correct answer for classes (i.e., the use case you probably most care about). Our ability to use std::atomic from libstdc++ has been steadily reduced due to it not supporting the feature set we want; just recently, we now require libstdc++ 4.7 to use it so we can get it to work with enums. Granted, atomic and type_traits are probably the most compiler-dependent headers of the lot, but it goes to show that just because it is implemented by a standard library doesn't mean it is correctly implemented. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
Joshua Cranmer wrote:
On 7/31/2013 9:19 PM, Mike Hommey wrote:
Now think of all those NS_LITERAL_STRING() and other horrible
boilerplate we have.
... and my next target is s/PRUnichar/char16_t/, the last step of
which basically amounts to killing NS_LITERAL_STRING. :-)
Will that include this:
public:
templateint N
nsTSubstring_CharT(const char_type (str)[N])
: mData(str)
, mLength(N - 1)
, mFlags(F_TERMINATED) {}
--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
Neil wrote:
Joshua Cranmer wrote:
On 7/31/2013 9:19 PM, Mike Hommey wrote:
Now think of all those NS_LITERAL_STRING() and other horrible
boilerplate we have.
... and my next target is s/PRUnichar/char16_t/, the last step of
which basically amounts to killing NS_LITERAL_STRING. :-)
Will that include this:
public:
templateint N
nsTSubstring_CharT(const char_type (str)[N])
: mData(str)
, mLength(N - 1)
, mFlags(F_TERMINATED) {}
Actually it would be neat to have a second flag there F_LITERAL which
makes all assigments depend on the original string.
--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 2013-07-31 1:41 PM, Joshua Cranmer wrote: With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? I'm not sure if it's easy to have this discussion in general without talking about a specific standard library feature. 2. To what degree should our custom ADTs (like nsTArray) be interoperable with the C++ standard library? I think some people would like to make the code more understandable to newer contributors, and some people would prefer to keep existing convention intact. 3. How should we handle bridge support for standardized features not yet universally-implemented? I think MFBT has been working fine so far. 4. When should we prefer our own implementations to standard library implementations? Usually doing our own implementation is faster, but there is a significant lag from contributing something upstream (if that's even possible) until that gets released in a toolchain that people use. So perhaps we should do both in parallel when there is the option. 5. To what degree should our platform-bridging libraries (xpcom/mfbt/necko/nspr) use or align with the C++ standard library? This is also hard to talk about without having a concrete thing under consideration. 6. Where support for an API we wish to use is not universal, what is the preferred way to mock that support? MFBT, I believe. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We've tuned tarray, nsthashtable, strings, etc. to meet our precise needs, and the implementations are consistent across all platforms. I can imagine things becoming quite messy we had three or four different implementations of these classes (in the different stdlibs), each with their own quirks, and if we couldn't change the implementations to meet our needs. I definitely think that some of our APIs could use some love, but it seems unlikely to me that replacing a complex class like nsTArray with std::vector would be a net win for us, as compared to simply improving nsTArray's interface. Even performing this experiment would be an expensive endeavor. By way of contrast, I think it's great that we're using simple classes, functions, and types from stdlib, such as static_assert and stdint. The downside here is much smaller, since we don't have to worry about the quirks of the different implementations, and since there's nothing we might want to tune. -Justin On Wed, Jul 31, 2013 at 12:08 PM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: On 2013-07-31 1:41 PM, Joshua Cranmer wrote: With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? I'm not sure if it's easy to have this discussion in general without talking about a specific standard library feature. 2. To what degree should our custom ADTs (like nsTArray) be interoperable with the C++ standard library? I think some people would like to make the code more understandable to newer contributors, and some people would prefer to keep existing convention intact. 3. How should we handle bridge support for standardized features not yet universally-implemented? I think MFBT has been working fine so far. 4. When should we prefer our own implementations to standard library implementations? Usually doing our own implementation is faster, but there is a significant lag from contributing something upstream (if that's even possible) until that gets released in a toolchain that people use. So perhaps we should do both in parallel when there is the option. 5. To what degree should our platform-bridging libraries (xpcom/mfbt/necko/nspr) use or align with the C++ standard library? This is also hard to talk about without having a concrete thing under consideration. 6. Where support for an API we wish to use is not universal, what is the preferred way to mock that support? MFBT, I believe. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 7/31/2013 2:08 PM, Ehsan Akhgari wrote: On 2013-07-31 1:41 PM, Joshua Cranmer wrote: With all of that stated, the questions I want to pose to the community at large are as follows: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? I'm not sure if it's easy to have this discussion in general without talking about a specific standard library feature. I'm most particularly referring to nsTArray, nsTHashtable and friends from the STL, as well as ns*String as Mozilla ADTs. In terms of all of the other code, I'm mostly referring to the large list of new APIs I provided in C++11 and C++14 as things we might want to use--std::chrono, std::thread and all of its friends, std::unique_ptr, std::function, std::tuple, and std::optional are the ones that look the most useful (std::function in particular could be superior to function pointers in our crappy EnumerateForwards-like methods). std::string_view, and the Filesystem and Networking draft TSs are also APIs (not yet finalized to any degree) that might influence potential changes we could make to our current non-IDL API regime. 3. How should we handle bridge support for standardized features not yet universally-implemented? I think MFBT has been working fine so far. I should be more clear: I'm talking about library features, like type traits, that aren't available in the complete selection we have available. Since we generally haven't been working around incomplete library features (with the exception of std::atomic which is... touchy to say the least), we don't really have a good example of what an intermediate stage looks like. Basically, should we: a) Implement mozilla::Duration to polyfill std::chrono::duration until it is available everywhere, then mass switch. b) Implement mozilla::duration/mozilla::chrono::duration until available everywhere, then mass switch. c) Implement mozilla::std::chrono::duration until available everywhere, then mass switch. c) Implement std::chrono::duration until available everywhere, then delete our polyfill header. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 7/31/2013 2:38 PM, Justin Lebar wrote: 1. How much, and where, should we be using standard C++ library functionality in Mozilla code? We've tuned tarray, nsthashtable, strings, etc. to meet our precise needs, and the implementations are consistent across all platforms. I can imagine things becoming quite messy we had three or four different implementations of these classes (in the different stdlibs), each with their own quirks, and if we couldn't change the implementations to meet our needs. For what it's worth, I don't think we can tenably replace nsTArray, nsTHashtable, or ns*String with std:: counterparts across the entire tree and expect any kind of performance enhancement. We also have needs like sizeOfIncludingThis/sizeOfExcludingThis that can't be as easily satisfied with STL code. Replacing const nsA[C]String with a std::string_view-esque class might provide a small performance enhancement, but it would need some experimentation and boils the ocean. It is possible to make our APIs match std::vector in usage, such that nsTArray and std::vector could be used fairly interchangeably, and that strikes me as a possibly worthwhile goal. On the other hand, we have an increasing number of semi-autonomous C++ side libraries which are already using the STL. Since I don't do development outside of the XPCOM-riddled hallways of Mozilla code, I don't know to what degree the incongruous APIs cause friction at these margins, but I would find it hard to be believe that no friction is being generated. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On 7/31/2013 9:19 PM, Mike Hommey wrote: On Wed, Jul 31, 2013 at 12:41:12PM -0500, Joshua Cranmer ? wrote: Thoughts/comments/corrections/questions/concerns/flames/insightful discussion? My feeling is that, while these are interesting questions, they are one step ahead. I think we should step back and start by defining what we want to achieve. I think the end goal should be for our code to be more idiomatic, and less boilerplate-y. Does that mean we should use more STL? maybe, but I'm not convinced it's the main concern. Probably most of our boilerplate issues comes from the stilted nature of XPIDL and XPCOM; deCOMtamination would solve a lot of issues. In same cases, it may desirable to add more C++-y APIs to things largely dominated by XPIDL (networking code is the prime example here); other places already have relatively tolerable C++ APIs ready to use (mozilla::Preferences, say). XPIDL is presently the only easy way to get both C++ and JS bindings to code, but the internals of how it works means that the resulting APIs (for C++ in particular) are far from natural code, due in part to the need to have predictable ABIs. I was recently mind-blown by the work the libreoffice people have been doing to refactor their code, particularly by page 16 in https://archive.fosdem.org/2013/schedule/event/challenges_libreoffice/attachments/slides/300/export/events/attachments/challenges_libreoffice/slides/300/2013_02_03_re_factoring.pdf Doing massive refactorings in our code is certainly possible, we just need to actually commit ourselves to doing those refactorings. I have a minor goal I'm working towards of removing most uses of NSPR from Gecko... Now think of all those NS_LITERAL_STRING() and other horrible boilerplate we have. ... and my next target is s/PRUnichar/char16_t/, the last step of which basically amounts to killing NS_LITERAL_STRING. :-) -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Standard C/C++ and Mozilla
On Wed, Jul 31, 2013 at 3:22 PM, Joshua Cranmer pidgeo...@gmail.com wrote: We also have needs like sizeOfIncludingThis/sizeOfExcludingThis that can't be as easily satisfied with STL code. This is, unsurprisingly, a requirement that's close to my heart. We actually have a few instances of std:: classes already, which leads to ridiculousness like https://bugzilla.mozilla.org/show_bug.cgi?id=806514. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
