Re: Is there a particular reason --with-compat isn't enabled by default?
Interesting, that wasn't in the specific rules when I was hunting it down. Thanks for the pointer. In either case, it should probably be enabled globally, per my prior arguments (but with respect to downstream distros) On 05/16/2018 01:14 PM, Maxim Dounin wrote: > Hello! > > On Wed, May 16, 2018 at 12:45:11PM -0400, Thomas Ward wrote: > >> Maxim, >> >> >> On 05/16/2018 12:24 PM, Maxim Dounin wrote: >>> Hello! >>> >>> On Wed, May 16, 2018 at 12:03:51PM -0400, Thomas Ward wrote: >>> On 05/10/2018 08:47 AM, Maxim Dounin wrote: > More importantly, "--with-compat" provides compatibility between > builds with different configure options. > > Without the "--with-compat" argument, one have to use exactly the > same "./configure" line to build both nginx and a module. The > "--with-compat" option relaxes this restriction, and it is more or > less enough to build both nginx and a module with "--with-compat". > > This generally simplifies building dynamic modules, and also > allows one to load the same module into different nginx variants > built with different flags. Is there a reason, then, that `--with-compat` is not a default build option? Is there a particular specific reason it was *not* set as a default compile-time option? >>> The main reasons are that "--with-compat" is not needed when not >>> using dynamic modules built separately, and compiling nginx with >>> "--with-compat" results in slightly less optimal binaries (with >>> various otherwise unneeded placeholder fields added to >>> structures). >>> >>> We've considered making it the default when it was introduced, but >>> decided to keep it as an explicit option then. This might worth >>> reconsidering. >>> >> I agree it might be worth a reconsideration, in fact I am a supporter of >> making `--with-compat` a default option, not only to make my job as a >> downstream packager easy, but because this would also benefit the >> nginx.org open-source nginx repository as well. With the number of >> third-party dynamic modules increasing, and more users wanting to >> utilize them, it may be detrimental to leave it disabled. When the >> support was first released, and dynamic modules were 'new', it made >> sense to leave it disabled, but with more and more modules becoming >> dynamically-supported it could be detrimental to leave this 'default >> disabled'. >> >> With specific regard to the nginx.org repositories, at last I checked a >> couple weeks ago, they aren't currently built with `--with-compat` >> (unless I missed a recent change to the OSS packaging on hg.nginx.org), >> which means it's equally difficult for users to use the nginx >> repositories without compiling NGINX themselves with third-party >> modules. If with-compat becomes the default, then this makes things >> easier for users using the nginx.org repos as well as other downstream >> distributions' packages going forward. > Repositories as available from nginx.org are built with > "--with-compat" since it was introduced in 2016: > > http://hg.nginx.org/pkg-oss/file/tip/rpm/SPECS/Makefile#l256 > http://hg.nginx.org/pkg-oss/rev/fa656182ba2a#l3.220 > ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
Hello! On Wed, May 16, 2018 at 12:45:11PM -0400, Thomas Ward wrote: > Maxim, > > > On 05/16/2018 12:24 PM, Maxim Dounin wrote: > > Hello! > > > > On Wed, May 16, 2018 at 12:03:51PM -0400, Thomas Ward wrote: > > > >> On 05/10/2018 08:47 AM, Maxim Dounin wrote: > >>> More importantly, "--with-compat" provides compatibility between > >>> builds with different configure options. > >>> > >>> Without the "--with-compat" argument, one have to use exactly the > >>> same "./configure" line to build both nginx and a module. The > >>> "--with-compat" option relaxes this restriction, and it is more or > >>> less enough to build both nginx and a module with "--with-compat". > >>> > >>> This generally simplifies building dynamic modules, and also > >>> allows one to load the same module into different nginx variants > >>> built with different flags. > >> Is there a reason, then, that `--with-compat` is not a default build > >> option? Is there a particular specific reason it was *not* set as a > >> default compile-time option? > > The main reasons are that "--with-compat" is not needed when not > > using dynamic modules built separately, and compiling nginx with > > "--with-compat" results in slightly less optimal binaries (with > > various otherwise unneeded placeholder fields added to > > structures). > > > > We've considered making it the default when it was introduced, but > > decided to keep it as an explicit option then. This might worth > > reconsidering. > > > I agree it might be worth a reconsideration, in fact I am a supporter of > making `--with-compat` a default option, not only to make my job as a > downstream packager easy, but because this would also benefit the > nginx.org open-source nginx repository as well. With the number of > third-party dynamic modules increasing, and more users wanting to > utilize them, it may be detrimental to leave it disabled. When the > support was first released, and dynamic modules were 'new', it made > sense to leave it disabled, but with more and more modules becoming > dynamically-supported it could be detrimental to leave this 'default > disabled'. > > With specific regard to the nginx.org repositories, at last I checked a > couple weeks ago, they aren't currently built with `--with-compat` > (unless I missed a recent change to the OSS packaging on hg.nginx.org), > which means it's equally difficult for users to use the nginx > repositories without compiling NGINX themselves with third-party > modules. If with-compat becomes the default, then this makes things > easier for users using the nginx.org repos as well as other downstream > distributions' packages going forward. Repositories as available from nginx.org are built with "--with-compat" since it was introduced in 2016: http://hg.nginx.org/pkg-oss/file/tip/rpm/SPECS/Makefile#l256 http://hg.nginx.org/pkg-oss/rev/fa656182ba2a#l3.220 -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
Maxim, On 05/16/2018 12:24 PM, Maxim Dounin wrote: > Hello! > > On Wed, May 16, 2018 at 12:03:51PM -0400, Thomas Ward wrote: > >> On 05/10/2018 08:47 AM, Maxim Dounin wrote: >>> More importantly, "--with-compat" provides compatibility between >>> builds with different configure options. >>> >>> Without the "--with-compat" argument, one have to use exactly the >>> same "./configure" line to build both nginx and a module. The >>> "--with-compat" option relaxes this restriction, and it is more or >>> less enough to build both nginx and a module with "--with-compat". >>> >>> This generally simplifies building dynamic modules, and also >>> allows one to load the same module into different nginx variants >>> built with different flags. >> Is there a reason, then, that `--with-compat` is not a default build >> option? Is there a particular specific reason it was *not* set as a >> default compile-time option? > The main reasons are that "--with-compat" is not needed when not > using dynamic modules built separately, and compiling nginx with > "--with-compat" results in slightly less optimal binaries (with > various otherwise unneeded placeholder fields added to > structures). > > We've considered making it the default when it was introduced, but > decided to keep it as an explicit option then. This might worth > reconsidering. > I agree it might be worth a reconsideration, in fact I am a supporter of making `--with-compat` a default option, not only to make my job as a downstream packager easy, but because this would also benefit the nginx.org open-source nginx repository as well. With the number of third-party dynamic modules increasing, and more users wanting to utilize them, it may be detrimental to leave it disabled. When the support was first released, and dynamic modules were 'new', it made sense to leave it disabled, but with more and more modules becoming dynamically-supported it could be detrimental to leave this 'default disabled'. With specific regard to the nginx.org repositories, at last I checked a couple weeks ago, they aren't currently built with `--with-compat` (unless I missed a recent change to the OSS packaging on hg.nginx.org), which means it's equally difficult for users to use the nginx repositories without compiling NGINX themselves with third-party modules. If with-compat becomes the default, then this makes things easier for users using the nginx.org repos as well as other downstream distributions' packages going forward. Thomas ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
Hello! On Wed, May 16, 2018 at 12:03:51PM -0400, Thomas Ward wrote: > On 05/10/2018 08:47 AM, Maxim Dounin wrote: > > More importantly, "--with-compat" provides compatibility between > > builds with different configure options. > > > > Without the "--with-compat" argument, one have to use exactly the > > same "./configure" line to build both nginx and a module. The > > "--with-compat" option relaxes this restriction, and it is more or > > less enough to build both nginx and a module with "--with-compat". > > > > This generally simplifies building dynamic modules, and also > > allows one to load the same module into different nginx variants > > built with different flags. > > Is there a reason, then, that `--with-compat` is not a default build > option? Is there a particular specific reason it was *not* set as a > default compile-time option? The main reasons are that "--with-compat" is not needed when not using dynamic modules built separately, and compiling nginx with "--with-compat" results in slightly less optimal binaries (with various otherwise unneeded placeholder fields added to structures). We've considered making it the default when it was introduced, but decided to keep it as an explicit option then. This might worth reconsidering. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
Maxim, Thanks for your reply. On 05/10/2018 08:47 AM, Maxim Dounin wrote: > More importantly, "--with-compat" provides compatibility between > builds with different configure options. > > Without the "--with-compat" argument, one have to use exactly the > same "./configure" line to build both nginx and a module. The > "--with-compat" option relaxes this restriction, and it is more or > less enough to build both nginx and a module with "--with-compat". > > This generally simplifies building dynamic modules, and also > allows one to load the same module into different nginx variants > built with different flags. Is there a reason, then, that `--with-compat` is not a default build option? Is there a particular specific reason it was *not* set as a default compile-time option? Thomas ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
Hello! On Thu, May 10, 2018 at 01:17:41PM +0300, Ruslan Ermilov wrote: > On Wed, May 09, 2018 at 12:29:06PM -0400, Thomas Ward wrote: > > In regards to several off-lists inquiries downstream about people trying > > to add additional third party modules, I've gone and started seeking > > justification for enabling --with-compat. > > > > Downstream in Ubuntu, I'm getting pushback in that the question of "Why > > do we need to enable this, what does it add?". I'm trying to find that > > justification for it, and the best I can find is Maxim's statements on a > > 2016 email/forum thread about how it actually makes dynamic module > > support truly work (in a nutshell). [1] > > > > Further, there's pushback about "Will package security updates and > > patches change the module ABI on security fixes or bug fixes?". I don't > > have a clear answer on this, and I had this question back when dynamic > > module support was introduced, but never got a clear answer on this > > point. It does beg consideration with regards to dynamic module support > > whether a simple patch applied to the same exact NGINX version will > > break ABI. The way we handle security patches and such downstream is we > > apply patches to the existing NGINX version via `quilt`, which applies > > the patch at build time. Whether this makes an ABI change or not I > > couldn't say, so I'm hunting a response from you, the devs, to give me a > > clear answer on this. > > > > So, for those who didn't read everything there's two questions here: > > > > (A) Other than making dynamic module support "work better", what does > > --with-compat actually do behind the scenes (In a nutshell)? > > It enables some macros and alters some structures in a way that's > compatible with NGINX Plus, built with the same option. Practically > this means that checksums of module loadable objects will be identical > between when using F/OSS sources and when using NGINX Plus sources. > Searching for "NGX_COMPAT" throughout the F/OSS source code will > give enough details. While compatibily with nginx-plus is indeed provided by --with-compat as well, I doubt that this aspect is interesting to Thomas. More importantly, "--with-compat" provides compatibility between builds with different configure options. Without the "--with-compat" argument, one have to use exactly the same "./configure" line to build both nginx and a module. The "--with-compat" option relaxes this restriction, and it is more or less enough to build both nginx and a module with "--with-compat". This generally simplifies building dynamic modules, and also allows one to load the same module into different nginx variants built with different flags. > > (B) Will a simple patch that patches security issues or adds fixes to > > something later on but doesn't change the core NGINX version numbering > > change the module ABI in such a way that it'll break modules built > > against nginx without that patch (assuming that --with-compat was added, > > since it's apparently needed to make dynamic modules 'actually work') > > If a patch is simple, this is highly unlikely. For a patch to > break the ABI, at least some externally visible structures should > be changed in some backwards incompatible ways. While it is unlikely, it is possible. Care should be taken when applying patches without changing the version number. Whether "--with-compat" is specified or not is irrelevant. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Is there a particular reason --with-compat isn't enabled by default?
On Wed, May 09, 2018 at 12:29:06PM -0400, Thomas Ward wrote: > In regards to several off-lists inquiries downstream about people trying > to add additional third party modules, I've gone and started seeking > justification for enabling --with-compat. > > Downstream in Ubuntu, I'm getting pushback in that the question of "Why > do we need to enable this, what does it add?". I'm trying to find that > justification for it, and the best I can find is Maxim's statements on a > 2016 email/forum thread about how it actually makes dynamic module > support truly work (in a nutshell). [1] > > Further, there's pushback about "Will package security updates and > patches change the module ABI on security fixes or bug fixes?". I don't > have a clear answer on this, and I had this question back when dynamic > module support was introduced, but never got a clear answer on this > point. It does beg consideration with regards to dynamic module support > whether a simple patch applied to the same exact NGINX version will > break ABI. The way we handle security patches and such downstream is we > apply patches to the existing NGINX version via `quilt`, which applies > the patch at build time. Whether this makes an ABI change or not I > couldn't say, so I'm hunting a response from you, the devs, to give me a > clear answer on this. > > So, for those who didn't read everything there's two questions here: > > (A) Other than making dynamic module support "work better", what does > --with-compat actually do behind the scenes (In a nutshell)? It enables some macros and alters some structures in a way that's compatible with NGINX Plus, built with the same option. Practically this means that checksums of module loadable objects will be identical between when using F/OSS sources and when using NGINX Plus sources. Searching for "NGX_COMPAT" throughout the F/OSS source code will give enough details. > (B) Will a simple patch that patches security issues or adds fixes to > something later on but doesn't change the core NGINX version numbering > change the module ABI in such a way that it'll break modules built > against nginx without that patch (assuming that --with-compat was added, > since it's apparently needed to make dynamic modules 'actually work') If a patch is simple, this is highly unlikely. For a patch to break the ABI, at least some externally visible structures should be changed in some backwards incompatible ways. ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Is there a particular reason --with-compat isn't enabled by default?
In regards to several off-lists inquiries downstream about people trying to add additional third party modules, I've gone and started seeking justification for enabling --with-compat. Downstream in Ubuntu, I'm getting pushback in that the question of "Why do we need to enable this, what does it add?". I'm trying to find that justification for it, and the best I can find is Maxim's statements on a 2016 email/forum thread about how it actually makes dynamic module support truly work (in a nutshell). [1] Further, there's pushback about "Will package security updates and patches change the module ABI on security fixes or bug fixes?". I don't have a clear answer on this, and I had this question back when dynamic module support was introduced, but never got a clear answer on this point. It does beg consideration with regards to dynamic module support whether a simple patch applied to the same exact NGINX version will break ABI. The way we handle security patches and such downstream is we apply patches to the existing NGINX version via `quilt`, which applies the patch at build time. Whether this makes an ABI change or not I couldn't say, so I'm hunting a response from you, the devs, to give me a clear answer on this. So, for those who didn't read everything there's two questions here: (A) Other than making dynamic module support "work better", what does --with-compat actually do behind the scenes (In a nutshell)? (B) Will a simple patch that patches security issues or adds fixes to something later on but doesn't change the core NGINX version numbering change the module ABI in such a way that it'll break modules built against nginx without that patch (assuming that --with-compat was added, since it's apparently needed to make dynamic modules 'actually work') Thomas ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
